Skip to content

Commit 5e8819c

Browse files
dgrove-ossdgrove-oss
committed
Upgrade to AppWrapper v1beta2
1. Upgrade AppWrapper from v1beta1 to v1beta2 2. Remove MCADv1 controller 3. Add AppWrapper controller 4. Remove InstaScale controller 5. Update build, test, and CI accordingly
1 parent 7419586 commit 5e8819c

25 files changed

+479
-1383
lines changed

‎.github/workflows/project-codeflare-release.yml

+17-52
Original file line numberDiff line numberDiff line change
@@ -10,15 +10,12 @@ on:
1010
replaces:
1111
description: 'The previous operator semantic version that this release replaces (for example: v0.0.0)'
1212
required: true
13-
mcad-version:
14-
description: 'Version of multi-cluster-app-dispatcher to be released (for example: v0.0.0)'
13+
appwrapper-version:
14+
description: 'Version of appwrapper to be released (for example: v0.0.0)'
1515
required: true
1616
codeflare-sdk-version:
1717
description: 'Version of CodeFlare-SDK to be released (for example: v0.0.0)'
1818
required: true
19-
instascale-version:
20-
description: 'Version of InstaScale to be released (for example: v0.0.0)'
21-
required: true
2219
kuberay-version:
2320
description: 'Tested version of KubeRay (for example: v0.0.0)'
2421
required: true
@@ -48,78 +45,46 @@ jobs:
4845
echo "Below are the release parameters set for the workflow:"
4946
echo "Operator Version: ${{ github.event.inputs.operator-version }}"
5047
echo "Replaces: ${{ github.event.inputs.replaces }}"
51-
echo "MCAD Version: ${{ github.event.inputs.mcad-version }}"
48+
echo "AppWrapper Version: ${{ github.event.inputs.appwrapper-version }}"
5249
echo "CodeFlare SDK Version: ${{ github.event.inputs.codeflare-sdk-version }}"
53-
echo "InstaScale Version: ${{ github.event.inputs.instascale-version }}"
5450
echo "Tested KubeRay Version: ${{ github.event.inputs.kuberay-version }}"
5551
echo "Is Stable: ${{ github.event.inputs.is-stable }}"
5652
echo "CodeFlare Repository Organization: ${{ github.event.inputs.codeflare-repository-organization }}"
5753
echo "Quay Organization: ${{ github.event.inputs.quay-organization }}"
5854
echo "Community Operators Prod Organization: ${{ github.event.inputs.community-operators-prod-organization }}"
5955
60-
release-mcad:
56+
release-appwrapper:
6157
needs: check-kuberay-version
6258
runs-on: ubuntu-latest
6359

6460
steps:
65-
- name: Check if MCAD release does exist
61+
- name: Check if AppWrapper release does exist
6662
run: |
67-
status_code=$(curl -s -o /dev/null -w "%{http_code}" https://github.com/project-codeflare/multi-cluster-app-dispatcher/releases/tag/${{ github.event.inputs.mcad-version }})
63+
status_code=$(curl -s -o /dev/null -w "%{http_code}" https://github.com/project-codeflare/appwrapper/releases/tag/${{ github.event.inputs.appwrapper-version }})
6864
if [[ "$status_code" == "200" ]]; then
69-
echo "MCAD release with version ${{ github.event.inputs.mcad-version }} already exist. Will not create MCAD release."
65+
echo "AppWrapper release with version ${{ github.event.inputs.appwrapper-version }} already exist. Will not create AppWrapper release."
7066
fi
71-
echo "MCAD_RELEASE_STATUS_CODE=$status_code" >> $GITHUB_ENV
67+
echo "APPWRAPPER_RELEASE_STATUS_CODE=$status_code" >> $GITHUB_ENV
7268
73-
- name: Release MCAD
69+
- name: Release AppWrapper
7470
run: |
75-
gh workflow run mcad-release.yml --repo ${{ github.event.inputs.codeflare-repository-organization }}/multi-cluster-app-dispatcher --ref go-1.20 --field tag=${{ github.event.inputs.mcad-version }}
71+
gh workflow run release.yml --repo ${{ github.event.inputs.codeflare-repository-organization }}/appwrapper --field tag=${{ github.event.inputs.appwrapper-version }}
7672
env:
7773
GITHUB_TOKEN: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}
7874
shell: bash
79-
if: ${{ env.MCAD_RELEASE_STATUS_CODE != '200' }}
75+
if: ${{ env.APPWRAPPER_RELEASE_STATUS_CODE != '200' }}
8076

81-
- name: Wait for MCAD run to finish
77+
- name: Wait for AppWrapper run to finish
8278
run: |
8379
# wait for a while for Run to be started
8480
sleep 5
85-
run_id=$(gh run list --workflow mcad-release.yml --repo ${{ github.event.inputs.codeflare-repository-organization }}/multi-cluster-app-dispatcher --limit 1 --json databaseId --jq .[].databaseId)
86-
gh run watch ${run_id} --repo ${{ github.event.inputs.codeflare-repository-organization }}/multi-cluster-app-dispatcher --interval 10 --exit-status
81+
run_id=$(gh run list --workflow release.yml --repo ${{ github.event.inputs.codeflare-repository-organization }}/appwrapper --limit 1 --json databaseId --jq .[].databaseId)
82+
gh run watch ${run_id} --repo ${{ github.event.inputs.codeflare-repository-organization }}/appwrapper --interval 10 --exit-status
8783
env:
8884
GITHUB_TOKEN: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}
8985
shell: bash
90-
if: ${{ env.MCAD_RELEASE_STATUS_CODE != '200' }}
91-
92-
release-instascale:
93-
needs: release-mcad
94-
runs-on: ubuntu-latest
95-
96-
steps:
97-
- name: Check if Instascale release does exist
98-
run: |
99-
status_code=$(curl -s -o /dev/null -w "%{http_code}" https://github.com/project-codeflare/instascale/releases/tag/${{ github.event.inputs.instascale-version }})
100-
if [[ "$status_code" == "200" ]]; then
101-
echo "Instascale release with version ${{ github.event.inputs.instascale-version }} already exist. Will not create Instascale release."
102-
fi
103-
echo "INSTASCALE_RELEASE_STATUS_CODE=$status_code" >> $GITHUB_ENV
86+
if: ${{ env.APPWRAPPER_RELEASE_STATUS_CODE != '200' }}
10487

105-
- name: Release InstaScale
106-
run: |
107-
gh workflow run instascale-release.yml --repo ${{ github.event.inputs.codeflare-repository-organization }}/instascale --ref ${{ github.ref }} --field tag=${{ github.event.inputs.instascale-version }} --field mcad-version=${{ github.event.inputs.mcad-version }}
108-
env:
109-
GITHUB_TOKEN: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}
110-
shell: bash
111-
if: ${{ env.INSTASCALE_RELEASE_STATUS_CODE != '200' }}
112-
113-
- name: Wait for InstaScale run to finish
114-
run: |
115-
# wait for a while for Run to be started
116-
sleep 5
117-
run_id=$(gh run list --workflow instascale-release.yml --repo ${{ github.event.inputs.codeflare-repository-organization }}/instascale --limit 1 --json databaseId --jq .[].databaseId)
118-
gh run watch ${run_id} --repo ${{ github.event.inputs.codeflare-repository-organization }}/instascale --interval 10 --exit-status
119-
env:
120-
GITHUB_TOKEN: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}
121-
shell: bash
122-
if: ${{ env.INSTASCALE_RELEASE_STATUS_CODE != '200' }}
12388

12489
release-codeflare-sdk:
12590
needs: check-kuberay-version
@@ -156,13 +121,13 @@ jobs:
156121
if: ${{ env.SDK_RELEASE_STATUS_CODE != '200' }}
157122

158123
release-codeflare-operator:
159-
needs: [release-mcad, release-instascale, release-codeflare-sdk]
124+
needs: [release-appwrapper, release-codeflare-sdk]
160125
runs-on: ubuntu-latest
161126

162127
steps:
163128
- name: Release CodeFlare operator
164129
run: |
165-
gh workflow run tag-and-build.yml --repo ${{ github.event.inputs.codeflare-repository-organization }}/codeflare-operator --ref ${{ github.ref }} --field is-stable=${{ github.event.inputs.is-stable }} --field version=${{ github.event.inputs.operator-version }} --field replaces=${{ github.event.inputs.replaces }} --field mcad-version=${{ github.event.inputs.mcad-version }} --field codeflare-sdk-version=${{ github.event.inputs.codeflare-sdk-version }} --field instascale-version=${{ github.event.inputs.instascale-version }} --field kuberay-version=${{ github.event.inputs.kuberay-version }} --field quay-organization=${{ github.event.inputs.quay-organization }} --field community-operators-prod-fork-organization=${{ github.event.inputs.codeflare-repository-organization }} --field community-operators-prod-organization=${{ github.event.inputs.community-operators-prod-organization }}
130+
gh workflow run tag-and-build.yml --repo ${{ github.event.inputs.codeflare-repository-organization }}/codeflare-operator --ref ${{ github.ref }} --field is-stable=${{ github.event.inputs.is-stable }} --field version=${{ github.event.inputs.operator-version }} --field replaces=${{ github.event.inputs.replaces }} --field appwrapper-version=${{ github.event.inputs.appwrapper-version }} --field codeflare-sdk-version=${{ github.event.inputs.codeflare-sdk-version }} --field kuberay-version=${{ github.event.inputs.kuberay-version }} --field quay-organization=${{ github.event.inputs.quay-organization }} --field community-operators-prod-fork-organization=${{ github.event.inputs.codeflare-repository-organization }} --field community-operators-prod-organization=${{ github.event.inputs.community-operators-prod-organization }}
166131
env:
167132
GITHUB_TOKEN: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}
168133
shell: bash

‎.github/workflows/tag-and-build.yml

+7-14
Original file line numberDiff line numberDiff line change
@@ -12,18 +12,14 @@ on:
1212
description: 'The previous semantic version that this tag replaces.'
1313
required: true
1414
default: 'v0.0.0-dev'
15-
mcad-version:
16-
description: 'Published version of multi-cluster-app-dispatcher'
15+
appwrapper-version:
16+
description: 'Published version of appwrapper'
1717
required: true
1818
default: 'v0.0.0-dev'
1919
codeflare-sdk-version:
2020
description: 'Published version of CodeFlare-SDK'
2121
required: true
2222
default: 'v0.0.0-dev'
23-
instascale-version:
24-
description: 'Published version of InstaScale'
25-
required: true
26-
default: 'v0.0.0-dev'
2723
kuberay-version:
2824
description: 'Tested version of KubeRay'
2925
required: true
@@ -90,16 +86,14 @@ jobs:
9086
- name: Adjust Compatibility Matrix in readme
9187
run: |
9288
sed -i -E "s|(.*CodeFlare Operator.*\[).*(\].*releases/tag/).*(\).*)|\1${{ github.event.inputs.version }}\2${{ github.event.inputs.version }}\3|" README.md
93-
sed -i -E "s|(.*Multi-Cluster App Dispatcher.*\[).*(\].*releases/tag/).*(\).*)|\1${{ github.event.inputs.mcad-version }}\2${{ github.event.inputs.mcad-version }}\3|" README.md
89+
sed -i -E "s|(.*AppWrapper.*\[).*(\].*releases/tag/).*(\).*)|\1${{ github.event.inputs.appwrapper-version }}\2${{ github.event.inputs.appwrapper-version }}\3|" README.md
9490
sed -i -E "s|(.*CodeFlare-SDK.*\[).*(\].*releases/tag/).*(\).*)|\1${{ github.event.inputs.codeflare-sdk-version }}\2${{ github.event.inputs.codeflare-sdk-version }}\3|" README.md
95-
sed -i -E "s|(.*InstaScale.*\[).*(\].*releases/tag/).*(\).*)|\1${{ github.event.inputs.instascale-version }}\2${{ github.event.inputs.instascale-version }}\3|" README.md
9691
sed -i -E "s|(.*KubeRay.*\[).*(\].*releases/tag/).*(\).*)|\1${{ github.event.inputs.kuberay-version }}\2${{ github.event.inputs.kuberay-version }}\3|" README.md
9792
98-
- name: Adjust MCAD, SDK and InstaScale dependencies in the code
93+
- name: Adjust AppWrapper and SDK dependencies in the code
9994
run: |
100-
sed -i -E "s/(.*MCAD_VERSION \?= ).*/\1${{ github.event.inputs.mcad-version }}/" Makefile
101-
sed -i -E "s/(.*MCAD_REF \?= ).*/\1release-\${MCAD_VERSION}/" Makefile
102-
sed -i -E "s/(.*INSTASCALE_VERSION \?= ).*/\1${{ github.event.inputs.instascale-version }}/" Makefile
95+
sed -i -E "s/(.*APPWRAPPER_VERSION \?= ).*/\1${{ github.event.inputs.appwrapper-version }}/" Makefile
96+
sed -i -E "s/(.*APPWRAPPER_REF \?= ).*/\1release-\${APPWRAPPER_VERSION}/" Makefile
10397
sed -i -E "s/(.*KUBERAY_VERSION \?= ).*/\1${{ github.event.inputs.kuberay-version }}/" Makefile
10498
10599
- name: Update image version in params.env
@@ -192,8 +186,7 @@ jobs:
192186
env:
193187
VERSION: ${{ github.event.inputs.version }}
194188
PREVIOUS_VERSION: ${{ github.event.inputs.replaces }}
195-
INSTASCALE_VERSION: ${{ github.event.inputs.instascale-version }}
196-
MCAD_VERSION: ${{ github.event.inputs.mcad-version }}
189+
APPWRAPPER_VERSION: ${{ github.event.inputs.appwrapper-version }}
197190
GH_TOKEN: ${{ secrets.CODEFLARE_MACHINE_ACCOUNT_TOKEN }}
198191
OPERATORS_REPO_FORK_ORG: ${{ github.event.inputs.community-operators-prod-fork-organization }}
199192
OPERATORS_REPO_ORG: ${{ github.event.inputs.community-operators-prod-organization }}

‎CONTRIBUTING.md

+7-7
Original file line numberDiff line numberDiff line change
@@ -14,14 +14,14 @@ The following should be installed in your working environment:
1414
## Basic Overview
1515
The main entrypoint for the operator is `main.go`
1616

17-
The MCAD and InstaScale custom resources are defined under the `api` dir:
18-
- See `mcad_types.go` and `instascale_types.go`
17+
The AppWrapper custom resources are defined under the `api` dir:
18+
- See `appwraper_types.go`
1919

20-
The MCAD and InstaScale resource templates can be found under `config/internal`:
21-
- Sorted under `mcad` and `instascale` subdirs
20+
The AppWrapper resource templates can be found under `config/internal`:
21+
- Sorted under `appwrapper` subdirs
2222

23-
The code for MCAD/InstaScale resource reconcilliation can be found in the `controllers` dir:
24-
- See `mcad_controller.go` and `instascale_controller.go`
23+
The code for AppWrapper resource reconcilliation can be found in the `controllers` dir:
24+
- See `appwrapper_controller.go`
2525

2626
## Building and Deployment
2727
If changes are made in the `api` dir, run: `make manifests`
@@ -53,7 +53,7 @@ The CodeFlare Operator currently has unit tests and pre-commit checks
5353
- Note that both are required for CI to pass on pull requests
5454

5555
To write and inspect unit tests:
56-
- MCAD and InstaScale unit tests under `mcad_controller_test.go` and `instascale_controller_test.go` in the `controllers` dir
56+
- AppWrapper unit tests under `appwrapper_controller_test.go` in the `controllers` dir
5757
- Unit test functions are defined in `suite_test.go` (with utils in `util/util.go`) in the `controllers dir`
5858
- Test cases defined under `controllers/testdata`
5959

‎Makefile

+9-16
Original file line numberDiff line numberDiff line change
@@ -11,15 +11,11 @@ PREVIOUS_VERSION ?= v0.0.0-dev
1111
VERSION ?= v0.0.0-dev
1212
BUNDLE_VERSION ?= $(VERSION:v%=%)
1313

14-
# INSTASCALE_VERSION defines the default version of the InstaScale controller
15-
INSTASCALE_VERSION ?= v0.4.0
16-
INSTASCALE_REPO ?= github.com/project-codeflare/instascale
17-
18-
# MCAD_VERSION defines the default version of the MCAD controller
19-
MCAD_VERSION ?= v1.40.0
20-
MCAD_REPO ?= github.com/project-codeflare/multi-cluster-app-dispatcher
21-
# Upstream MCAD is currently only creating release tags of the form `vX.Y.Z` (i.e the version)
22-
MCAD_CRD ?= ${MCAD_REPO}/config/crd?ref=${MCAD_VERSION}
14+
# APPWRAPPER_VERSION defines the default version of the AppWrapper controller
15+
APPWRAPPER_VERSION ?= v0.6.2
16+
APPWRAPPER_REPO ?= github.com/project-codeflare/appwrapper
17+
# Upstream AppWrapper is currently only creating release tags of the form `vX.Y.Z` (i.e the version)
18+
APPWRAPPER_CRD ?= ${APPWRAPPER_REPO}/config/crd?ref=${APPWRAPPER_VERSION}
2319

2420
# KUBERAY_VERSION defines the default version of the KubeRay operator (used for testing)
2521
KUBERAY_VERSION ?= v1.0.0
@@ -139,8 +135,8 @@ help: ## Display this help.
139135
.PHONY: manifests
140136
manifests: controller-gen kustomize install-yq ## Generate RBAC objects.
141137
$(CONTROLLER_GEN) rbac:roleName=manager-role webhook paths="./..."
142-
$(SED) -i -E "s|(- )\${MCAD_REPO}.*|\1\${MCAD_CRD}|" config/crd/mcad/kustomization.yaml
143-
$(KUSTOMIZE) build config/crd/mcad | $(YQ) -s '"crd-" + .spec.names.singular' --no-doc
138+
$(SED) -i -E "s|(- )\${APPWRAPPER_REPO}.*|\1\${APPWRAPPER_CRD}|" config/crd/appwrapper/kustomization.yaml
139+
$(KUSTOMIZE) build config/crd/appwrapper | $(YQ) -s '"crd-" + .spec.names.singular' --no-doc
144140
mv crd-*.yml config/crd
145141

146142
.PHONY: fmt
@@ -156,8 +152,6 @@ vet: ## Run go vet against code.
156152

157153
.PHONY: modules
158154
modules: ## Update Go dependencies.
159-
go get $(MCAD_REPO)@$(MCAD_VERSION)
160-
go get $(INSTASCALE_REPO)@$(INSTASCALE_VERSION)
161155
go get github.com/ray-project/kuberay/ray-operator@$(KUBERAY_VERSION)
162156
go mod tidy
163157

@@ -166,8 +160,7 @@ build: fmt vet ## Build manager binary.
166160
go build \
167161
-ldflags " \
168162
-X 'main.OperatorVersion=$(BUILD_VERSION)' \
169-
-X 'main.McadVersion=$(MCAD_VERSION)' \
170-
-X 'main.InstaScaleVersion=$(INSTASCALE_VERSION)' \
163+
-X 'main.AppWrapperVersion=$(APPWRAPPER_VERSION)' \
171164
-X 'main.BuildDate=$(BUILD_DATE)' \
172165
" \
173166
-o bin/manager main.go
@@ -208,7 +201,7 @@ deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in
208201

209202
.PHONY: undeploy
210203
undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
211-
$(SED) -i -E "s|(- )\${MCAD_REPO}.*|\1\${MCAD_CRD}|" config/crd/mcad/kustomization.yaml
204+
$(SED) -i -E "s|(- )\${APPWRAPPER_REPO}.*|\1\${APPWRAPPER_CRD}|" config/appwrapper/appwrapper/kustomization.yaml
212205
$(KUSTOMIZE) build config/${ENV} | kubectl delete --ignore-not-found=$(ignore-not-found) -f -
213206
git restore config/*
214207

‎README.md

+9-11
Original file line numberDiff line numberDiff line change
@@ -1,18 +1,17 @@
11
# codeflare-operator
22

3-
Operator for installation and lifecycle management of CodeFlare distributed workload stack, starting with MCAD and InstaScale
3+
Operator for installation and lifecycle management of CodeFlare distributed workload stack, starting with AppWrapper
44

55
<!-- Don't delete these comments, they are used to generate Compatibility Matrix table for release automation -->
66
<!-- Compatibility Matrix start -->
77
CodeFlare Stack Compatibility Matrix
88

9-
| Component | Version |
10-
|------------------------------|---------------------------------------------------------------------------------------------------|
11-
| CodeFlare Operator | [v1.2.0](https://github.com/project-codeflare/codeflare-operator/releases/tag/v1.2.0) |
12-
| Multi-Cluster App Dispatcher | [v1.40.0](https://github.com/project-codeflare/multi-cluster-app-dispatcher/releases/tag/v1.40.0) |
13-
| CodeFlare-SDK | [v0.14.0](https://github.com/project-codeflare/codeflare-sdk/releases/tag/v0.14.0) |
14-
| InstaScale | [v0.4.0](https://github.com/project-codeflare/instascale/releases/tag/v0.4.0) |
15-
| KubeRay | [v1.0.0](https://github.com/opendatahub-io/kuberay/releases/tag/v1.0.0) |
9+
| Component | Version |
10+
|--------------------------|---------------------------------------------------------------------------------------------------|
11+
| CodeFlare Operator | [v1.2.0](https://github.com/project-codeflare/codeflare-operator/releases/tag/v1.2.0) |
12+
| AppWrapper | [v0.6.2](https://github.com/project-codeflare/appwrapper/releases/tag/v0.6.2) |
13+
| CodeFlare-SDK | [v0.14.0](https://github.com/project-codeflare/codeflare-sdk/releases/tag/v0.14.0) |
14+
| KubeRay | [v1.0.0](https://github.com/opendatahub-io/kuberay/releases/tag/v1.0.0) |
1615
<!-- Compatibility Matrix end -->
1716

1817
## Development
@@ -90,7 +89,7 @@ The e2e tests can be executed locally by running the following commands:
9089
9190
To properly run e2e tests on disconnected cluster user has to provide additional environment variables to properly configure testing environment:
9291
93-
- `CODEFLARE_TEST_PYTORCH_IMAGE` - image tag for image used to run training job using MCAD
92+
- `CODEFLARE_TEST_PYTORCH_IMAGE` - image tag for image used to run training job using AppWrapper
9493
- `CODEFLARE_TEST_RAY_IMAGE` - image tag for Ray cluster image
9594
- `MNIST_DATASET_URL` - URL where MNIST dataset is available
9695
- `PIP_INDEX_URL` - URL where PyPI server with needed dependencies is running
@@ -118,8 +117,7 @@ For ODH tests additional environment variables are needed:
118117
There may be instances in which a new CodeFlare stack release requires releases of only a subset of the stack components. Examples could be hotfixes for a specific component. In these instances:
119118

120119
1. Build updated components as needed:
121-
- Build and release [MCAD](https://github.com/project-codeflare/multi-cluster-app-dispatcher)
122-
- Build and release [InstaScale](https://github.com/project-codeflare/instascale)
120+
- Build and release [AppWrapper](https://github.com/project-codeflare/appwrapper)
123121
- Build and release [CodeFlare-SDK](https://github.com/project-codeflare/codeflare-sdk)
124122

125123
2. Invoke [tag-and-build.yml](https://github.com/project-codeflare/codeflare-operator/actions/workflows/tag-and-build.yml) GitHub action, this action will create a repository tag, build and push operator image.

‎config/crd/appwrapper/kustomization.yaml

+4
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
apiVersion: kustomize.config.k8s.io/v1beta1
2+
kind: Kustomization
3+
resources:
4+
- github.com/project-codeflare/appwrapper/config/crd?ref=v0.6.2

0 commit comments

Comments
 (0)