Skip to content

Commit ab2443f

Browse files
blaggacaoblaggacao
committed
docs: update readme with better examples
1 parent c2ca7a2 commit ab2443f

File tree

2 files changed

+189
-101
lines changed

2 files changed

+189
-101
lines changed

‎README.md

Lines changed: 188 additions & 100 deletions
Original file line numberDiff line numberDiff line change
@@ -1,155 +1,243 @@
1-
# Usage
1+
# Standard Action
2+
3+
_for [Standard] & [Paisano]_.
4+
5+
[Paisano]: https://github.com/paisano-nix
6+
[Standard]: https://github.com/divnix/std
7+
8+
Don't waste any time on extra work. Use Standard Action to automatically
9+
detect CI targets that need re-doing; implemented on top of familiar GH Actions.
10+
11+
## Features
12+
13+
- Evaluate once and distribute final build instructions to workers
14+
- Once configured, `discovery` picks up new targets automatically
15+
- Optional `proviso` script can detect if work needs to be done
16+
17+
> **Note on `proviso`**: one example is the oci block type which
18+
> [checks if the image] is already in the registry and only schedules
19+
> a build if its missing. If `proviso` queries private remote state
20+
> then the `discovery` environment must provide all authentication
21+
> prior to running the discovery step.
22+
23+
[checks if the image]: https://github.com/divnix/std/blob/main/src/std/fwlib/blockTypes/containers-proviso.sh
24+
25+
## Usage
26+
27+
**Minimumn nix version `v2.16.1`**
228

329
- Works with https://github.com/divnix/std
430
- Since GitHub CI doesn't support yaml anchors, explode your file with: `yq '. | explode(.)' ci.raw.yaml > ci.yaml`
5-
- To set up AWS Credentials for an S3 Cache, find details [here](https://github.com/aws-actions/configure-aws-credentials)
6-
- **Warning:** This is still under active development and testing. You're likely better off waiting a little while, still.
7-
- But it's already being used with success :smile:
31+
32+
### Standalone
33+
34+
```nix
35+
{
36+
/* ... */
37+
outputs = {std, ...}@inputs: std.growOn {
38+
/* ... */
39+
cellBlocks = with std.blockTypes; [
40+
(installables "packages" {ci.build = true;})
41+
(containers "oci-images" {ci.publish = true;})
42+
(kubectl "deployments" {ci.apply = true;})
43+
];
44+
/* ... */
45+
};
46+
}
47+
```
48+
49+
<details><summary><h4>GH Action file</h4></summary>
850

951
```yaml
10-
# .github/workflows/ci.yml
11-
name: Standard CI
52+
# yq '. | explode(.)' this.yml > .github/workflows/std.yml
53+
name: CI/CD
1254

1355
on:
56+
pull_request:
57+
branches:
58+
- main
1459
push:
1560
branches:
1661
- main
17-
workflow_dispatch:
1862

1963
permissions:
64+
id-token: write
2065
contents: read
2166

67+
concurrency:
68+
group: std-${{ github.workflow }}-${{ github.ref }}
69+
cancel-in-progress: true
70+
2271
jobs:
2372
discover:
2473
outputs:
2574
hits: ${{ steps.discovery.outputs.hits }}
26-
nix_conf: ${{ steps.discovery.outputs.nix_conf }}
27-
2875
runs-on: ubuntu-latest
29-
concurrency:
30-
group: ${{ github.workflow }}
3176
steps:
32-
- name: Standard Discovery
33-
uses: divnix/std-action/discover@main
34-
id: discovery
77+
# Important: use this as it also detects flake configuration
78+
- uses: blaggacao/nix-quick-install-action@detect-nix-flakes-config
79+
# if you want to use nixbuild
80+
- uses: nixbuild/nixbuild-action@v17
3581
with:
36-
github_pat: ${{ secrets.HUB_PAT }}
82+
nixbuild_ssh_key: ${{ secrets.SSH_PRIVATE_KEY }}
83+
generate_summary_for: job
84+
# significantly speeds up things in small projects
85+
- uses: DeterminateSystems/magic-nix-cache-action@main
86+
- uses: divnix/std-action/discover@main
87+
id: discovery
3788

38-
build-packages: &run-job
89+
build: &job
3990
needs: discover
91+
name: ${{ matrix.target.jobName }}
92+
runs-on: ubuntu-latest
93+
if: fromJSON(needs.discover.outputs.hits).packages.build != '{}'
4094
strategy:
4195
matrix:
4296
target: ${{ fromJSON(needs.discover.outputs.hits).packages.build }}
43-
name: ${{ matrix.target.cell }} - ${{ matrix.target.name }}
44-
runs-on: ubuntu-latest
4597
steps:
46-
- name: Configure AWS Credentials
47-
uses: aws-actions/configure-aws-credentials@v1-node16
98+
# Important: use this as it also detects flake configuration
99+
- uses: blaggacao/nix-quick-install-action@detect-nix-flakes-config
100+
# if you want to use nixbuild
101+
- uses: nixbuild/nixbuild-action@v17
48102
with:
49-
role-to-assume: arn:aws:iam::123456789100:role/my-github-actions-role
50-
aws-region: us-east-2
103+
nixbuild_ssh_key: ${{ secrets.SSH_PRIVATE_KEY }}
104+
generate_summary_for: job
105+
- uses: DeterminateSystems/magic-nix-cache-action@main
51106
- uses: divnix/std-action/run@main
52-
with:
53-
extra_nix_config: |
54-
${{ needs.discover.outputs.nix_conf }}
55-
json: ${{ toJSON(matrix.target) }}
56-
# optional:
57-
github_pat: ${{ secrets.HUB_PAT }}
58-
nix_key: ${{ secrets.NIX_SECRET_KEY }}
59-
nix_ssh_key: ${{ secrets.NIXBUILD_SSH }}
60-
cache: s3://nix?endpoint=sfo3.digitaloceanspaces.com
61-
builder: ssh-ng://eu.nixbuild.net
62-
ssh_known_hosts: "eu.nixbuild.net ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPIQCZc54poJ8vqawd8TraNryQeJnvH1eLpIDgbiqymM"
63-
64-
build-devshells:
65-
<<: *run-job
107+
108+
images:
109+
<<: *job
110+
needs: [discover, build]
111+
if: fromJSON(needs.discover.outputs.hits).oci-images.publish != '{}'
66112
strategy:
67113
matrix:
68-
target: ${{ fromJSON(needs.discover.outputs.hits).devshells.build }}
69-
70-
publish-containers:
71-
<<: *run-job
114+
target: ${{ fromJSON(needs.discover.outputs.hits).oci-images.publish }}
115+
116+
deploy:
117+
<<: *job
118+
needs: [discover, images]
119+
environment:
120+
name: development
121+
url: https://my.dev.example.com
122+
if: fromJSON(needs.discover.outputs.hits).deployments.apply != '{}'
72123
strategy:
73124
matrix:
74-
target: ${{ fromJSON(needs.discover.outputs.hits).containers.publish }}
125+
target: ${{ fromJSON(needs.discover.outputs.hits).deployments.apply }}
75126
```
76127
77-
## Notes & Explanation
128+
</details>
78129
79-
### Notes on the Build Matrix
130+
### Persistent Discovery Host
80131
81-
Hits from the discovery phase are namespaced by Block and Action.
132+
#### Requirements
82133
83-
That means:
134+
- `nix` >= v2.16.1
135+
- `zstd`
136+
- (gnu) `parallel`
137+
- `jq`
138+
- `base64`
139+
- `bash` > v5
84140

85-
- In: `target: ${{ fromJSON(needs.discover.outputs.hits).packages.build }}`
86-
- `packages` is the name of a Standard Block
87-
- `build` is the name of an Action of that Block
141+
The persistent host must also implement the `nixConfig` detection capabilities
142+
implemented by [this script][script].
88143

89-
This example would be defined in `flake.nix` as such
144+
[script]: https://github.com/nixbuild/nix-quick-install-action/blob/5752d21669438be20da4de77327ae963e98c82a3/read-nix-config-from-flake.sh
90145

91146
```nix
92147
{
93148
/* ... */
94149
outputs = {std, ...}@inputs: std.growOn {
95150
/* ... */
96151
cellBlocks = with std.blockTypes; [
97-
(installables "packages" {ci.build = true;})
98-
(containers "containers" {ci.publish = true;})
152+
(devshells "envs" {ci.build = true;})
153+
(containers "oci-images" {ci.publish = true;})
99154
];
100155
/* ... */
101156
};
102157
}
103158
```
104159

105-
An example schema of the json returned by the dicovery phase:
160+
<details><summary><h4>GH Action file</h4></summary>
106161

107-
```json
108-
{
109-
"containers": {
110-
"publish": [
111-
{
112-
"action": "publish",
113-
"actionDrv": "/nix/store/6b0i2ww5drcdfa6hgxijx39zbcq57rwl-publish.drv",
114-
"actionFragment": "\"__std\".\"actions\".\"x86_64-linux\".\"_automation\".\"containers\".\"vscode\".\"publish",
115-
"block": "containers",
116-
"blockType": "containers",
117-
"cell": "_automation",
118-
"name": "vscode",
119-
"targetDrv": "/nix/store/4hs8x5lgb9nkvjfrxj7azv95hi77avxn-image-std-vscode.json.drv",
120-
"targetFragment": "\"x86_64-linux\".\"_automation\".\"containers\".\"vscode\""
121-
}
122-
]
123-
},
124-
"devshells": {
125-
"build": [
126-
{
127-
"action": "build",
128-
"actionDrv": "/nix/store/zmlva6xlngzj098znyy47p72rxjzgka3-build.drv",
129-
"actionFragment": "\"__std\".\"actions\".\"x86_64-linux\".\"_automation\".\"devshells\".\"default\".\"build",
130-
"block": "devshells",
131-
"blockType": "devshells",
132-
"cell": "_automation",
133-
"name": "default",
134-
"targetDrv": "/nix/store/xq4sl7pf51gp0a036garz56kkr160n5c-Standard.drv",
135-
"targetFragment": "\"x86_64-linux\".\"_automation\".\"devshells\".\"default\""
136-
}
137-
]
138-
},
139-
"packages": {
140-
"build": [
141-
{
142-
"action": "build",
143-
"actionDrv": "/nix/store/l4y4gzpgym5wbvn42avsaf24nqj0d27y-build.drv",
144-
"actionFragment": "\"__std\".\"actions\".\"x86_64-linux\".\"std\".\"packages\".\"adrgen\".\"build",
145-
"block": "packages",
146-
"blockType": "installables",
147-
"cell": "std",
148-
"name": "adrgen",
149-
"targetDrv": "/nix/store/mwidj7li8b7zypq83ap0fmmwxqx58qn6-adrgen-2022-08-08.drv",
150-
"targetFragment": "\"x86_64-linux\".\"std\".\"packages\".\"adrgen\""
151-
}
152-
]
153-
}
154-
}
162+
```yaml
163+
# yq '. | explode(.)' this.yml > .github/workflows/std.yml
164+
name: CI/CD
165+
166+
on:
167+
pull_request:
168+
branches:
169+
- main
170+
push:
171+
branches:
172+
- main
173+
174+
env:
175+
DISCOVERY_USER_NAME: gha-runner
176+
DISCOVERY_KNOWN_HOSTS_ENTRY: "10.10.10.10 ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEOVVDZydvD+diYa6A3EtA3WGw5NfN0wv7ckQxa/fX1O"
177+
178+
permissions:
179+
id-token: write
180+
contents: read
181+
182+
concurrency:
183+
group: ${{ github.sha }}
184+
cancel-in-progress: true
185+
186+
jobs:
187+
discover:
188+
outputs:
189+
hits: ${{ steps.discovery.outputs.hits }}
190+
runs-on: [self-hosted, discovery]
191+
steps:
192+
- name: Standard Discovery
193+
uses: divnix/std-action/discover@main
194+
id: discovery
195+
# avoids transporting derivations via GH Cache
196+
with: { ffBuildInstructions: true }
197+
198+
image: &run-job
199+
needs: discover
200+
strategy:
201+
fail-fast: false
202+
matrix:
203+
target: ${{ fromJSON(needs.discover.outputs.hits).oci-images.publish }}
204+
if: fromJSON(needs.discover.outputs.hits).oci-images.publish != '{}'
205+
name: ${{ matrix.target.jobName }}
206+
runs-on: ubuntu-latest
207+
steps:
208+
# sets up ssh credentials for `ssh discovery ...`
209+
- uses: divnix/std-action/setup-discovery-ssh@main
210+
with:
211+
ssh_key: ${{ secrets.SSH_PRIVATE_KEY_CI }}
212+
user_name: ${{ env.DISCOVERY_USER_NAME }}
213+
ssh_known_hosts_entry: ${{ env.DISCOVERY_KNOWN_HOSTS_ENTRY }}
214+
- uses: divnix/std-action/run@main
215+
# avoids retreiving derivations via GH Cache and uses `ssh discovery ...` instead
216+
with: { ffBuildInstructions: true }
217+
218+
build:
219+
<<: *run-job
220+
strategy:
221+
matrix:
222+
target: ${{ fromJSON(needs.discover.outputs.hits).envs.build }}
223+
if: fromJSON(needs.discover.outputs.hits).envs.build != '{}'
155224
```
225+
226+
</details>
227+
228+
## Notes & Explanation
229+
230+
### Notes on the Build Matrix
231+
232+
Hits from the discovery phase are namespaced by Block and Action.
233+
234+
That means:
235+
236+
- In: `target: ${{ fromJSON(needs.discover.outputs.hits).packages.build }}`
237+
- `packages` is the name of a Standard Block
238+
- `build` is the name of an Action of that Block
239+
240+
### Debugging
241+
242+
Watch out for `base64`-encoded blobs in the logs, you can inspect the
243+
working data of that context by doing: `base64 -d <<< copy-blob-here | jq`.

‎discover/eval.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -142,7 +142,7 @@ echo "::group::📞️ Pass artifacts to the build matrix ..."
142142
if [[ "$SKIP_DRV_EXPORT" == "false" ]]; then
143143
command mkdir -p "$EVALSTORE_EXPORT"
144144
for drv in $(command jq --compact-output --raw-output '.[].actionDrv' <<<"$PROVISIONED"); do
145-
nix-store --query --requisites "$drv" | command nix-store --stdin --export | command zstd > "$EVALSTORE_EXPORT/$(basename $drv).zst"
145+
command nix-store --query --requisites "$drv" | command nix-store --stdin --export | command zstd > "$EVALSTORE_EXPORT/$(basename $drv).zst"
146146
done
147147
fi
148148
}

0 commit comments

Comments
 (0)