From 4628c195fc412fa2e0f9acd1fc3893b95be03f9d Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Mon, 30 Jan 2023 22:07:55 +0000 Subject: [PATCH 1/4] add formal proposal for lifecycle scripts --- .../features-contribute-lifecycle-scripts.md | 142 ++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 proposals/features-contribute-lifecycle-scripts.md diff --git a/proposals/features-contribute-lifecycle-scripts.md b/proposals/features-contribute-lifecycle-scripts.md new file mode 100644 index 00000000..260e14f1 --- /dev/null +++ b/proposals/features-contribute-lifecycle-scripts.md @@ -0,0 +1,142 @@ +# [Proposal] Allow Dev Container Features to contribute lifecycle scripts + +Related to: https://github.com/devcontainers/spec/issues/60, https://github.com/devcontainers/spec/issues/181 + +## Goal + +Allow Feature authors to provide lifecycle hooks for their Features during a dev container build. + +## Proposal + + Introduce the following properties to [devcontainer-feature.json](https://containers.dev/implementors/features/#devcontainer-feature-json-properties), mirroring the behavior and syntax of the `devcontainer.json` lifecycle hooks. + +- "onCreateCommand" +- "updateContentCommand" +- "postCreateCommand" +- "postStartCommand" +- "postAttachCommand" + +Note that `initializeCommand` is omitted, pending further discussions around a secure design. + +Additionally, introduce a `${featureRoot}` environment variable, which is expanded at build time to the root of the expanded Feature the variable is used from. + +Feature lifecycle hooks will be prepended to the lifecycle hooks (if any) declared by the current `devcontainer.json`. Lifecycle hooks will be prepended in the order that the Feature's installation script was executed. Commands will be joined in such a way that a failure in any one lifecycle hook, will indicate a failure for that entire lifecycle hook execution. + +### Parallel Execution + +Any Features that declare lifecycle hooks using the [parallel execution syntax](https://containers.dev/implementors/spec/#parallel-exec) will be executed in parallel with any other Features or user-contributed lifecycle scripts. Any Features that do not use this syntax will continue to be run in sequence, blocking subsequent runs until the script exits successfully. + +Each parallel stage inherited from a Feature will be prefixed with the Feature `id`. Eg: `featureA_migrateDB`. + +## Examples + + +### Executing a script in the workspace folder + +The follow example illustrates executing a `postCreateCommand` script in the [project workspace folder](https://containers.dev/implementors/spec/#project-workspace-folder). + +```jsonc +{ + "id": "featureA", + "version": "1.0.0" + "postCreateCommand": "scriptInWorkspaceFolder.sh" +} + +``` + +### Executing a script bundled with the Feature + +The follow example illustrates executing a `postCreateCommand` script bundled with the Feature by using the `${featureRoot}` variable. + +```jsonc +{ + "id": "featureB", + "version": "1.0.0" + "postCreateCommand": "${featureRoot}/bundledScript.sh", + "installsAfter": [ + "featureA" + ] +} +``` + +At build time, the `${featureRoot}` variable will be expanded to the temporary directory that contains all the Feature's assets. For example, it may be expanded to `/tmp/vsch/container-features/featureB_1`. + +When authored, `featureB`'s file structure would look something like: + +``` +. +├── src +│ ├── featureB +│ │ ├── bundledScript.sh +│ │ ├── devcontainer-feature.json +│ │ └── install.sh +``` + +### Installation Order (without parallel execution syntax) + +Given the following `devcontainer.json` and the two examples Features above. + +```jsonc +{ + "image": "ubuntu", + "features": { + "featureA": {}, + "featureB": {}, + }, + "postCreateCommand": "scriptInWorkspaceFolder.sh" +} +``` + +The three `postCreate` lifecycle events declared or inherited for this dev container would be serially executed (blocking the next script) in the following order: + +- featureA +- featureB +- the user's dev container + +Note that the current working directory during the execution of all three scripts is equal to the [project workspace folder](https://containers.dev/implementors/spec/#project-workspace-folder). + +### Installation Order (parallel execution) + +Given a Feature defined as follows: + +```jsonc +{ + "id": "featureC", + "version": "1.0.0", + "postCreateCommand": { + "server": "npm start", + "db": ["mysql", "-u", "root", "-p", "my database"] + }, + "installsAfter": [ + "featureA", + "featureB" + ] +} +``` + +The following dev container configuration: + +```jsonc +{ + "image": "ubuntu", + "features": { + "featureA": {}, + "featureB": {}, + "featureC": {}, + }, + "postCreateCommand": { + "git": "git clone https://github.com/devcontainers/cli", + "getTool": "wget https://example.com/tool/" + } +} +``` + +With each bullet point indicating a blocking operation for the following scripts, the execution for the `postCreate` lifecycle hook would behave like: + +- featureA +- featureB +- In parallel the execution of + - `featureC_server` + - `featureC_db` + - `git` + - `getTool` \ No newline at end of file From d3ae48c5b4f761ef4126f9c28d15a618ddc61046 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Fri, 3 Feb 2023 17:40:51 +0000 Subject: [PATCH 2/4] update spec --- .../features-contribute-lifecycle-scripts.md | 98 ++++++------------- 1 file changed, 32 insertions(+), 66 deletions(-) diff --git a/proposals/features-contribute-lifecycle-scripts.md b/proposals/features-contribute-lifecycle-scripts.md index 260e14f1..1fae20d4 100644 --- a/proposals/features-contribute-lifecycle-scripts.md +++ b/proposals/features-contribute-lifecycle-scripts.md @@ -18,50 +18,52 @@ Allow Feature authors to provide lifecycle hooks for their Features during a dev Note that `initializeCommand` is omitted, pending further discussions around a secure design. -Additionally, introduce a `${featureRoot}` environment variable, which is expanded at build time to the root of the expanded Feature the variable is used from. +Additionally, introduce a `${featureRoot}` dev container variable, which is expanded at build time to the root of the Feature the variable is referenced from. -Feature lifecycle hooks will be prepended to the lifecycle hooks (if any) declared by the current `devcontainer.json`. Lifecycle hooks will be prepended in the order that the Feature's installation script was executed. Commands will be joined in such a way that a failure in any one lifecycle hook, will indicate a failure for that entire lifecycle hook execution. +As with all lifecycle hooks, commands are executed from the context (cwd) of the [project workspace folder](https://containers.dev/implementors/spec/#project-workspace-folder). -### Parallel Execution +All other semantic match the existing [Lifecycle Scripts](https://containers.dev/implementors/json_reference/#lifecycle-scripts) behavior exactly. -Any Features that declare lifecycle hooks using the [parallel execution syntax](https://containers.dev/implementors/spec/#parallel-exec) will be executed in parallel with any other Features or user-contributed lifecycle scripts. Any Features that do not use this syntax will continue to be run in sequence, blocking subsequent runs until the script exits successfully. +### Execution -Each parallel stage inherited from a Feature will be prefixed with the Feature `id`. Eg: `featureA_migrateDB`. +Any Features that declare one of the aforementioned lifecycle hook properties will have their command executed _in parallel with any other Features, or user-contributed, lifecycle scripts_ during the target lifecycle point. + +> A consequence of executing scripts in parallel requires that operations in one script do not block another. The execution order of individual lifecycle scripts within a given lifecycle hook is undefined, and any perceived ordering should not be relied upon. + +See here for more information on [lifecycle script parallel execution](https://containers.dev/implementors/spec/#parallel-exec). ## Examples ### Executing a script in the workspace folder -The follow example illustrates executing a `postCreateCommand` script in the [project workspace folder](https://containers.dev/implementors/spec/#project-workspace-folder). +The follow example illustrates contributing an `onCreateCommand` and `postCreateCommand` script to `featureA`. At each lifecycle hook during the build, the provided script will be executed in the [project workspace folder](https://containers.dev/implementors/spec/#project-workspace-folder), following the same semantics as [user-defined lifecycle hooks](https://containers.dev/implementors/json_reference/#lifecycle-scripts). ```jsonc { "id": "featureA", - "version": "1.0.0" - "postCreateCommand": "scriptInWorkspaceFolder.sh" + "version": "1.0.0", + "onCreateCommand": "myOnCreate.sh && myOnCreate2.sh", + "postCreateCommand": "myPostCreate.sh" } ``` ### Executing a script bundled with the Feature -The follow example illustrates executing a `postCreateCommand` script bundled with the Feature by using the `${featureRoot}` variable. +The following example illustrates executing a `postCreateCommand` script bundled with the Feature utilizing the `${featureRoot}` variable. ```jsonc { "id": "featureB", - "version": "1.0.0" - "postCreateCommand": "${featureRoot}/bundledScript.sh", - "installsAfter": [ - "featureA" - ] + "version": "1.0.0", + "postCreateCommand": "${featureRoot}/bundledScript.sh" } ``` -At build time, the `${featureRoot}` variable will be expanded to the temporary directory that contains all the Feature's assets. For example, it may be expanded to `/tmp/vsch/container-features/featureB_1`. +At build time, the `${featureRoot}` variable will be expanded to the temporary directory within the container that contains all the Feature's assets. For example, it may be expanded to `/tmp/vsch/container-features/featureB_1`. -When authored, `featureB`'s file structure would look something like: +Given this example, `featureB`'s file structure would look something like: ``` . @@ -72,9 +74,9 @@ When authored, `featureB`'s file structure would look something like: │ │ └── install.sh ``` -### Installation Order (without parallel execution syntax) +### Installation -Given the following `devcontainer.json` and the two examples Features above. +Given the following `devcontainer.json` and the two example Features above. ```jsonc { @@ -83,60 +85,24 @@ Given the following `devcontainer.json` and the two examples Features above. "featureA": {}, "featureB": {}, }, - "postCreateCommand": "scriptInWorkspaceFolder.sh" -} -``` - -The three `postCreate` lifecycle events declared or inherited for this dev container would be serially executed (blocking the next script) in the following order: - -- featureA -- featureB -- the user's dev container - -Note that the current working directory during the execution of all three scripts is equal to the [project workspace folder](https://containers.dev/implementors/spec/#project-workspace-folder). - -### Installation Order (parallel execution) - -Given a Feature defined as follows: - -```jsonc -{ - "id": "featureC", - "version": "1.0.0", - "postCreateCommand": { + "postCreateCommand": "userPostCreate.sh", + "postAttachCommand": { "server": "npm start", "db": ["mysql", "-u", "root", "-p", "my database"] }, - "installsAfter": [ - "featureA", - "featureB" - ] } ``` -The following dev container configuration: +The following timeline of events would occur. -```jsonc -{ - "image": "ubuntu", - "features": { - "featureA": {}, - "featureB": {}, - "featureC": {}, - }, - "postCreateCommand": { - "git": "git clone https://github.com/devcontainers/cli", - "getTool": "wget https://example.com/tool/" - } -} -``` +> Note that: +> +>1. Each bullet point below is _blocking_. No subsequent lifecycle hooks shall proceed until the current hook completes. +> 2. If one of the lifecycle scripts fails, any subsequent scripts will not be executed. For instance, if `postCreateCommand` fails, `postStartCommand` and any following hooks will be skipped. +> -With each bullet point indicating a blocking operation for the following scripts, the execution for the `postCreate` lifecycle hook would behave like: +- `featureA`'s onCreateCommand +- `featureA`'s postCreateCommand, `featureB`'s postCreateCommand, and the user's postCreateCommand **(all executed in parallel)** +- The user's postCreateCommand **(each command running in parallel)** -- featureA -- featureB -- In parallel the execution of - - `featureC_server` - - `featureC_db` - - `git` - - `getTool` \ No newline at end of file +Suppose `featureB`'s postCreateCommand exited with a non-zero exit code. In this case, the `postAttachCommand` will never fire. From ff2e815f8caf6695f3f07d0c7ef8f721919c62b4 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Mon, 6 Feb 2023 11:09:31 -0800 Subject: [PATCH 3/4] run command serially --- .../features-contribute-lifecycle-scripts.md | 32 +++++++++++-------- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/proposals/features-contribute-lifecycle-scripts.md b/proposals/features-contribute-lifecycle-scripts.md index 1fae20d4..7e8a212c 100644 --- a/proposals/features-contribute-lifecycle-scripts.md +++ b/proposals/features-contribute-lifecycle-scripts.md @@ -22,15 +22,11 @@ Additionally, introduce a `${featureRoot}` dev container variable, which is expa As with all lifecycle hooks, commands are executed from the context (cwd) of the [project workspace folder](https://containers.dev/implementors/spec/#project-workspace-folder). -All other semantic match the existing [Lifecycle Scripts](https://containers.dev/implementors/json_reference/#lifecycle-scripts) behavior exactly. +All other semantic match the existing [Lifecycle Scripts](https://containers.dev/implementors/json_reference/#lifecycle-scripts) and [lifecycle script parallel execution](https://containers.dev/implementors/spec/#parallel-exec) behavior exactly. ### Execution -Any Features that declare one of the aforementioned lifecycle hook properties will have their command executed _in parallel with any other Features, or user-contributed, lifecycle scripts_ during the target lifecycle point. - -> A consequence of executing scripts in parallel requires that operations in one script do not block another. The execution order of individual lifecycle scripts within a given lifecycle hook is undefined, and any perceived ordering should not be relied upon. - -See here for more information on [lifecycle script parallel execution](https://containers.dev/implementors/spec/#parallel-exec). +When a dev container is brought up, for each lifecycle hook, each Feature that contributes a command to a lifecycle hook shall have the command executed in sequence, following the same execution order as outlined in [Feature installation order](https://containers.dev/implementors/features/#installation-order), and always before any user-provided lifecycle commands. ## Examples @@ -44,7 +40,11 @@ The follow example illustrates contributing an `onCreateCommand` and `postCreate "id": "featureA", "version": "1.0.0", "onCreateCommand": "myOnCreate.sh && myOnCreate2.sh", - "postCreateCommand": "myPostCreate.sh" + "postCreateCommand": "myPostCreate.sh", + "postAttachCommand": { + "command01": "myPostAttach.sh arg01", + "command02": "myPostAttach.sh arg02", + }, } ``` @@ -57,7 +57,10 @@ The following example illustrates executing a `postCreateCommand` script bundled { "id": "featureB", "version": "1.0.0", - "postCreateCommand": "${featureRoot}/bundledScript.sh" + "postCreateCommand": "${featureRoot}/bundledScript.sh", + "installsAfter": [ + "featureA" + ] } ``` @@ -97,12 +100,15 @@ The following timeline of events would occur. > Note that: > ->1. Each bullet point below is _blocking_. No subsequent lifecycle hooks shall proceed until the current hook completes. -> 2. If one of the lifecycle scripts fails, any subsequent scripts will not be executed. For instance, if `postCreateCommand` fails, `postStartCommand` and any following hooks will be skipped. +>1. Each bullet point below is _blocking_. No subsequent lifecycle commands shall proceed until the current command completes. +> 2. If one of the lifecycle scripts fails, any subsequent scripts will not be executed. For instance, if a given `postCreate` command fails, the `postStart` hook and any following hooks will be skipped. > - `featureA`'s onCreateCommand -- `featureA`'s postCreateCommand, `featureB`'s postCreateCommand, and the user's postCreateCommand **(all executed in parallel)** -- The user's postCreateCommand **(each command running in parallel)** +- `featureA`'s postCreateCommand +- `featureB`'s postCreateCommand +- The user's postCreateCommand +- `featureA`'s postAttachCommand **(parallel syntax, each command running in parallel)** +- The user's postAttachCommand **(parallel syntax, each command running in parallel)** -Suppose `featureB`'s postCreateCommand exited with a non-zero exit code. In this case, the `postAttachCommand` will never fire. +Suppose `featureB`'s postCreateCommand exited were to exit unsuccessfully (non-zero exit code). In this case, the `postAttachCommand` will never fire. From c17395dcd3ae80af5aa2ba1bcefdec5be866a670 Mon Sep 17 00:00:00 2001 From: Josh Spicer Date: Tue, 7 Feb 2023 10:22:06 -0800 Subject: [PATCH 4/4] '${featureRoot}' -> '${featureRootFolder} --- proposals/features-contribute-lifecycle-scripts.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/proposals/features-contribute-lifecycle-scripts.md b/proposals/features-contribute-lifecycle-scripts.md index 7e8a212c..a1c9dca7 100644 --- a/proposals/features-contribute-lifecycle-scripts.md +++ b/proposals/features-contribute-lifecycle-scripts.md @@ -18,7 +18,7 @@ Allow Feature authors to provide lifecycle hooks for their Features during a dev Note that `initializeCommand` is omitted, pending further discussions around a secure design. -Additionally, introduce a `${featureRoot}` dev container variable, which is expanded at build time to the root of the Feature the variable is referenced from. +Additionally, introduce a `${featureRootFolder}` dev container variable, which is expanded at build time to the root of the Feature the variable is referenced from. As with all lifecycle hooks, commands are executed from the context (cwd) of the [project workspace folder](https://containers.dev/implementors/spec/#project-workspace-folder). @@ -51,20 +51,20 @@ The follow example illustrates contributing an `onCreateCommand` and `postCreate ### Executing a script bundled with the Feature -The following example illustrates executing a `postCreateCommand` script bundled with the Feature utilizing the `${featureRoot}` variable. +The following example illustrates executing a `postCreateCommand` script bundled with the Feature utilizing the `${featureRootFolder}` variable. ```jsonc { "id": "featureB", "version": "1.0.0", - "postCreateCommand": "${featureRoot}/bundledScript.sh", + "postCreateCommand": "${featureRootFolder}/bundledScript.sh", "installsAfter": [ "featureA" ] } ``` -At build time, the `${featureRoot}` variable will be expanded to the temporary directory within the container that contains all the Feature's assets. For example, it may be expanded to `/tmp/vsch/container-features/featureB_1`. +At build time, the `${featureRootFolder}` variable will be expanded to the temporary directory within the container that contains all the Feature's assets. For example, it may be expanded to `/tmp/vsch/container-features/featureB_1`. Given this example, `featureB`'s file structure would look something like: