Skip to content

Commit aac44f3

Browse files
snazysnazy
committed
WIP Release infra stuff
Introduce a couple of shell scripts to automate the release process. Some docs are included under `releases/README.md`. Generally, releases at the ASF follow the following workflow: 1. Draft a release 2. Start a VOTE on the dev mailing list 3. If the VOTE fails, the release has failed - "go to step 1" 4. If the VOTE passes, publish the release The above process is, without release branches, reflected in the scripts: 1. `releases/bin/draft-release.sh --major <major-version-number> --minor <minor-version-number> --commit <Git-commit-ID>` 2. if the vote passes: `releases/bin/publish-release.sh --major <major-version-number> --minor <minor-version-number>` 3. if the vote fails, just run `draft-release.sh` again The change includes scripts to handle version branches, however, using those is not required for the two release scripts above. It's important to know that the scripts handle changes to the `version.txt` file and that a specific syntax for Git tags is expected, which is required to automatically use/generate RC and final versions, including the artifact publishing via Sonatype.
1 parent 4b380a5 commit aac44f3

15 files changed

+1655
-148
lines changed

‎NEXT_RELEASE_NOTES.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
<!--
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
-->

‎build-logic/src/main/kotlin/publishing/PublishingHelperPlugin.kt

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -47,15 +47,8 @@ import org.gradle.plugins.signing.SigningPlugin
4747
* The task `sourceTarball` (available on the root project) generates a source tarball using `git
4848
* archive`.
4949
*
50-
* The task `releaseEmailTemplate` generates the release-vote email subject + body. Outputs on the
51-
* console and in the `build/distributions/` directory.
52-
*
5350
* Signing tip: If you want to use `gpg-agent`, set the `useGpgAgent` Gradle project property
5451
*
55-
* The following command publishes the project artifacts to your local maven repository, generates
56-
* the source tarball - and uses `gpg-agent` to sign all artifacts and the tarball. Note that this
57-
* requires a Git tag!
58-
*
5952
* ```
6053
* ./gradlew publishToMavenLocal sourceTarball -Prelease -PuseGpgAgent
6154
* ```

‎build-logic/src/main/kotlin/publishing/rootProject.kt

Lines changed: 12 additions & 129 deletions
Original file line numberDiff line numberDiff line change
@@ -19,14 +19,11 @@
1919

2020
package publishing
2121

22-
import io.github.gradlenexus.publishplugin.NexusPublishExtension
2322
import io.github.gradlenexus.publishplugin.NexusPublishPlugin
24-
import io.github.gradlenexus.publishplugin.internal.StagingRepositoryDescriptorRegistryBuildService
2523
import org.gradle.api.Project
26-
import org.gradle.api.services.BuildServiceRegistration
24+
import org.gradle.api.tasks.Delete
2725
import org.gradle.api.tasks.Exec
2826
import org.gradle.kotlin.dsl.apply
29-
import org.gradle.kotlin.dsl.named
3027
import org.gradle.kotlin.dsl.register
3128
import org.gradle.plugins.signing.Sign
3229

@@ -41,11 +38,22 @@ internal fun configureOnRootProject(project: Project) =
4138
val isRelease = project.hasProperty("release")
4239
val isSigning = isRelease || project.hasProperty("signArtifacts")
4340

41+
val cleanDistributionsDir = tasks.register<Delete>("cleanDistributionsDir")
42+
cleanDistributionsDir.configure {
43+
outputs.cacheIf { false }
44+
45+
val e = project.extensions.getByType(PublishingHelperExtension::class.java)
46+
delete(e.distributionDir)
47+
}
48+
4449
val sourceTarball = tasks.register<Exec>("sourceTarball")
4550
sourceTarball.configure {
4651
group = "build"
4752
description =
4853
"Generate a source tarball for a release to be uploaded to dist.apache.org/repos/dist"
54+
outputs.cacheIf { false }
55+
56+
dependsOn(cleanDistributionsDir)
4957

5058
val e = project.extensions.getByType(PublishingHelperExtension::class.java)
5159
doFirst { mkdir(e.distributionDir) }
@@ -84,129 +92,4 @@ internal fun configureOnRootProject(project: Project) =
8492
}
8593
sourceTarball.configure { finalizedBy(signSourceTarball) }
8694
}
87-
88-
val releaseEmailTemplate = tasks.register("releaseEmailTemplate")
89-
releaseEmailTemplate.configure {
90-
group = "publishing"
91-
description =
92-
"Generate release-vote email subject + body, including the staging repository URL, if run during the Maven release."
93-
94-
mustRunAfter("initializeApacheStagingRepository")
95-
96-
doFirst {
97-
val e = project.extensions.getByType(PublishingHelperExtension::class.java)
98-
val asfName = e.asfProjectName.get()
99-
100-
val gitInfo = MemoizedGitInfo.gitInfo(rootProject)
101-
val gitCommitId = gitInfo["Apache-Polaris-Build-Git-Head"]
102-
103-
val repos = project.extensions.getByType(NexusPublishExtension::class.java).repositories
104-
val repo = repos.iterator().next()
105-
106-
val stagingRepositoryUrlRegistryRegistration =
107-
gradle.sharedServices.registrations.named<
108-
BuildServiceRegistration<StagingRepositoryDescriptorRegistryBuildService, *>
109-
>(
110-
"stagingRepositoryUrlRegistry"
111-
)
112-
val staginRepoUrl =
113-
if (stagingRepositoryUrlRegistryRegistration.isPresent) {
114-
val stagingRepositoryUrlRegistryBuildServiceRegistration =
115-
stagingRepositoryUrlRegistryRegistration.get()
116-
val stagingRepositoryUrlRegistryService =
117-
stagingRepositoryUrlRegistryBuildServiceRegistration.getService()
118-
if (stagingRepositoryUrlRegistryService.isPresent) {
119-
val registry = stagingRepositoryUrlRegistryService.get().registry
120-
try {
121-
val stagingRepoDesc = registry.get(repo.name)
122-
val stagingRepoId = stagingRepoDesc.stagingRepositoryId
123-
"https://repository.apache.org/content/repositories/$stagingRepoId/"
124-
} catch (e: IllegalStateException) {
125-
"NO STAGING REPOSITORY ($e)"
126-
}
127-
} else {
128-
"NO STAGING REPOSITORY (no registry service) !!"
129-
}
130-
} else {
131-
"NO STAGING REPOSITORY (no build service) !!"
132-
}
133-
134-
val asfProjectName = fetchAsfProjectName(asfName)
135-
136-
val versionNoRc = version.toString().replace("-rc-?[0-9]+".toRegex(), "")
137-
138-
val subjectFile = e.distributionFile("vote-email-subject.txt").relativeTo(projectDir)
139-
val bodyFile = e.distributionFile("vote-email-body.txt").relativeTo(projectDir)
140-
141-
val emailSubject = "[VOTE] Release $asfProjectName $version"
142-
subjectFile.writeText(emailSubject)
143-
144-
val emailBody =
145-
"""
146-
Hi everyone,
147-
148-
I propose that we release the following RC as the official
149-
$asfProjectName $versionNoRc release.
150-
151-
* This corresponds to the tag: apache-$asfName-$version
152-
* https://github.com/apache/$asfName/commits/apache-$asfName-$version
153-
* https://github.com/apache/$asfName/tree/$gitCommitId
154-
155-
The release tarball, signature, and checksums are here:
156-
* https://dist.apache.org/repos/dist/dev/incubator/$asfName/apache-$asfName-$version
157-
158-
You can find the KEYS file here:
159-
* https://dist.apache.org/repos/dist/release/incubator/$asfName/KEYS
160-
161-
Convenience binary artifacts are staged on Nexus. The Maven repository URL is:
162-
* $staginRepoUrl
163-
164-
Please download, verify, and test.
165-
166-
Please vote in the next 72 hours.
167-
168-
[ ] +1 Release this as Apache $asfName $version
169-
[ ] +0
170-
[ ] -1 Do not release this because...
171-
172-
Only PPMC members and mentors have binding votes, but other community members are
173-
encouraged to cast non-binding votes. This vote will pass if there are
174-
3 binding +1 votes and more binding +1 votes than -1 votes.
175-
176-
NB: if this vote pass, a new vote has to be started on the Incubator general mailing
177-
list.
178-
179-
Thanks
180-
Regards
181-
"""
182-
183-
logger.lifecycle(
184-
"""
185-
186-
187-
The email for your release vote mail:
188-
-------------------------------------
189-
190-
Suggested subject: (also in file $subjectFile)
191-
192-
$emailSubject
193-
194-
Suggested body: (also in file $bodyFile)
195-
196-
$emailBody
197-
198-
"""
199-
.trimIndent()
200-
)
201-
bodyFile.writeText(emailBody.trimIndent())
202-
}
203-
}
204-
205-
if (isRelease) {
206-
sourceTarball.configure { finalizedBy(releaseEmailTemplate) }
207-
}
208-
209-
afterEvaluate {
210-
tasks.named("closeApacheStagingRepository") { mustRunAfter(releaseEmailTemplate) }
211-
}
21295
}

‎releases/README.md

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
# Apache Polaris Release Infrastructure
2+
3+
This directory holds all the infrastructure parts that are required to draft a Polaris release (RC) and to eventually
4+
release it.
5+
6+
Releases can be created from any Git commit, version branches are supported.
7+
8+
Generally, releases at the ASF follow the following workflow:
9+
10+
1. Draft a release
11+
2. Start a VOTE on the dev mailing list
12+
3. If the VOTE fails, the release has failed - "go to step 1"
13+
4. If the VOTE passes, publish the release
14+
15+
The Polaris project decided to run all release related via GitHub actions - a release is never drafted nor actually
16+
released from a developer machine.
17+
18+
## Technical process
19+
20+
In the `release/bin/` directory are the scripts that are required to draft a release and to publish it as "GA". All
21+
scripts can be called with the `--help` argument to get some usage information. There is also a `--dry-run` option
22+
to inspect what _would_ happen.
23+
24+
The technical process for a release follows the workflow above:
25+
26+
1. `releases/bin/draft-release.sh --major <major-version-number> --minor <minor-version-number> --commit <Git-commit-ID>`
27+
creates a release-candidate. The `--commit` argument is optional and defaults to the HEAD of the local Git
28+
worktree. The patch version number and RC-number are generated automatically. This means, that RC-numbers are
29+
automatically incremented as long as there is no "final" release tag. In that case, the patch version number
30+
is incremented and the RC-number is set to 1.
31+
32+
The Git tag name that will be used follows the pattern `polaris-<major>.<minor>.<patch>-RC<rc-number>`
33+
34+
The content of the `version.txt` file is set to the full release version, for example `1.2.3`.
35+
36+
Gradle will run with the arguments `-Prelease publishToApache closeApacheStagingRepository sourceTarball`.
37+
Both the release artifacts (jars) and the source tarball are signed. The signing and Nexus credentials must be
38+
provided externally using the `ORG_GRADLE_PROJECT_*` environment variables. In dry-run mode, this step runs
39+
Gradle with the arguments `-PjarWithGitInfo -PsignArtifacts -PuseGpgAgent publishToMavenLocal sourceTarball publishToMavenLocal`.
40+
41+
The staging repository ID, which is needed to release the staging repository, and URL are extracted from the
42+
Gradle output and memoized in the files `releases/current-release-staging-repository-id` and
43+
`releases/current-release-staging-repository-url`. The source Git commit ID is memoized in the file
44+
`releases/current-release-commit-id`.
45+
46+
The source tarball will be uploaded to the Apache infrastructure.
47+
48+
Release notes will be generated and included in the file `site/content/in-dev/unreleasd/release-notes.md`. This
49+
makes the release notes available later on the project website within the versioned docs.
50+
51+
The suggested release VOTE email subject and body with the correct links and information are provided.
52+
53+
The last step is to push the Git tag with a Git commit containing the above file changes.
54+
2. Once the release VOTE passed: `releases/bin/publish-release.sh --major <major-version-number> --minor <minor-version-number>`.
55+
56+
The command will find the latest RC tag for the latest patch release of the given major/minor version.
57+
58+
The final Git tag name that will be used follows the pattern `polaris-<major>.<minor>.<patch>` and created from
59+
the latest RC-tag for that version.
60+
61+
The Git tag is then pushed.
62+
63+
Documentation pages for the release will then be copied from the `site/content/in-dev/unreleased` into the
64+
`site/content/releases/<major>.<minor>.<patch>` folder within the `versioned-docs` branch.
65+
The changes for the updated `versioned-docs` branch are then pushed to Git.
66+
67+
The suggested ANNOUNCEMENT email subject and body are provided.
68+
69+
Creating a release in GitHub is the last step.
70+
71+
## Technical requirements
72+
73+
To use the scripts in the `releases/bin/` folder, it is required to have a _full_ clone with all tags and branches.
74+
Shallow clones, which is the default when checking out a Git repository in GitHub actions, will _not_ work properly!
75+
76+
On top, all scripts need privileges to be able to push tags and branches to the GitHub repository - this is an
77+
essential requirement for the scripts to do their job.
78+
79+
The `draft-release.sh` and `publish-release.sh` scripts also require the following environment variables providing
80+
the necessary secrets:
81+
82+
* `ORG_GRADLE_PROJECT_signingKey`
83+
* `ORG_GRADLE_PROJECT_signingPassword`
84+
* `ORG_GRADLE_PROJECT_sonatypeUsername`
85+
* `ORG_GRADLE_PROJECT_sonatypePassword`
86+
87+
GitHub actions running the scripts must provide those secrets and privileges.
88+
89+
## Version branches
90+
91+
The Polaris project may use maintenance version branches following the pattern `release/<major>.x` and
92+
`release/<major>.<minor>`. The scripts mentioned above are already support version branches and have validations for
93+
this use case. Using version branches is not a requirement for the release scripts.
94+
95+
The two scripts `releases/bin/create-release-branch.sh` plus the informative `releases/bin/list-release-branches.sh`
96+
are there to help with version branches. The former must be invoked on the main branch and creates a new major-version
97+
branch using the `release/<major>.x` pattern. The latter must be invoked on a major-version branch and creates a new
98+
minor version branch using the `release/<major>.<minor>` pattern.
99+
100+
# TODO
101+
102+
* Close staging repo when RC is abandoned
103+
* Adopt website (top bar menu, maintained releases, left site menu)

0 commit comments

Comments
 (0)