|
| 1 | +<!--- |
| 2 | + Licensed to the Apache Software Foundation (ASF) under one |
| 3 | + or more contributor license agreements. See the NOTICE file |
| 4 | + distributed with this work for additional information |
| 5 | + regarding copyright ownership. The ASF licenses this file |
| 6 | + to you under the Apache License, Version 2.0 (the |
| 7 | + "License"); you may not use this file except in compliance |
| 8 | + with the License. You may obtain a copy of the License at |
| 9 | +
|
| 10 | + http://www.apache.org/licenses/LICENSE-2.0 |
| 11 | +
|
| 12 | + Unless required by applicable law or agreed to in writing, |
| 13 | + software distributed under the License is distributed on an |
| 14 | + "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| 15 | + KIND, either express or implied. See the License for the |
| 16 | + specific language governing permissions and limitations |
| 17 | + under the License. |
| 18 | +--> |
| 19 | + |
| 20 | + |
| 21 | +## Process Overview |
| 22 | + |
| 23 | +As part of the Apache governance model, official releases consist of signed |
| 24 | +source tarballs approved by the DataFusion PMC. |
| 25 | + |
| 26 | +We then use the code in the approved artifacts to release to crates.io. |
| 27 | + |
| 28 | +### Change Log |
| 29 | + |
| 30 | +We maintain a `CHANGELOG.md` so our users know what has been changed between releases. |
| 31 | + |
| 32 | +You will need a GitHub Personal Access Token for the following steps. Follow |
| 33 | +[these instructions](https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/creating-a-personal-access-token) |
| 34 | +to generate one if you do not already have one. |
| 35 | + |
| 36 | +The changelog is generated using a Python script which needs `PyGitHub`, installed using pip: |
| 37 | + |
| 38 | +```shell |
| 39 | +pip3 install PyGitHub |
| 40 | +``` |
| 41 | + |
| 42 | +To generate the changelog, set the `GITHUB_TOKEN` environment variable to a valid token and then run the script |
| 43 | +providing two commit ids or tags followed by the version number of the release being created. The following |
| 44 | +example generates a change log of all changes between the first commit and the current HEAD revision. |
| 45 | + |
| 46 | +```shell |
| 47 | +export GITHUB_TOKEN=<your-token-here> |
| 48 | +python ./dev/release/generate-changelog.py v0.51.0 HEAD 0.52.0 > changelog/0.52.0.md |
| 49 | +``` |
| 50 | + |
| 51 | +This script creates a changelog from GitHub PRs based on the labels associated with them as well as looking for |
| 52 | +titles starting with `feat:`, `fix:`, or `docs:`. |
| 53 | + |
| 54 | +Add an entry to CHANGELOG.md for the new version |
| 55 | + |
| 56 | +## Prepare release commits and PR |
| 57 | + |
| 58 | +### Update Version |
| 59 | + |
| 60 | +Checkout the main commit to be released |
| 61 | + |
| 62 | +```shell |
| 63 | +git fetch apache |
| 64 | +git checkout apache/main |
| 65 | +``` |
| 66 | + |
| 67 | +Manually update the version in the root `Cargo.toml` to the release version (e.g. `0.52.0`). |
| 68 | + |
| 69 | +Lastly commit the version change: |
| 70 | + |
| 71 | +```shell |
| 72 | +git commit -a -m 'Update version' |
| 73 | +``` |
| 74 | + |
| 75 | +## Prepare release candidate artifacts |
| 76 | + |
| 77 | +After the PR gets merged, you are ready to create release artifacts from the |
| 78 | +merged commit. |
| 79 | + |
| 80 | +(Note you need to be a committer to run these scripts as they upload to the apache svn distribution servers) |
| 81 | + |
| 82 | +### Pick a Release Candidate (RC) number |
| 83 | + |
| 84 | +Pick numbers in sequential order, with `0` for `rc0`, `1` for `rc1`, etc. |
| 85 | + |
| 86 | +### Create git tag for the release: |
| 87 | + |
| 88 | +While the official release artifacts are signed tarballs and zip files, we also |
| 89 | +tag the commit it was created for convenience and code archaeology. |
| 90 | + |
| 91 | +Using a string such as `v0.52.0` as the `<version>`, create and push the tag by running these commands: |
| 92 | + |
| 93 | +For example to tag version `0.52.0` |
| 94 | + |
| 95 | +```shell |
| 96 | +git fetch apache |
| 97 | +git tag v0.52.0-rc1 apache/main |
| 98 | +# push tag to Github remote |
| 99 | +git push apache v0.52.0-rc1 |
| 100 | +``` |
| 101 | + |
| 102 | +### Create, sign, and upload artifacts |
| 103 | + |
| 104 | +Run `create-tarball.sh` with the `<version>` tag and `<rc>` and you found in previous steps: |
| 105 | + |
| 106 | +```shell |
| 107 | +GITHUB_TOKEN=<TOKEN> ./dev/release/create-tarball.sh 0.52.0 1 |
| 108 | +``` |
| 109 | + |
| 110 | +The `create-tarball.sh` script |
| 111 | + |
| 112 | +1. creates and uploads all release candidate artifacts to the [datafusion |
| 113 | + dev](https://dist.apache.org/repos/dist/dev/datafusion) location on the |
| 114 | + apache distribution svn server |
| 115 | + |
| 116 | +2. provide you an email template to |
| 117 | + send to [email protected] for release voting. |
| 118 | + |
| 119 | +### Vote on Release Candidate artifacts |
| 120 | + |
| 121 | +Send the email output from the script to [email protected]. |
| 122 | + |
| 123 | +For the release to become "official" it needs at least three PMC members to vote +1 on it. |
| 124 | + |
| 125 | +### Verifying Release Candidates |
| 126 | + |
| 127 | +The `dev/release/verify-release-candidate.sh` is a script in this repository that can assist in the verification process. Run it like: |
| 128 | + |
| 129 | +```shell |
| 130 | +./dev/release/verify-release-candidate.sh 0.52.0 1 |
| 131 | +``` |
| 132 | + |
| 133 | +#### If the release is not approved |
| 134 | + |
| 135 | +If the release is not approved, fix whatever the problem is, merge changelog |
| 136 | +changes into main if there is any and try again with the next RC number. |
| 137 | + |
| 138 | +## Finalize the release |
| 139 | + |
| 140 | +NOTE: steps in this section can only be done by PMC members. |
| 141 | + |
| 142 | +### After the release is approved |
| 143 | + |
| 144 | +Move artifacts to the release location in SVN, using the `release-tarball.sh` script: |
| 145 | + |
| 146 | +```shell |
| 147 | +./dev/release/release-tarball.sh 0.52.0 1 |
| 148 | +``` |
| 149 | + |
| 150 | +Congratulations! The release is now official! |
| 151 | + |
| 152 | +### Publish on Crates.io |
| 153 | + |
| 154 | +Only approved releases of the tarball should be published to |
| 155 | +crates.io, in order to conform to Apache Software Foundation |
| 156 | +governance standards. |
| 157 | + |
| 158 | +A DataFusion committer can publish this crate after an official project release has |
| 159 | +been made to crates.io using the following instructions. |
| 160 | + |
| 161 | +Follow [these |
| 162 | +instructions](https://doc.rust-lang.org/cargo/reference/publishing.html) to |
| 163 | +create an account and login to crates.io before asking to be added as an owner |
| 164 | +to the sqlparser DataFusion crates. |
| 165 | + |
| 166 | +Download and unpack the official release tarball |
| 167 | + |
| 168 | +Verify that the Cargo.toml in the tarball contains the correct version |
| 169 | +(e.g. `version = "0.52.0"`) and then publish the crates by running the following commands |
| 170 | + |
| 171 | +```shell |
| 172 | +cargo publish |
| 173 | +``` |
| 174 | + |
| 175 | +If necessary, also publish the `sqlparser_derive` crate: |
| 176 | + |
| 177 | +crates.io homepage: https://crates.io/crates/sqlparser_derive |
| 178 | + |
| 179 | +```shell |
| 180 | +(cd derive && cargo publish |
| 181 | +``` |
0 commit comments