Skip to content

Dcv 2948 fix search on devdocs site #60

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
38 changes: 38 additions & 0 deletions .flake8
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
[flake8]
exclude =
.git,
.venv,
__pycache__,
*_template.py,
*/migrations/*,
*/node_modules/*,
k8s/kubesphere,
k8s/cluster/global,
.generated,
docker,
local,
config,
src/core/workspace,
src/core/old-workspace,
src/airbyte,
src/code-server,
src/dagster,
src/dbt-rpc,
src/superset,
src/observe,
src/pomerium
max-complexity = 10
max-line-length = 120
extend-ignore = E203
ignore =
E266
E741
E713
E731
W503
C901
per-file-ignores =
cli.py:E501
scripts/docker_images.py:E501
scripts/k8s_deploy.py:E501
*_generator.py:F541
5 changes: 5 additions & 0 deletions .gitguardian.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
version: 2

secret:
ignored-paths:
- '.gitsecret/**'
42 changes: 42 additions & 0 deletions .github/pull_request_template.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
<!---

Provide a short summary of the purpose of this PR.

-->

## Testing steps:

<!---

Uncomment this section and explain the steps needed to make this PR work.

-->

## Checklist:

<!---

This checklist is mostly useful as a reminder of small things that can easily be

forgotten – it is meant as a helpful tool rather than hoops to jump through.

Put an `x` in all the items that apply, make notes next to any that haven't been

addressed, and remove any items that are not relevant to this PR.

-->

- [ ] [Mandatory] My pull request represents one logical piece of work and my commits are related and look clean.

- [ ] [Mandatory] I ran the integration tests (`./cli.py integration_tests`)

- [ ] I have bumped minor or major version accordingly on `.version.yml` (see [Docs](https://github.com/datacoves/datacoves/blob/main/docs/how-tos/datacoves-versioning.md))

- [ ] I created new 1Password items (`./cli.py sync_secrets`)

- [ ] I have built new docker images on this branch (`./cli.py build_and_push`) so `./cli.py set_release` needs to be run to test it

- [ ] This requires configuration changes on the cluster (please specify configuration changes below)

- [ ] IF there are configuration changes or other changes to the release, please SET the 'special release step' label on this issue!

59 changes: 59 additions & 0 deletions .github/workflows/build_deploy_airflow-airflow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Build and deploy airflow-airflow

concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true

on:
workflow_dispatch:
push:
branches:
- main
- prev
paths:
- src/airflow/airflow/*
- src/airflow/airflow/**/*
- src/common/*
- src/common/**/*

jobs:
collect-airflow-profiles: # will output all airflow profiles under the profiles dir
runs-on: ubuntu-latest
outputs:
dir: ${{ steps.collect.outputs.dir }}
steps:
- uses: actions/checkout@v2
- id: collect
run: |
cd src/airflow/airflow
echo "::set-output name=dir::$(ls -1 profiles/ | jq -R -s -c 'split("\n")[:-1]')"

build_and_deploy_airflow-airflow:
name: Build and deploy airflow-airflow
runs-on: ubuntu-latest
container: datacoves/ci-multiarch:latest
needs: [collect-airflow-profiles]
strategy:
matrix:
dir: ${{ fromJson(needs.collect-airflow-profiles.outputs.dir) }}
env:
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}

steps:
- name: Trust directory
run: git config --global --add safe.directory /__w/datacoves/datacoves

- name: Checkout branch
uses: actions/checkout@v3
with:
fetch-depth: 2
ref: ${{ github.event.pull_request.head.sha }}

- name: Lint
run: flake8
shell: bash

- name: Build and deploy airflow-airflow base
run: ./cli.py ci_build_and_push airflow/airflow datacovesprivate ${{ matrix.dir }}
shell: bash
60 changes: 60 additions & 0 deletions .github/workflows/build_deploy_ci-airflow.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
name: Build and deploy ci-airflow

concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true

on:
workflow_dispatch:
push:
branches:
- main
- prev
paths:
- src/ci/airflow/*
- src/ci/airflow/**/*
- src/common/*
- src/common/**/*

jobs:
collect-airflow-profiles: # will output all airflow profiles under the profiles dir
runs-on: ubuntu-latest
outputs:
dir: ${{ steps.collect.outputs.dir }}
steps:
- uses: actions/checkout@v2
- id: collect
run: |
cd src/ci/airflow
echo "::set-output name=dir::$(ls -1 profiles/ | jq -R -s -c 'split("\n")[:-1]')"

build_and_deploy_ci-airflow:
name: Build and deploy ci-airflow
runs-on: ubuntu-latest
container: datacoves/ci-multiarch:latest
needs: [collect-airflow-profiles]
strategy:
matrix:
dir: ${{ fromJson(needs.collect-airflow-profiles.outputs.dir) }}
env:
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DATACOVES_GITHUB_API_TOKEN: ${{ secrets.DATACOVES_GITHUB_API_TOKEN }}

steps:
- name: Trust directory
run: git config --global --add safe.directory /__w/datacoves/datacoves

- name: Checkout branch
uses: actions/checkout@v3
with:
fetch-depth: 2
ref: ${{ github.event.pull_request.head.sha }}

- name: Lint
run: flake8
shell: bash

- name: Build and deploy ci-airflow
run: |
./cli.py ci_build_and_push ci/airflow datacoves ${{ matrix.dir }}
59 changes: 59 additions & 0 deletions .github/workflows/build_deploy_ci-basic.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
name: Build and deploy ci-basic

concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true

on:
workflow_dispatch:
push:
branches:
- main
- prev
paths:
- src/ci/basic/*
- src/ci/basic/**/*
- src/common/*
- src/common/**/*

jobs:
build_and_deploy_ci-basic:
name: Build and deploy ci-basic
runs-on: ubuntu-latest

container: datacoves/ci-multiarch:latest

env:
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
DATACOVES_GITHUB_API_TOKEN: ${{ secrets.DATACOVES_GITHUB_API_TOKEN }}

steps:
- name: Trust directory
run: git config --global --add safe.directory /__w/datacoves/datacoves

- name: Checkout branch
uses: actions/checkout@v3
with:
fetch-depth: 2
ref: ${{ github.event.pull_request.head.sha }}

- name: Build and deploy ci-basic base
run: ./cli.py ci_build_and_push ci/basic datacoves base
shell: bash

- name: Build and deploy ci-basic dbt-snowflake
run: ./cli.py ci_build_and_push ci/basic datacoves dbt-snowflake
shell: bash

- name: Build and deploy ci-basic dbt-redshift
run: ./cli.py ci_build_and_push ci/basic datacoves dbt-redshift
shell: bash

- name: Build and deploy ci-basic dbt-bigquery
run: ./cli.py ci_build_and_push ci/basic datacoves dbt-bigquery
shell: bash

- name: Build and deploy ci-basic dbt-databricks
run: ./cli.py ci_build_and_push ci/basic datacoves dbt-databricks
shell: bash
44 changes: 44 additions & 0 deletions .github/workflows/build_deploy_ci-multiarch.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Build and deploy ci-multiarch

concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true

on:
workflow_dispatch:
push:
branches:
- main
- prev
paths:
- src/ci/multiarch/*
- src/ci/multiarch/**/*
- requirements.txt

jobs:
build_and_deploy_ci-multiarch:
name: Build and deploy ci-multiarch
runs-on: ubuntu-latest

container: datacoves/ci-multiarch:latest

env:
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}

steps:
- name: Trust directory
run: git config --global --add safe.directory /__w/datacoves/datacoves

- name: Checkout branch
uses: actions/checkout@v3
with:
fetch-depth: 2
ref: ${{ github.event.pull_request.head.sha }}
- name: Install missing requirements
run: |
apk update && apk upgrade && apk add --no-cache python3-dev
pip install -r requirements.txt
- name: Build and deploy ci-multiarch
run: ./cli.py ci_build_and_push ci/multiarch datacoves
shell: bash
62 changes: 62 additions & 0 deletions .github/workflows/build_deploy_code-server-code-server.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
name: Build and deploy code-server-code-server

concurrency:
group: ${{ github.workflow }}
cancel-in-progress: true

on:
workflow_dispatch:
push:
branches:
- main
- prev
paths:
- src/code-server/code-server/*
- src/code-server/code-server/**/*
- src/common/*
- src/common/**/*

jobs:
build_and_deploy_code-server-code-server:
name: Build and deploy code-server-code-server
runs-on: ubuntu-latest

container: datacoves/ci-multiarch:latest

env:
DOCKER_TLS_CERTDIR: "/certs"
DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}

steps:
- name: Trust directory
run: git config --global --add safe.directory /__w/datacoves/datacoves

- name: Checkout branch
uses: actions/checkout@v3
with:
fetch-depth: 2
ref: ${{ github.event.pull_request.head.sha }}

- name: Lint
run: flake8
shell: bash

- name: Build and deploy code-server-code-server base
run: ./cli.py ci_build_and_push code-server/code-server datacovesprivate base
shell: bash

- name: Build and deploy code-server-code-server dbt-snowflake
run: ./cli.py ci_build_and_push code-server/code-server datacovesprivate dbt-snowflake
shell: bash

- name: Build and deploy code-server-code-server dbt-redshift
run: ./cli.py ci_build_and_push code-server/code-server datacovesprivate dbt-redshift
shell: bash

- name: Build and deploy code-server-code-server dbt-bigquery
run: ./cli.py ci_build_and_push code-server/code-server datacovesprivate dbt-bigquery
shell: bash

- name: Build and deploy code-server-code-server dbt-databricks
run: ./cli.py ci_build_and_push code-server/code-server datacovesprivate dbt-databricks
shell: bash
Loading
Loading