-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
feature: add Dockerfile.maximal to optimize CI (#1252) #1339
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
tirthpatel90
wants to merge
16
commits into
oraios:main
from
tirthpatel90:feature/optimize-docker-ci
Closed
Changes from 15 commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
946027f
Testing maximal docker build on GH servers
tirthpatel90 0ccb617
Added Phase 2 languages to maximal image
tirthpatel90 838d650
Fix Julia installation path in Dockerfile
tirthpatel90 e7577a9
Fix Julia executable stack security issue using execstack
tirthpatel90 9d3699b
Bump Julia version to 1.10.4 to resolve openlibm executable stack bug
tirthpatel90 2c06bc9
Complete unified Dockerfile.maximal with multi-stage Julia and R headers
tirthpatel90 0a347ab
Force-install execstack from archives to bypass Debian 12 Docker secu…
tirthpatel90 900ee9e
Fix Julia install using proper Docker multi-stage build
tirthpatel90 90749bb
Fix Julia executable stack bug using Ubuntu prelink archive
tirthpatel90 7df0863
Optimize Julia memory footprint and switch to pre-compiled R binaries…
tirthpatel90 9c0425f
Disable Julia auto-precompilation and set noninteractive frontend to …
tirthpatel90 b352519
chore: remove test workflow file
tirthpatel90 6a6a047
feat: Add OCaml support to maximal image
tirthpatel90 8447e98
ci: add workflow to publish maximal image to ghcr
tirthpatel90 281c890
ci: add workflow to test maximal docker image speed
tirthpatel90 c51f896
ci: fix github advanced security permission warnings
tirthpatel90 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| name: Publish Maximal Docker Image | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - feature/optimize-docker-ci | ||
| workflow_dispatch: # for manual run from GitHub UI | ||
|
|
||
| jobs: | ||
| build-and-push-image: | ||
| runs-on: ubuntu-latest | ||
|
|
||
| # permission to upload the image to GitHub Container Registry | ||
| permissions: | ||
| contents: read | ||
| packages: write | ||
|
|
||
| steps: | ||
| - name: Checkout repository | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Set up Docker Buildx | ||
| uses: docker/setup-buildx-action@v3 | ||
|
|
||
| - name: Log in to GitHub Container Registry | ||
| uses: docker/login-action@v3 | ||
| with: | ||
| registry: ghcr.io | ||
| username: ${{ github.actor }} | ||
| password: ${{ secrets.GITHUB_TOKEN }} | ||
|
|
||
| - name: Build and push Docker image | ||
| uses: docker/build-push-action@v5 | ||
| with: | ||
| context: . | ||
| file: Dockerfile.maximal | ||
| push: true | ||
| # this will tag the image as latest, you can also add more tags if needed | ||
| tags: ghcr.io/tirthpatel90/serena-maximal:latest | ||
| # Caching on for future runs, this will speed up the build process by reusing layers | ||
| cache-from: type=gha | ||
| cache-to: type=gha,mode=max |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| name: Test with Maximal Image | ||
|
|
||
| on: | ||
| push: | ||
| branches: | ||
| - feature/optimize-docker-ci | ||
| workflow_dispatch: | ||
|
|
||
| jobs: | ||
| fast-docker-test: | ||
| name: Linux Tests (Containerized) | ||
| runs-on: ubuntu-latest | ||
|
|
||
| container: | ||
| image: ghcr.io/tirthpatel90/serena-maximal:latest | ||
|
|
||
| steps: | ||
| - name: Checkout code | ||
| uses: actions/checkout@v4 | ||
|
|
||
| - name: Quick Workspace Sync | ||
| shell: bash | ||
| run: | | ||
| # Sirf code update karne ke liye fast sync (takes 2 seconds) | ||
| uv sync --extra dev --locked | ||
| uv pip list | ||
|
|
||
| - name: Test with pytest (Lightning Fast) | ||
| shell: bash | ||
| run: uv run poe test -q --tb=short | ||
|
|
||
| - name: Type-checking with mypy | ||
| shell: bash | ||
| run: uv run poe type-check | ||
Check warningCode scanning / CodeQL Workflow does not contain permissions Medium test
Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,146 @@ | ||
| # --------------------------------------------------------- | ||
| # STAGE 1: The Maximal Base (All dependencies baked in) | ||
| # --------------------------------------------------------- | ||
| FROM python:3.11-slim AS base | ||
| SHELL ["/bin/bash", "-c"] | ||
|
|
||
| ENV PYTHONUNBUFFERED=1 | ||
| ENV PYTHONDONTWRITEBYTECODE=1 | ||
|
|
||
| # Prevent apt-get from hanging indefinitely on interactive prompts | ||
| ARG DEBIAN_FRONTEND=noninteractive | ||
|
|
||
| # 1. Install ALL system dependencies together | ||
| RUN apt-get update && apt-get install -y --no-install-recommends \ | ||
| curl build-essential git ssh sudo wget zip unzip \ | ||
| sed \ | ||
| ccls \ | ||
| fpc fpc-source \ | ||
| libcurl4-openssl-dev libssl-dev libicu-dev zlib1g-dev \ | ||
| pkg-config libuv1-dev libxml2-dev \ | ||
| cpanminus libanyevent-perl libio-aio-perl \ | ||
| default-jdk \ | ||
| ruby-full \ | ||
| golang-go \ | ||
| r-base \ | ||
| opam \ | ||
| && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # 2. Install pipx | ||
| RUN python3 -m pip install --no-cache-dir pipx | ||
| ENV PATH="/root/.local/bin:${PATH}" | ||
|
|
||
| # 3. Install Julia (Bypass Precompilation Hang) | ||
| ENV JULIA_NUM_THREADS=1 | ||
| ENV JULIA_CPU_THREADS=1 | ||
| ENV JULIA_PKG_PRECOMPILE_AUTO=0 | ||
| RUN curl -L https://julialang-s3.julialang.org/bin/linux/x64/1.10/julia-1.10.4-linux-x86_64.tar.gz -o julia.tar.gz \ | ||
| && mkdir -p /opt/julia \ | ||
| && tar -xzf julia.tar.gz -C /opt/julia --strip-components=1 \ | ||
| && rm julia.tar.gz \ | ||
| && wget http://archive.ubuntu.com/ubuntu/pool/universe/p/prelink/execstack_0.0.20131005-1_amd64.deb \ | ||
| && dpkg -i execstack_0.0.20131005-1_amd64.deb \ | ||
| && rm execstack_0.0.20131005-1_amd64.deb \ | ||
| && execstack -c /opt/julia/lib/julia/libopenlibm.so \ | ||
| && execstack -c /opt/julia/lib/julia/sys.so || true \ | ||
| && ln -s /opt/julia/bin/julia /usr/local/bin/julia \ | ||
| && julia -e 'using Pkg; Pkg.add("LanguageServer")' | ||
|
|
||
| # 4. Install R Language Server (Using Pre-compiled Binaries) | ||
| RUN Rscript -e "install.packages('languageserver', repos='https://packagemanager.posit.co/cran/__linux__/bookworm/latest')" | ||
|
|
||
| # 5. Install Node.js and Elm | ||
| ENV NVM_VERSION=0.40.3 | ||
| ENV NODE_VERSION=22.18.0 | ||
| RUN curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v${NVM_VERSION}/install.sh | bash | ||
| ENV NVM_DIR=/root/.nvm | ||
| RUN . "$NVM_DIR/nvm.sh" && nvm install ${NODE_VERSION} \ | ||
| && nvm use v${NODE_VERSION} \ | ||
| && nvm alias default v${NODE_VERSION} \ | ||
| && npm install -g [email protected] | ||
| ENV PATH="${NVM_DIR}/versions/node/v${NODE_VERSION}/bin/:${PATH}" | ||
|
|
||
| # 6. Install uv | ||
| RUN curl -LsSf https://astral.sh/uv/install.sh | sh | ||
|
|
||
| # 7. Install Rust | ||
| ENV RUSTUP_HOME=/usr/local/rustup | ||
| ENV CARGO_HOME=/usr/local/cargo | ||
| ENV PATH="${CARGO_HOME}/bin:${PATH}" | ||
| RUN curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \ | ||
| --default-toolchain stable \ | ||
| --profile minimal \ | ||
| && rustup component add rust-analyzer | ||
|
|
||
| # 8. Install Go gopls | ||
| RUN go install golang.org/x/tools/gopls@latest | ||
| ENV PATH="/root/go/bin:${PATH}" | ||
|
|
||
| # 9. Install Ruby LSP | ||
| RUN gem install ruby-lsp | ||
|
|
||
| # 10. Install Perl Language Server | ||
| RUN PERL_MM_USE_DEFAULT=1 cpanm --notest --force Perl::LanguageServer | ||
|
|
||
| # 11. Install Terraform | ||
| RUN wget https://releases.hashicorp.com/terraform/1.5.0/terraform_1.5.0_linux_amd64.zip \ | ||
| && unzip terraform_1.5.0_linux_amd64.zip \ | ||
| && mv terraform /usr/local/bin/ \ | ||
| && rm terraform_1.5.0_linux_amd64.zip | ||
|
|
||
| # 12. Install Zig and ZLS | ||
| RUN wget https://github.com/zigtools/zls/releases/download/0.14.0/zls-x86_64-linux.tar.xz \ | ||
| && tar -xf zls-x86_64-linux.tar.xz \ | ||
| && mv zls /usr/local/bin/ \ | ||
| && rm zls-x86_64-linux.tar.xz \ | ||
| && wget https://ziglang.org/download/0.14.0/zig-linux-x86_64-0.14.0.tar.xz \ | ||
| && tar -xf zig-linux-x86_64-0.14.0.tar.xz -C /usr/local --strip-components=1 \ | ||
| && rm zig-linux-x86_64-0.14.0.tar.xz | ||
|
|
||
| # 13. Install Lua Language Server | ||
| ENV LUA_LS_VERSION="3.15.0" | ||
| RUN mkdir -p /opt/lua-ls \ | ||
| && wget https://github.com/LuaLS/lua-language-server/releases/download/${LUA_LS_VERSION}/lua-language-server-${LUA_LS_VERSION}-linux-x64.tar.gz \ | ||
| && tar -xzf lua-language-server-${LUA_LS_VERSION}-linux-x64.tar.gz -C /opt/lua-ls \ | ||
| && rm lua-language-server-${LUA_LS_VERSION}-linux-x64.tar.gz \ | ||
| && chmod +x /opt/lua-ls/bin/lua-language-server \ | ||
| && ln -s /opt/lua-ls/bin/lua-language-server /usr/local/bin/lua-language-server | ||
|
|
||
| # 14. Install Regal (Rego Language Server) | ||
| ENV REGAL_VERSION="0.39.0" | ||
| RUN curl -L -o regal https://github.com/StyraInc/regal/releases/download/v${REGAL_VERSION}/regal_Linux_x86_64 \ | ||
| && chmod +x regal \ | ||
| && mv regal /usr/local/bin/ | ||
|
|
||
| # 15. Install OCaml, Dune, and OCaml-LSP-Server | ||
| # Disable sandboxing for Docker compatibility | ||
| RUN opam init -a -y --disable-sandboxing \ | ||
| && eval $(opam env) \ | ||
| && opam install -y dune 'ocaml-lsp-server>=1.23.0' | ||
| ENV PATH="/root/.opam/default/bin:${PATH}" | ||
|
|
||
| # --------------------------------------------------------- | ||
| # STAGE 2: Serena App Setup | ||
| # --------------------------------------------------------- | ||
| WORKDIR /workspaces/serena | ||
|
|
||
| # Copy all files for development | ||
| COPY . /workspaces/serena/ | ||
|
|
||
| # Create Serena configuration | ||
| ENV SERENA_HOME=/workspaces/serena/config | ||
| RUN mkdir -p $SERENA_HOME | ||
| RUN cp src/serena/resources/serena_config.template.yml $SERENA_HOME/serena_config.yml | ||
| RUN sed -i 's/^gui_log_window: .*/gui_log_window: False/' $SERENA_HOME/serena_config.yml | ||
| RUN sed -i 's/^web_dashboard_listen_address: .*/web_dashboard_listen_address: 0.0.0.0/' $SERENA_HOME/serena_config.yml | ||
| RUN sed -i 's/^web_dashboard_open_on_launch: .*/web_dashboard_open_on_launch: False/' $SERENA_HOME/serena_config.yml | ||
|
|
||
| # Create virtual environment and install dependencies | ||
| RUN uv venv | ||
| RUN . .venv/bin/activate \ | ||
| && uv pip install -r pyproject.toml -e . \ | ||
| && uv pip install ansible-core ansible-lint | ||
|
|
||
| ENV PATH="/workspaces/serena/.venv/bin:${PATH}" | ||
|
|
||
| ENTRYPOINT ["/bin/bash", "-c", "source .venv/bin/activate && $0 $@"] |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.