Skip to content

fix: improve Windows/WSL compatibility, add multi-stage executor Dockerfile, and add http to https fallback for git clone #740

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
SSwser wants to merge 4 commits into
base: main
Choose a base branch
from
Open

fix: improve Windows/WSL compatibility, add multi-stage executor Dockerfile, and add http to https fallback for git clone #740

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
32e036f
fix: improve Windows/WSL local dev compatibility
SSwser Mar 12, 2026
1c77fa6
feat: add multi-stage Dockerfile for executor with source/binary targets
SSwser Mar 12, 2026
e06e170
feat: add http to https fallback for git clone with token
SSwser Mar 12, 2026
3a4caec
fix(config): resolve Windows/WSL path compatibility in Next.js config
SSwser Mar 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions build_image.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,8 @@ docker buildx build --network=host ${PUSH_FLAG} --platform linux/amd64,linux/arm
# Build frontend image
docker buildx build --network=host ${PUSH_FLAG} --platform linux/amd64,linux/arm64 -t ghcr.io/wecode-ai/wegent-web:${VERSION} -f docker/frontend/Dockerfile .

# Build executor image
docker buildx build --network=host ${PUSH_FLAG} --platform linux/amd64,linux/arm64 -t ghcr.io/wecode-ai/wegent-executor:${VERSION} -f docker/executor/Dockerfile .
# Build executor image (default target: binary)
docker buildx build --network=host ${PUSH_FLAG} --platform linux/amd64,linux/arm64 --target binary -t ghcr.io/wecode-ai/wegent-executor:${VERSION} -f docker/executor/Dockerfile .

# Build executor manager image
docker buildx build --network=host ${PUSH_FLAG} --platform linux/amd64,linux/arm64 -t ghcr.io/wecode-ai/wegent-executor-manager:${VERSION} -f docker/executor_manager/Dockerfile .
Expand Down
4 changes: 2 additions & 2 deletions build_image_mac.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -83,8 +83,8 @@ docker build --network=host ${PUSH_FLAG} -t ghcr.io/wecode-ai/wegent-backend:${V
# Build frontend image
docker build --network=host ${PUSH_FLAG} -t ghcr.io/wecode-ai/wegent-web:${VERSION} -f docker/frontend/Dockerfile .

# Build executor image
docker build --network=host ${PUSH_FLAG} -t ghcr.io/wecode-ai/wegent-executor:${VERSION} -f docker/executor/Dockerfile .
# Build executor image (default target: binary)
docker build --network=host ${PUSH_FLAG} --target binary -t ghcr.io/wecode-ai/wegent-executor:${VERSION} -f docker/executor/Dockerfile .

# Build executor manager image
docker build --network=host ${PUSH_FLAG} -t ghcr.io/wecode-ai/wegent-executor-manager:${VERSION} -f docker/executor_manager/Dockerfile .
Expand Down
67 changes: 39 additions & 28 deletions docker/executor/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,28 +2,10 @@
#
# SPDX-License-Identifier: Apache-2.0

# Build stage: compile executor to standalone binary
FROM ghcr.io/wecode-ai/wegent-base-python3.12:latest AS builder

WORKDIR /app

# Copy pyproject.toml and install dependencies
COPY executor/pyproject.toml /app/executor/pyproject.toml

RUN cd /app/executor && uv pip install --system --no-cache -r pyproject.toml

# Install PyInstaller for building standalone binary (using uv for speed)
RUN uv pip install --system pyinstaller

# Copy source code
COPY executor /app/executor
COPY shared /app/shared

# Build standalone binary
RUN cd /app/executor && bash build.sh

# Runtime stage: minimal image with only the binary
FROM ghcr.io/wecode-ai/wegent-base-python3.12:latest
# ============================================================================
# Stage 1: deps — system packages + Python dependencies (shared by all targets)
# ============================================================================
FROM ghcr.io/wecode-ai/wegent-base-python3.12:latest AS deps

WORKDIR /app

Expand All @@ -42,14 +24,12 @@ RUN dnf install -y \
&& rm -rf /var/cache/dnf/*

# Install Claude marketplace skills and npm packages for document generation
# Install to /root so it's available in the base image
RUN claude plugin marketplace add anthropics/skills \
&& npm install -g pptxgenjs playwright sharp react react-dom react-icons docx pdf-lib \
&& npx playwright install chromium \
&& npm cache clean --force

# Install Python packages for document generation
# These are commonly used by document skills
RUN pip install python-pptx openpyxl python-docx reportlab Pillow pandas \
pypdf pdfplumber pytesseract \
"markitdown[pptx]" defusedxml weasyprint \
Expand All @@ -66,13 +46,44 @@ RUN mkdir -p /usr/share/fonts/truetype/noto && \
-o /usr/share/fonts/truetype/noto/NotoSansSC.ttf) && \
fc-cache -f -v


# Copy only the standalone binary from builder
COPY --from=builder /app/executor/dist/executor /app/executor
# Install Python executor dependencies (cached unless pyproject.toml changes)
COPY executor/pyproject.toml /app/executor/pyproject.toml
RUN cd /app/executor && uv pip install --system --no-cache -r pyproject.toml
Comment on lines +49 to +51
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Mar 16, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "1) Check whether executor declares local shared dependency"
rg -n '(\.\./shared|shared\s*@\s*file:|path\s*=\s*"\.\./shared"|tool\.uv\.sources)' executor/pyproject.toml || true

echo
echo "2) Inspect deps install order in docker/executor/Dockerfile"
sed -n '46,55p' docker/executor/Dockerfile

echo
echo "3) Compare with standalone install order"
sed -n '96,106p' docker/standalone/Dockerfile

Repository: wecode-ai/Wegent

Length of output: 1061

🏁 Script executed:

cat executor/pyproject.toml

Repository: wecode-ai/Wegent

Length of output: 2758

🏁 Script executed:

head -55 docker/executor/Dockerfile | tail -20

Repository: wecode-ai/Wegent

Length of output: 1019

🏁 Script executed:

rg -n 'from shared|import shared' executor/

Repository: wecode-ai/Wegent

Length of output: 13240

🏁 Script executed:

grep -r "shared" executor/pyproject.toml

Repository: wecode-ai/Wegent

Length of output: 175

Executor code imports from shared but pyproject.toml doesn't declare it—this causes runtime failures.

The executor extensively imports from shared module (logger, models, status, telemetry, utils), but executor/pyproject.toml does not declare a dependency on shared. At lines 49-51, when executor deps are installed in the deps stage, the shared module is not yet available, causing ImportError at runtime.

Proposed Docker fix 
 COPY executor/pyproject.toml /app/executor/pyproject.toml
-RUN cd /app/executor && uv pip install --system --no-cache -r pyproject.toml
+COPY shared /app/shared
+RUN uv pip install --system --no-cache /app/shared \
+    && cd /app/executor \
+    && uv pip install --system --no-cache -r pyproject.toml

Better fix: Add shared as a dependency in executor/pyproject.toml (e.g., "wegent-shared = { path = "../shared" }), matching the pattern used in backend and other services.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@docker/executor/Dockerfile` around lines 49 - 51, Add the local shared
package as a dependency in executor/pyproject.toml (e.g., add an entry like
"wegent-shared = { path = \"../shared\" }" in the dependencies section so the
executor can import logger/models/status/telemetry/utils), and ensure the Docker
build installs that local path by making the deps stage include the shared
source before installing (adjust the Dockerfile deps stage to COPY the shared
directory into the image or otherwise make the ../shared path available during
the RUN pip install step).


ENV PORT=10001
ENV PYTHONPATH=/app
ENV NODE_PATH=/usr/lib/node_modules

# Run the standalone binary
# ============================================================================
# Stage 2: builder — compiles executor Python source to standalone binary
# ============================================================================
FROM deps AS builder

# Install PyInstaller
RUN uv pip install --system pyinstaller

# Copy source code and build binary
COPY executor /app/executor
COPY shared /app/shared
RUN cd /app/executor && bash build.sh
Comment on lines +57 to +68
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify hidden imports in executor.spec vs build_local.py

echo "=== Checking executor.spec for hidden imports ==="
rg -n "hiddenimports|hidden-import" executor/executor.spec || echo "No hiddenimports found in spec"

echo ""
echo "=== Checking build_local.py for hidden imports ==="
rg -n "hidden-import" executor/scripts/build_local.py | head -30

echo ""
echo "=== Checking runtime_hooks in executor.spec ==="
rg -n "runtime_hooks" executor/executor.spec

echo ""
echo "=== Checking if rthook_version.py exists ==="
fd "rthook_version.py" executor/

Repository: wecode-ai/Wegent

Length of output: 4284

🏁 Script executed:

#!/bin/bash
# Check for specific missing dependencies in executor.spec
echo "=== Checking for grpc, protobuf, watchdog, psutil in executor.spec ==="
rg -n "grpc|protobuf|watchdog|psutil" executor/executor.spec || echo "None found"

echo ""
echo "=== Checking executor pyproject.toml for these dependencies ==="
rg -n "grpc|protobuf|watchdog|psutil|socketio|aiohttp" executor/pyproject.toml

echo ""
echo "=== Verify if rthook_version.py is intended to be used ==="
head -20 executor/hooks/rthook_version.py

Repository: wecode-ai/Wegent

Length of output: 1133

Critical: PyInstaller spec is missing hidden imports for installed runtime dependencies.

The builder stage uses executor.spec for PyInstaller, but several dependencies installed in pyproject.toml are not declared in the spec's hiddenimports:

  1. socketio & aiohttp — Installed at lines 54-55 of pyproject.toml; required for local mode WebSocket communication
  2. grpcio, grpcio-tools & protobuf — Installed at lines 36-38; required for envd filesystem service
  3. watchdog — Installed at line 39; required for filesystem monitoring
  4. psutil — Installed at line 40; required for system metrics

Additionally, runtime_hooks=[] is empty in the spec, but executor/hooks/rthook_version.py exists and is designed to handle the --version flag before module initialization. This hook should be registered.

The Docker image will build successfully but the binary may fail at runtime when these modules are dynamically imported. Note that executor/scripts/build_local.py does include these hidden imports (e.g., lines 345-353 for socketio/aiohttp), showing this is a known pattern.

Update executor.spec to include the missing hidden imports and register the runtime hook.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@docker/executor/Dockerfile` around lines 57 - 68, The PyInstaller spec
(executor.spec) is missing hiddenimports and the runtime hook; update
executor.spec to add the runtime hidden imports for modules installed at runtime
(at minimum: "socketio", "aiohttp", "grpcio", "grpcio_tools" or "grpc",
"protobuf", "watchdog", and "psutil") into the spec's hiddenimports list (follow
the pattern used in executor/scripts/build_local.py for socketio/aiohttp), and
register the runtime hook by adding "executor/hooks/rthook_version.py" to the
spec's runtime_hooks array so the --version hook runs before module
initialization; ensure names match the importable package names used at runtime
and keep the spec consistent with build_local.py examples.


# ============================================================================
# Stage 3: source — dev target, runs Python source directly (no compilation)
# docker build --target source -t wegent-executor:local-dev .
# ============================================================================
FROM deps AS source

COPY executor /app/executor
COPY shared /app/shared

CMD ["python", "/app/executor/main.py"]

# ============================================================================
# Stage 4: binary — default target, runs compiled standalone binary
# docker build -t wegent-executor:latest .
# ============================================================================
FROM deps AS binary

COPY --from=builder /app/executor/dist/executor /app/executor

CMD ["/app/executor"]
115 changes: 61 additions & 54 deletions frontend/next.config.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,16 +5,20 @@
// eslint-disable-next-line @typescript-eslint/no-require-imports
const path = require('path')

// Check if running with Turbopack (development mode with --turbopack flag)
const isTurbopack = process.env.TURBOPACK === '1'
// Resolve path with POSIX-style separators for WSL/Windows compatibility
// Using path.resolve can produce Windows-style paths (D:\...) in WSL environments
// which webpack cannot handle. Converting to forward slashes fixes this.
function resolveAlias(relativePath) {
return path.resolve(__dirname, relativePath).split(path.sep).join('/')
}

/** @type {import('next').NextConfig} */
const nextConfig = {
reactStrictMode: false,
output: 'standalone',
// Allow cross-origin requests in development mode
// This prevents "Cross origin request detected" warning
allowedDevOrigins: ['localhost:3000'],
allowedDevOrigins: ['localhost', '127.0.0.1'],
// Transpile node_modules that ship modern JS syntax for iOS 16 Safari compatibility
transpilePackages: [
'mermaid',
Expand All @@ -30,60 +34,63 @@ const nextConfig = {
'@replit/codemirror-vim',
'katex',
],
// Webpack configuration for production builds
// Note: In development mode with Turbopack, this is not used
// The warning "Webpack is configured while Turbopack is not" can be safely ignored
// as these optimizations are primarily for production builds which use webpack
...(isTurbopack
? {}
: {
webpack: (config, { isServer, _dev }) => {
// Force replace remark-gfm with our iOS 16 compatible version
// This is needed because @uiw/react-md-editor depends on remark-gfm
// which uses lookbehind regex not supported by iOS 16
config.resolve.alias = {
...config.resolve.alias,
'remark-gfm': path.resolve(__dirname, 'src/lib/remark-gfm-safe.ts'),
}
// Turbopack configuration (development mode)
// Mirrors the remark-gfm alias from webpack config for iOS 16 Safari compatibility
// NOTE: Turbopack resolveAlias requires a project-relative path (starting with ./)
// NOT an absolute path — absolute paths (even with forward slashes) are treated as
// external modules which causes "chunking context does not support external modules"
turbopack: {
resolveAlias: {
'remark-gfm': './src/lib/remark-gfm-safe.ts',
},
},
// Webpack configuration (production builds)
webpack: (config, { isServer, _dev }) => {
// Force replace remark-gfm with our iOS 16 compatible version
// This is needed because @uiw/react-md-editor depends on remark-gfm
// which uses lookbehind regex not supported by iOS 16
config.resolve.alias = {
...config.resolve.alias,
'remark-gfm': resolveAlias('src/lib/remark-gfm-safe.ts'),
}

// Handle chunk loading issues
config.optimization = {
...config.optimization,
// Prevent over-aggressive code splitting that can cause chunk loading errors
splitChunks: {
...config.optimization?.splitChunks,
chunks: 'all',
cacheGroups: {
vendor: {
test: /[\\/]node_modules[\\/]/,
name: 'vendors',
chunks: 'all',
priority: 10,
},
common: {
name: 'common',
minChunks: 2,
chunks: 'all',
priority: 5,
},
},
},
// Enable module concatenation to reduce bundle size
concatenateModules: true,
}
// Handle chunk loading issues
config.optimization = {
...config.optimization,
// Prevent over-aggressive code splitting that can cause chunk loading errors
splitChunks: {
...config.optimization?.splitChunks,
chunks: 'all',
cacheGroups: {
vendor: {
test: /[\\/]node_modules[\\/]/,
name: 'vendors',
chunks: 'all',
priority: 10,
},
common: {
name: 'common',
minChunks: 2,
chunks: 'all',
priority: 5,
},
},
},
// Enable module concatenation to reduce bundle size
concatenateModules: true,
}

// Handle dynamic imports more gracefully
if (!isServer) {
config.resolve.fallback = {
...config.resolve.fallback,
fs: false,
path: false,
}
}
// Handle dynamic imports more gracefully
if (!isServer) {
config.resolve.fallback = {
...config.resolve.fallback,
fs: false,
path: false,
}
}

return config
},
}),
return config
},
// Experimental features to improve stability
experimental: {
// Disable CSS chunking to fix Safari/iOS bug where CSS files
Expand Down
2 changes: 1 addition & 1 deletion frontend/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "TURBOPACK=1 next dev --turbopack",
"dev": "next dev --turbopack",
"build": "NODE_OPTIONS='--max-old-space-size=2048' next build",
"start": "next start",
"lint": "next lint",
Expand Down
1 change: 1 addition & 0 deletions frontend/public/mockServiceWorker.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Revert the modification to this generated file.

This file contains an explicit warning: "Please do NOT modify this file." The added blank line appears to be an unintentional side effect of line ending normalization during the Windows/WSL compatibility fixes. Generated library files should remain pristine to avoid merge conflicts during updates and to maintain alignment with the upstream MSW library version.

🔄 Suggested fix

Remove the blank line at the start of the file to restore the original generated content:

-
 /* tslint:disable */
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/* tslint:disable */
🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@frontend/public/mockServiceWorker.js` at line 1, Revert the accidental edit
to the generated mockServiceWorker.js by removing the leading blank line so the
file exactly matches the upstream MSW output; locate the generated file
(mockServiceWorker.js) and restore the original content ensuring the warning
string "Please do NOT modify this file." remains at the very top with no
preceding newline, then commit the restored file so it stays identical to the
library-provided version.

/* tslint:disable */

/**
Expand Down
12 changes: 11 additions & 1 deletion shared/utils/git_util.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,9 +72,19 @@ def clone_repo(project_url, branch, project_path, user_name=None, token=None):
f"get git token from url: {project_url}, branch:{branch}, project:{project_path}"
)
if token:
return clone_repo_with_token(
success, error = clone_repo_with_token(
project_url, branch, project_path, user_name, token
)
# If http:// clone failed, retry with https:// in case the server enforces HTTPS
if not success and project_url.startswith("http://"):
https_url = "https://" + project_url[7:]
logger.info(
f"http clone failed, retrying with https: {https_url}"
)
success, error = clone_repo_with_token(
https_url, branch, project_path, user_name, token
)
return success, error
return False, "Token is not provided"


Expand Down
Loading