Initial release: SuperScrape v0.1.0

Web scraping + AI visual intelligence using Camoufox anti-detection browser.

Features:
- Amazon, Instagram, Reddit, eBay, Walmart, Etsy, Shopee scrapers
- GPT Vision image analysis pipeline
- Visual intelligence reports (Markdown + JSON + HTML + PDF)
- FastAPI server with SSE job tracking
- Next.js frontend
- Docker deployment
- GitHub Actions CI (Ruff + pytest + mypy)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

Author: PHY041

.dockerignore

@@ -0,0 +1,14 @@
+venv/
+.venv/
+__pycache__/
+*.pyc
+.git/
+.env
+.env.example
+frontend/
+nginx/
+docker-compose.yml
+todbiccz_*
+visual_report_*
+tests/
+examples/

.env.example

@@ -0,0 +1,5 @@
+# Required: OpenAI API key for GPT Vision analysis
+OPENAI_API_KEY=sk-proj-...
+
+# Optional: BytePlus API key for Seedream lifestyle images
+BYTEPLUSES_API_KEY=

.github/workflows/ci.yml

@@ -0,0 +1,43 @@
+name: CI
+
+on:
+  push:
+    branches: [main]
+  pull_request:
+    branches: [main]
+
+jobs:
+  lint:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install ruff
+      - run: ruff check .
+      - run: ruff format --check .
+
+  typecheck:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+      - run: pip install -e ".[dev]"
+      - run: mypy superscrape/ --ignore-missing-imports
+        continue-on-error: true
+
+  test:
+    runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        python-version: ["3.10", "3.11", "3.12"]
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+      - run: pip install -e ".[dev]"
+      - run: pytest tests/ -x --tb=short

.gitignore

@@ -0,0 +1,45 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+dist/
+build/
+.eggs/
+*.egg
+.venv/
+venv/
+
+# Generated outputs
+visual_report_*
+todbiccz_*
+
+# Client project data
+projects/
+
+# Secrets
+.env
+.env.local
+
+# Node
+node_modules/
+.next/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Testing / Coverage
+.pytest_cache/
+htmlcov/
+.coverage
+*.log
+.mypy_cache/
+
+# Git backup
+.git_backup/

CONTRIBUTING.md

@@ -0,0 +1,74 @@
+# Contributing to SuperScrape
+
+Thanks for your interest in contributing! Here's how to get started.
+
+## Development Setup
+
+```bash
+# Clone the repo
+git clone https://github.com/PHY041/superscrape.git
+cd superscrape
+
+# Create a virtual environment
+python -m venv venv
+source venv/bin/activate
+
+# Install in editable mode with dev dependencies
+pip install -e ".[dev]"
+
+# Install the Camoufox browser binary
+python -c "from camoufox.sync_api import Camoufox; print('ready')"
+```
+
+## Code Style
+
+We use [Ruff](https://docs.astral.sh/ruff/) for linting and formatting.
+
+```bash
+# Lint
+ruff check .
+
+# Format
+ruff format .
+
+# Type check
+mypy superscrape/
+```
+
+Before submitting a PR, make sure all checks pass:
+
+```bash
+ruff check . && ruff format --check . && pytest tests/ -x
+```
+
+## Running Tests
+
+```bash
+# Unit tests (no API key or browser needed)
+pytest tests/ -x
+
+# Verbose output
+pytest tests/ -xvs
+```
+
+## Adding a New Platform Scraper
+
+1. Create `superscrape/sites/your_platform.py`
+2. Inherit from `BaseScraper` in `superscrape/sites/base.py`
+3. Implement the required methods (see `amazon.py` as reference)
+4. Add a CLI command in `superscrape/cli/main.py`
+5. Add tests in `tests/test_your_platform.py`
+
+## Pull Request Process
+
+1. Fork the repo and create a feature branch
+2. Make your changes with tests
+3. Ensure `ruff check .` and `pytest tests/` pass
+4. Submit a PR with a clear description of changes
+
+## Reporting Issues
+
+Open an issue at https://github.com/PHY041/superscrape/issues with:
+- Steps to reproduce
+- Expected vs actual behavior
+- Python version and OS

Dockerfile

@@ -0,0 +1,47 @@
+FROM python:3.12-slim
+
+# System deps for Camoufox (headless Firefox) + WeasyPrint
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    xvfb \
+    xauth \
+    curl \
+    fonts-noto-cjk \
+    libglib2.0-0 \
+    libnss3 \
+    libnspr4 \
+    libdbus-1-3 \
+    libatk1.0-0 \
+    libatk-bridge2.0-0 \
+    libatspi2.0-0 \
+    libdrm2 \
+    libxcomposite1 \
+    libxdamage1 \
+    libxfixes3 \
+    libxrandr2 \
+    libgbm1 \
+    libpango-1.0-0 \
+    libcairo2 \
+    libasound2 \
+    libgtk-3-0 \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /app
+
+COPY pyproject.toml README.md ./
+COPY superscrape/ superscrape/
+COPY api/ api/
+
+# Install Python deps (non-editable for Docker)
+RUN pip install --no-cache-dir .
+
+# Download Camoufox browser binary
+RUN python -c "from camoufox.sync_api import Camoufox; print('Camoufox binary ready')" || true
+
+EXPOSE 8001
+
+HEALTHCHECK --interval=30s --timeout=5s --start-period=15s --retries=3 \
+    CMD curl -f http://localhost:8001/health || exit 1
+
+# xvfb-run provides virtual display for headless Camoufox
+CMD xvfb-run --auto-servernum --server-args="-screen 0 1280x720x24" \
+    uvicorn api.main:app --host 0.0.0.0 --port 8001

LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Haoyang Pang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.