Skip to content

Commit cfab554

Browse files
committed
refactor: 取消content分支并更新抓取流程
1 parent 84e8618 commit cfab554

43 files changed

Lines changed: 2308 additions & 57 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/deploy.yml

Lines changed: 11 additions & 20 deletions
Original file line numberDiff line numberDiff line change
@@ -6,10 +6,12 @@ on:
66
paths:
77
- "starlight/**"
88
- ".github/workflows/deploy.yml"
9-
repository_dispatch:
10-
types: [content-updated]
119
workflow_dispatch:
1210

11+
concurrency:
12+
group: github-pages
13+
cancel-in-progress: true
14+
1315
permissions:
1416
contents: read
1517
pages: write
@@ -19,29 +21,18 @@ jobs:
1921
build:
2022
runs-on: ubuntu-latest
2123
steps:
22-
- name: Checkout Main (Scripts & Starlight)
23-
uses: actions/checkout@v6
24-
25-
- name: Checkout Content
24+
- name: Checkout Main
2625
uses: actions/checkout@v6
27-
with:
28-
ref: content
29-
path: content
30-
fetch-depth: 0
3126

32-
- name: Sync Content to Starlight
27+
- name: Validate content tree
3328
run: |
34-
TARGET="starlight/src/content/docs"
35-
SOURCE="content/docs"
36-
37-
mkdir -p "$TARGET"
38-
39-
if [ -d "$SOURCE" ]; then
40-
cp -r "$SOURCE"/* "$TARGET"/
29+
if [ ! -d "starlight/src/content/docs" ]; then
30+
echo "::error::Missing content docs directory: starlight/src/content/docs"
31+
exit 1
4132
fi
4233
43-
echo "Content synced:"
44-
ls -R starlight/src/content/docs | head -n 30
34+
echo "Content ready at starlight/src/content/docs"
35+
find starlight/src/content/docs -maxdepth 2 -type f | head -n 30
4536
4637
- name: Install, build, and upload site
4738
uses: withastro/action@v5

.gitignore

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,5 +36,3 @@ yarn-error.log*
3636
*.temp
3737
.cache/
3838

39-
# Content
40-
starlight/src/content/

README.md

Lines changed: 8 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -12,17 +12,14 @@
1212

1313
## 结构
1414

15-
本项目采用 **Git Worktree** 实现代码与内容的分离
15+
本项目采用 **单分支结构**,代码与文档内容统一维护在 `main` 分支中
1616

17-
- **Main 分支** (`/`): 存放项目基础设施代码。
18-
- `scripts/`: Python 爬虫核心代码。
19-
- `crawl.py`: 爬虫入口。
20-
- `wiki_configs.json`: 爬虫源配置。
21-
- `starlight/`: 文档网站前端项目 (Astro)。
22-
- `TRANSLATION_PROMPT.md`: 翻译规范与术语表。
23-
24-
- **Content 分支** (挂载于 `starlight/src/content/`): 存放所有文档数据。
25-
- `docs/`: Markdown 文档源文件(爬虫输出目标)。
17+
- `scripts/`: Python 爬虫核心代码。
18+
- `crawl.py`: 爬虫入口。
19+
- `wiki_configs.json`: 爬虫源配置。
20+
- `starlight/`: 文档网站前端项目 (Astro)。
21+
- `src/content/docs/`: Markdown 文档源文件(爬虫输出目标)。
22+
- `TRANSLATION_PROMPT.md`: 翻译规范与术语表。
2623

2724
## 开始
2825

@@ -36,15 +33,9 @@
3633

3734
### 2. 克隆仓库
3835

39-
由于本项目采用 **Git Worktree** 管理文档内容(代码与内容分离),请严格按照以下步骤克隆与初始化:
40-
4136
```bash
42-
# 克隆主仓库
4337
git clone https://github.com/Ziphyrien/Plugins-Wiki.git
4438
cd Plugins-Wiki
45-
46-
# 挂载 content 分支到 starlight/src/content
47-
git worktree add starlight/src/content content
4839
```
4940

5041
### 3. 安装依赖
@@ -80,7 +71,7 @@ python crawl.py coinsengine
8071
python crawl.py all
8172
```
8273

83-
> **注意**: 爬取的内容会自动输出到 Worktree 挂载的 `starlight/src/content/docs` 目录中。
74+
> **注意**: 爬取的内容会直接输出到仓库内的 `starlight/src/content/docs` 目录中。
8475
8576
### 5. 本地预览
8677

scripts/core/base_crawler.py

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,10 +13,9 @@
1313
class BaseCrawler(ABC):
1414
"""Abstract base class for all wiki crawlers."""
1515

16-
def __init__(self, name: str, config: dict, content_root: Path):
16+
def __init__(self, name: str, config: dict):
1717
self.name = name
1818
self.config = config
19-
self.content_root = Path(content_root)
2019
self.base_url = config["base_url"].rstrip("/")
2120
self.output_dir = Path(config["output_dir"])
2221
self.session = requests.Session()
@@ -39,12 +38,17 @@ def run(self) -> tuple[int, int]:
3938
Main entry point. Returns (success_count, fail_count).
4039
"""
4140
logger.info("=" * 60)
42-
logger.info(f"文档同步工具 - {self.name} ({self.__class__.__name__})")
41+
logger.info(f"文档抓取工具 - {self.name} ({self.__class__.__name__})")
4342
logger.info("=" * 60)
4443

4544
try:
4645
pages = self.fetch_pages()
4746
total = len(pages)
47+
48+
if total == 0:
49+
logger.warning("⚠️ 未发现可抓取页面")
50+
return 0, 0
51+
4852
logger.info(f"📄 发现 {total} 个页面")
4953
logger.info("-" * 60)
5054

@@ -56,7 +60,7 @@ def run(self) -> tuple[int, int]:
5660
failed += 1
5761

5862
logger.info("-" * 60)
59-
logger.info(f"📊 同步完成: {success} 成功, {failed} 失败")
63+
logger.info(f"📊 抓取完成: {success} 成功, {failed} 失败")
6064
return success, failed
6165

6266
except Exception as e:

scripts/core/crawler_gitbook.py

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,6 @@
22
GitBook crawler implementation.
33
"""
44
import xml.etree.ElementTree as ET
5-
from pathlib import Path
65
from urllib.parse import unquote
76
import logging
87

@@ -16,8 +15,8 @@
1615
class GitBookCrawler(BaseCrawler):
1716
"""Crawler for GitBook-based wikis using sitemap."""
1817

19-
def __init__(self, name: str, config: dict, content_root: Path):
20-
super().__init__(name, config, content_root)
18+
def __init__(self, name: str, config: dict):
19+
super().__init__(name, config)
2120
self.sitemap_url = config["sitemap_url"]
2221

2322
def fetch_pages(self) -> list[dict]:

scripts/core/crawler_retype.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -98,8 +98,8 @@ def convert_img(self, el, text, parent_tags):
9898
class RetypeCrawler(BaseCrawler):
9999
"""Crawler for Retype-based documentation sites."""
100100

101-
def __init__(self, name: str, config: dict, content_root: Path):
102-
super().__init__(name, config, content_root)
101+
def __init__(self, name: str, config: dict):
102+
super().__init__(name, config)
103103
self.config_url = config["config_url"]
104104

105105
def fetch_pages(self) -> list[dict]:

scripts/crawl.py

Lines changed: 63 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -4,6 +4,7 @@
44
"""
55
import argparse
66
import logging
7+
import shutil
78
import sys
89
from pathlib import Path
910

@@ -22,33 +23,86 @@
2223

2324
# Paths
2425
SCRIPT_DIR = Path(__file__).parent
25-
PROJECT_ROOT = SCRIPT_DIR.parent
26-
CONTENT_ROOT = PROJECT_ROOT.parent / "content"
26+
PROJECT_ROOT = SCRIPT_DIR.parent.resolve()
2727

2828
CRAWLER_MAP = {
2929
"gitbook": GitBookCrawler,
3030
"retype": RetypeCrawler,
3131
}
3232

3333

34+
def resolve_output_dir(config: dict) -> Path | None:
35+
"""Resolve and validate output_dir relative to the project root."""
36+
raw_output_dir = str(config.get("output_dir", "")).strip()
37+
if not raw_output_dir:
38+
logger.error("配置缺少 output_dir")
39+
return None
40+
41+
output_dir = (PROJECT_ROOT / raw_output_dir).resolve()
42+
43+
try:
44+
output_dir.relative_to(PROJECT_ROOT)
45+
except ValueError:
46+
logger.error(f"output_dir 超出项目目录: {output_dir}")
47+
return None
48+
49+
return output_dir
50+
51+
52+
def create_staging_dir(output_dir: Path) -> Path:
53+
"""Create a clean staging directory next to the final output directory."""
54+
staging_dir = output_dir.parent / f".{output_dir.name}.staging"
55+
if staging_dir.exists():
56+
shutil.rmtree(staging_dir)
57+
staging_dir.mkdir(parents=True, exist_ok=True)
58+
return staging_dir
59+
60+
61+
def publish_staging_dir(staging_dir: Path, output_dir: Path) -> None:
62+
"""Atomically replace the final output directory with staged content."""
63+
if output_dir.exists():
64+
shutil.rmtree(output_dir)
65+
staging_dir.rename(output_dir)
66+
67+
3468
def run_crawler(name: str, config: dict) -> tuple[int, int]:
3569
"""Run a crawler for the given wiki config."""
3670
crawler_type = config.get("type")
3771
crawler_class = CRAWLER_MAP.get(crawler_type)
3872

3973
if not crawler_class:
4074
logger.error(f"未知爬虫类型: {crawler_type}")
41-
return 0, 0
75+
return 0, 1
4276

43-
# Resolve output_dir relative to PROJECT_ROOT
44-
raw_output_dir = config.get("output_dir", "")
45-
abs_output_dir = (PROJECT_ROOT / raw_output_dir).resolve()
77+
output_dir = resolve_output_dir(config)
78+
if output_dir is None:
79+
return 0, 1
80+
81+
staging_dir = create_staging_dir(output_dir)
82+
logger.info(f"输出目录: {output_dir.relative_to(PROJECT_ROOT)}")
4683

4784
crawl_config = config.copy()
48-
crawl_config["output_dir"] = abs_output_dir
85+
crawl_config["output_dir"] = staging_dir
86+
87+
crawler = crawler_class(name, crawl_config)
88+
89+
try:
90+
success, failed = crawler.run()
91+
92+
if success == 0 and failed == 0:
93+
logger.error("未抓取到任何页面,保留现有输出目录")
94+
return 0, 1
95+
96+
if failed > 0:
97+
logger.error("存在抓取失败页面,保留现有输出目录")
98+
return success, failed
4999

50-
crawler = crawler_class(name, crawl_config, CONTENT_ROOT)
51-
return crawler.run()
100+
publish_staging_dir(staging_dir, output_dir)
101+
logger.info(f"📦 已发布到: {output_dir.relative_to(PROJECT_ROOT)}")
102+
return success, failed
103+
finally:
104+
if staging_dir.exists():
105+
shutil.rmtree(staging_dir)
52106

53107

54108
def main():
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
---
2+
title: Commands
3+
sidebar:
4+
order: 2
5+
---
6+
7+
Command aliases can be changed in the **engine.yml**. By default it's `/coinsengine` or `/coe`.
8+
9+
## Flags
10+
11+
* `-s` - makes silent command execution (target player won't be notified).
12+
* `-sf` - disables command feedback to the command sender.
13+
14+
## Plugin Commands
15+
16+
`<> Required` `[] Optional`
17+
18+
* `/coe [help]` - List of all plugin commands.
19+
* `/coe reload` - Reload the plugin.
20+
* `/coe create <name> <symbol> <decimals>` - [Creates](features/currencies#creation) a new currency.
21+
* `/coe reset <player>` - Reset player's balance for all currencies.
22+
* `/coe resetall [currency]` - Reset balance of all or specific currency for all players.
23+
* `/coe migrate <plugin> <currency>` - [Migrate](features/migration) balances from specific plugin to specific currency.
24+
25+
## Currency Commands
26+
27+
Each currency has it's own command aliases defined in the [currency config](features/currencies) under `Command_Aliases` option.
28+
29+
Commands listed below are valid for the currency with the `coins` command alias.
30+
31+
`<> Required` `[] Optional`
32+
33+
* `/coins balance [player]` - View "coins" balance.
34+
* `/coins pay <player> <amount>` - Pay some "coins" some player.
35+
* `/coins payments [player] [-s]` - Toggle "coins" payments from other players.
36+
* `/coins giveall <amount> [-s] [-sf]` - Give "coins" to all online players.
37+
* `/coins give <player> <amount> [-s] [-sf]` - Give "coins" to a player.
38+
* `/coins take <player> <amount> [-s] [-sf]` - Take "coins" from a player.
39+
* `/coins set <player> <amount> [-s] [-sf]` - Set "coins" balance for a player.
40+
* `/coins exchange <currency> <amount>` - Exchange "coins" for other currency.
41+
* `/coins top [page]` - Top players with "coins". Requires the [Leaderboards](features/leaderboards) feature to be enabled.
42+
43+
:::note
44+
[Number Shortcuts](https://nightexpressdev.com/nightcore/configuration/number-formation/#number-shortcuts) are supported in the `amount` argument!
45+
:::
46+
47+
There is the `commands.yml` config file inside the CoinsEngine directory, where you can edit and disable any currency-related command.
48+
49+
**Command Options:**
50+
51+
* `Children` - Controls whether the command is available as of the currency command. Example: `/coins balance`
52+
* `Dedicated` - Controls whether the command is available as command for primary currency. Example: `/balance`
53+
54+
## Dedicated Commands
55+
56+
* `/wallet [player]` - View balance of all currencies. Requires the [Wallet](features/wallet) feature to be enabled.
57+
* `/balance [player]` - View balance of primary currency.
58+
* `/baltop [page]` - Top players by primary currency. Requires the [Leaderboards](features/leaderboards) feature to be enabled.
59+
* `/pay <player> <amount>` - Pay player with primary currency.
60+
* `/paytoggle [player]` - Toggle primary currency payments from players.
61+
62+
:::note
63+
Primary currency is the one that linked with [Vault](hooks/vault).
64+
:::
65+

0 commit comments

Comments
 (0)