Skip to content

Commit a93a88b

Browse files
pyramationpyramation
committed
tags
1 parent 610edc9 commit a93a88b

16 files changed

+1709
-34
lines changed

‎MIGRATE_DOCS.md‎

Lines changed: 350 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,350 @@
1+
# LaunchQL Migrate Documentation
2+
3+
## Overview
4+
5+
LaunchQL Migrate (`@launchql/migrate`) is a low-level migration engine that serves as a pure TypeScript replacement for Sqitch. It provides database migration management with support for deployment, reversion, and verification of database changes.
6+
7+
## Architecture
8+
9+
```mermaid
10+
graph TB
11+
subgraph "@launchql/core"
12+
Core[Core Orchestration]
13+
ProjectMgmt[Project Management]
14+
HighLevel[High-level APIs]
15+
end
16+
17+
subgraph "@launchql/migrate"
18+
Parser[Plan Parser]
19+
Validator[Reference Validator]
20+
Deploy[Deploy Engine]
21+
Revert[Revert Engine]
22+
Verify[Verify Engine]
23+
Registry[Change Registry]
24+
Transaction[Transaction Manager]
25+
end
26+
27+
subgraph "Database"
28+
PG[(PostgreSQL)]
29+
Schema[sqitch schema]
30+
Changes[changes table]
31+
Dependencies[dependencies table]
32+
Tags[tags table]
33+
end
34+
35+
Core --> Parser
36+
Core --> Deploy
37+
Core --> Revert
38+
Core --> Verify
39+
40+
Parser --> Validator
41+
Deploy --> Registry
42+
Deploy --> Transaction
43+
Revert --> Registry
44+
Revert --> Transaction
45+
Verify --> Registry
46+
47+
Registry --> Schema
48+
Transaction --> PG
49+
```
50+
51+
## Core Components
52+
53+
### 1. Plan Parser (`src/parser/`)
54+
55+
The plan parser reads and validates Sqitch plan files, supporting all reference formats defined in the Sqitch specification.
56+
57+
#### Key Files:
58+
- `plan-parser.ts` - Main parser with validation
59+
- `validators.ts` - Reference validation logic
60+
- `plan.ts` - Legacy parser (backwards compatibility)
61+
62+
#### Supported Reference Formats:
63+
64+
```typescript
65+
// Plain change name
66+
"users_table"
67+
68+
// Tag reference
69+
"@v1.0"
70+
71+
// Change at specific tag
72+
"[email protected]"
73+
74+
// SHA1 reference (40 hex chars)
75+
"abc1234567890123456789012345678901234567"
76+
77+
// Project-qualified forms
78+
"other_project:users_table"
79+
"other_project:@v1.0"
80+
"other_project:abc1234567890123456789012345678901234567"
81+
82+
// Symbolic references
83+
"HEAD" // Last change
84+
"ROOT" // First change
85+
"@HEAD" // Alternative form
86+
"@ROOT" // Alternative form
87+
88+
// Relative references
89+
"HEAD^" // One change before HEAD
90+
"HEAD^^" // Two changes before HEAD
91+
"HEAD^3" // Three changes before HEAD
92+
"@v1.0~" // One change after tag v1.0
93+
"@v1.0~2" // Two changes after tag v1.0
94+
```
95+
96+
### 2. Deploy Engine (`src/commands/deploy.ts`)
97+
98+
Deploys database changes in dependency order.
99+
100+
#### Deploy Flow:
101+
102+
```mermaid
103+
sequenceDiagram
104+
participant CLI
105+
participant Deploy
106+
participant Registry
107+
participant Transaction
108+
participant DB
109+
110+
CLI->>Deploy: deploy(options)
111+
Deploy->>Registry: getDeployedChanges()
112+
Registry->>DB: SELECT FROM changes
113+
DB-->>Registry: deployed changes
114+
Registry-->>Deploy: Set<deployed>
115+
116+
Deploy->>Deploy: calculateDeployOrder()
117+
118+
loop For each change
119+
Deploy->>Registry: isDeployed(change)
120+
alt Not deployed
121+
Deploy->>Transaction: begin()
122+
Deploy->>DB: Execute deploy.sql
123+
Deploy->>Registry: recordChange()
124+
Deploy->>Transaction: commit()
125+
else Already deployed
126+
Deploy->>Deploy: skip
127+
end
128+
end
129+
```
130+
131+
### 3. Revert Engine (`src/commands/revert.ts`)
132+
133+
Reverts deployed changes in reverse dependency order.
134+
135+
#### Revert Flow:
136+
137+
```mermaid
138+
sequenceDiagram
139+
participant CLI
140+
participant Revert
141+
participant Registry
142+
participant Transaction
143+
participant DB
144+
145+
CLI->>Revert: revert(target, options)
146+
Revert->>Registry: getDeployedChanges()
147+
Revert->>Revert: calculateRevertOrder()
148+
149+
loop For each change
150+
Revert->>Registry: hasDependents(change)
151+
alt No dependents
152+
Revert->>Transaction: begin()
153+
Revert->>DB: Execute revert.sql
154+
Revert->>Registry: removeChange()
155+
Revert->>Transaction: commit()
156+
else Has dependents
157+
Revert->>Revert: error
158+
end
159+
end
160+
```
161+
162+
### 4. Verify Engine (`src/commands/verify.ts`)
163+
164+
Verifies that deployed changes match their expected state.
165+
166+
### 5. Change Registry (`src/registry/`)
167+
168+
Manages the sqitch schema and tracks deployed changes, dependencies, and tags.
169+
170+
## API Reference
171+
172+
### Plan Parser API
173+
174+
```typescript
175+
import { parsePlanFileWithValidation } from '@launchql/migrate';
176+
177+
// Parse a plan file with full validation
178+
const result = parsePlanFileWithValidation('/path/to/sqitch.plan');
179+
180+
if (result.errors.length > 0) {
181+
// Handle validation errors
182+
result.errors.forEach(error => {
183+
console.error(`Line ${error.line}: ${error.message}`);
184+
});
185+
} else {
186+
// Use the parsed plan
187+
const plan = result.plan;
188+
console.log(`Project: ${plan.project}`);
189+
console.log(`Changes: ${plan.changes.length}`);
190+
console.log(`Tags: ${plan.tags.length}`);
191+
}
192+
```
193+
194+
### Reference Validation API
195+
196+
```typescript
197+
import { isValidChangeName, isValidTagName, parseReference } from '@launchql/migrate';
198+
199+
// Validate change names
200+
isValidChangeName('users_table'); // true
201+
isValidChangeName('@invalid'); // false
202+
203+
// Validate tag names
204+
isValidTagName('v1.0'); // true
205+
isValidTagName('tag/with/slash'); // false
206+
207+
// Parse references
208+
const ref = parseReference('other_project:[email protected]');
209+
// Returns: {
210+
// project: 'other_project',
211+
// change: 'users_table',
212+
// tag: 'v1.0'
213+
// }
214+
```
215+
216+
### Deploy API
217+
218+
```typescript
219+
import { deploy } from '@launchql/migrate';
220+
221+
// Deploy all changes
222+
await deploy({
223+
planFile: '/path/to/sqitch.plan',
224+
connection: pgConnection,
225+
transaction: true, // Use transaction mode
226+
verify: true // Verify after deploy
227+
});
228+
229+
// Deploy to a specific target
230+
await deploy({
231+
planFile: '/path/to/sqitch.plan',
232+
connection: pgConnection,
233+
target: '@v1.0' // Deploy up to tag v1.0
234+
});
235+
```
236+
237+
### Revert API
238+
239+
```typescript
240+
import { revert } from '@launchql/migrate';
241+
242+
// Revert to a specific change
243+
await revert({
244+
planFile: '/path/to/sqitch.plan',
245+
connection: pgConnection,
246+
target: 'users_table',
247+
transaction: true
248+
});
249+
250+
// Revert last 3 changes
251+
await revert({
252+
planFile: '/path/to/sqitch.plan',
253+
connection: pgConnection,
254+
target: 'HEAD^3'
255+
});
256+
```
257+
258+
### Registry API
259+
260+
```typescript
261+
import { Registry } from '@launchql/migrate';
262+
263+
const registry = new Registry(pgConnection);
264+
265+
// Check if a change is deployed
266+
const isDeployed = await registry.isChangeDeployed('project', 'change_name');
267+
268+
// Get all deployed changes
269+
const deployed = await registry.getDeployedChanges('project');
270+
271+
// Record a new deployment
272+
await registry.recordChange({
273+
project: 'my_project',
274+
change: 'users_table',
275+
dependencies: ['initial_schema'],
276+
deployedBy: 'developer',
277+
deployedAt: new Date()
278+
});
279+
```
280+
281+
## Transaction Management
282+
283+
All operations support both transactional and non-transactional modes:
284+
285+
```typescript
286+
// With transaction (default) - rolls back on error
287+
await deploy({
288+
transaction: true,
289+
// ... other options
290+
});
291+
292+
// Without transaction - continues on error
293+
await deploy({
294+
transaction: false,
295+
// ... other options
296+
});
297+
```
298+
299+
## Cross-Project Dependencies
300+
301+
LaunchQL Migrate supports dependencies across different projects:
302+
303+
```typescript
304+
// In project A's plan file
305+
users_table 2024-01-01T00:00:00Z dev <dev@example.com>
306+
307+
// In project B's plan file
308+
user_profiles [project_a:users_table] 2024-01-02T00:00:00Z dev <dev@example.com>
309+
```
310+
311+
The system ensures that cross-project dependencies are satisfied before deployment and prevents reversion of changes that have cross-project dependents.
312+
313+
## Error Handling
314+
315+
All operations return detailed error information:
316+
317+
```typescript
318+
try {
319+
await deploy(options);
320+
} catch (error) {
321+
if (error.code === 'DEPENDENCY_NOT_MET') {
322+
console.error(`Missing dependency: ${error.dependency}`);
323+
} else if (error.code === 'VERIFICATION_FAILED') {
324+
console.error(`Verification failed for: ${error.change}`);
325+
}
326+
}
327+
```
328+
329+
## Best Practices
330+
331+
1. **Always use transactions in production** - This ensures database consistency
332+
2. **Verify after deployment** - Use the `verify` option to ensure changes were applied correctly
333+
3. **Use meaningful change names** - Follow the naming conventions (alphanumeric, underscore, hyphen)
334+
4. **Tag releases** - Use tags to mark stable points in your migration history
335+
5. **Test migrations** - Always test in a development environment first
336+
337+
## Migration from Sqitch
338+
339+
LaunchQL Migrate is designed to be compatible with existing Sqitch plan files and database schemas. To migrate:
340+
341+
1. Install `@launchql/migrate`
342+
2. Point it to your existing `sqitch.plan` file
343+
3. Use the same database connection
344+
4. All existing deployments will be recognized
345+
346+
The main differences from Sqitch:
347+
- Pure TypeScript implementation (no Perl required)
348+
- Better error messages and validation
349+
- Programmatic API for integration
350+
- Support for modern JavaScript/TypeScript tooling

‎packages/migrate/__tests__/fixtures/plan-invalid/bad-symbolic-refs.plan‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,32 @@
1+
%syntax-version=1.0.0
2+
%project=bad-symbolic
3+
%uri=https://github.com/test/bad-symbolic
4+
5+
# Base changes
6+
first 2024-01-01T00:00:00Z dev <[email protected]> # First change
7+
second 2024-01-01T00:00:01Z dev <[email protected]> # Second change
8+
9+
# Invalid symbolic references
10+
11+
# Invalid dependency references
12+
bad_dep1 [HEAD^-1] 2024-01-01T00:00:02Z dev <[email protected]> # Negative relative count
13+
bad_dep2 [@ROOT^^~] 2024-01-01T00:00:03Z dev <[email protected]> # Mixed ^ and ~
14+
bad_dep3 [change:with:colons] 2024-01-01T00:00:04Z dev <[email protected]> # Too many colons
15+
bad_dep4 [@invalid@tag] 2024-01-01T00:00:05Z dev <[email protected]> # @ in tag name
16+
bad_dep5 [change name with spaces] 2024-01-01T00:00:06Z dev <[email protected]> # Spaces in dependency
17+
18+
# Invalid SHA1 references
19+
bad_sha1 [40763784148fa190d75bad036730ef44d1c2eac] 2024-01-01T00:00:07Z dev <[email protected]> # Too short (39 chars)
20+
bad_sha2 [40763784148fa190d75bad036730ef44d1c2eac6x] 2024-01-01T00:00:08Z dev <[email protected]> # Too long (41 chars)
21+
bad_sha3 [40763784148fa190d75bad036730ef44d1c2eacg] 2024-01-01T00:00:09Z dev <[email protected]> # Invalid hex char 'g'
22+
bad_sha4 [40763784148fa190d75bad036730ef44d1c2ea ] 2024-01-01T00:00:10Z dev <[email protected]> # Space in SHA1
23+
24+
# Out of bounds relative references (these would be runtime errors, not parse errors)
25+
# But including them to test error handling
26+
out_of_bounds1 [@ROOT^] 2024-01-01T00:00:11Z dev <[email protected]> # Can't go before ROOT
27+
out_of_bounds2 [@HEAD~100] 2024-01-01T00:00:12Z dev <[email protected]> # Too far forward
28+
29+
# Invalid tag references in dependencies
30+
bad_tag_dep1 [@foo/bar] 2024-01-01T00:00:13Z dev <[email protected]> # Tag with slash
31+
bad_tag_dep2 [change@@double] 2024-01-01T00:00:14Z dev <[email protected]> # Double @
32+
bad_tag_dep3 [change@] 2024-01-01T00:00:15Z dev <[email protected]> # Empty tag name

0 commit comments

Comments
 (0)