Feat/pnpm #255
Merged
Feat/pnpm #255
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Contributor
|
Please rebase. now we have moved launchql/Dockerfile to root Dockerfile. |
Contributor
merged |
pyramation
commented
Nov 11, 2025
packages/cli/README.md
Outdated
|
|
||
| * [launchql/server](https://github.com/launchql/launchql/tree/main/packages/server): **⚡ Express-based API server** powered by PostGraphile to expose a secure, scalable GraphQL API over your Postgres database. | ||
| * [launchql/explorer](https://github.com/launchql/launchql/tree/main/packages/explorer): **🔎 Visual API explorer** with GraphiQL for browsing across all databases and schemas—useful for debugging, documentation, and API prototyping. | ||
| - [launchql/server](https://github.com/launchql/launchql/tree/main/packages/server): **⚡ Express-based API server** powered by PostGraphile to expose a secure, scalable GraphQL API over your Postgres database. |
Contributor
Author
There was a problem hiding this comment.
if possible don't change * to - — will be hard later to batch update others
Contributor
There was a problem hiding this comment.
maybe prettier change it automatically, let me check
…fig.json for TypeScript compilation, and preventing .js and .d.ts files from being generated in the src folder
Replace lerna run commands with pnpm -r to avoid NX dependency issues and simplify workspace script execution.
Replace symlink-workspace package with custom Node.js script to support pnpm workspace bin linking, as symlink-workspace doesn't work with pnpm's isolated node_modules structure.
Ensure TypeScript compiler uses correct config file to prevent output files from being generated in source directories instead of dist folder.
Add @launchql/logger, pg, and @types/pg to dependencies. Yarn automatically hoists dependencies to workspace root, allowing packages to access dependencies not declared in their package.json. pnpm requires explicit declarations, preventing phantom dependencies and ensuring better reproducibility
Add explicit dependencies to packages that were relying on yarn hoisting: - cli: @launchql/logger - url-domains: express, @types/express - introspectron: pg, @types/pg - pg-codegen: @launchql/logger, pg-cache, pg-env - pgsql-test: @launchql/logger Remove resolutions field from all packages (pnpm uses root overrides). Yarn hoists dependencies automatically, but pnpm requires explicit declarations in each package.json for better dependency isolation and reproducibility
- Add pg and @types/pg as devDependencies in packages/cli/package.json Required for TypeScript tests that import pg in pnpm's isolated structure - Fix module template build script to use 'tsc -p tsconfig.json' instead of 'tsc' Ensures consistent build behavior across generated modules - Update test snapshots to reflect corrected template output and file ordering
… pnpm - Add [email protected] as devDependency in packages/launchql-gen/package.json - Required for TypeScript tests that import graphql in pnpm's isolated structure - GraphQL 15 includes built-in type definitions, no @types/graphql needed
…ith pnpm - Add @types/babel__generator@^7.6.8 as devDependency in packages/pg-codegen/package.json - Required for TypeScript tests that import @babel/generator in pnpm's isolated structure - Fixes TS7016 error: Could not find a declaration file for module '@babel/generator'
- Add glob@^11.0.2 and @types/glob@^8.1.0 as devDependencies in packages/s3-streamer/package.json - Required for TypeScript tests that import glob in pnpm's isolated structure - Update test snapshots to reflect hash value changes
…ependencies - Add moduleNameMapper configuration in packages/pgsql-test/jest.config.js - Maps @launchql/* packages to packages/*/dist directories for Jest resolution - Fixes Jest module resolution issues with pnpm's isolated structure - Required for tests that import @launchql/core and other workspace packages
- Add moduleNameMapper configuration in packages/cli/jest.config.js - Maps @launchql/* packages to packages/*/dist directories - Maps pg-env and pg-cache to their dist directories - Fixes Jest module resolution issues with pnpm's isolated structure - Update test snapshots to match corrected template build script format
- Add moduleNameMapper configuration in packages/introspectron/jest.config.js - Maps @launchql/* packages to packages/*/dist directories - Maps pgsql-test, graphile-test, pg-env, and pg-cache to their dist directories - Fixes Jest module resolution issues with pnpm's isolated structure - Required for tests that import workspace packages like pgsql-test and graphile-test
- Add moduleNameMapper configuration in packages/launchql-gen/jest.config.js - Maps @launchql/* packages to packages/*/dist directories - Maps pgsql-test, graphile-test, pg-env, and pg-cache to their dist directories - Fixes Jest module resolution issues with pnpm's isolated structure - Required for tests that import graphile-test which depends on pgsql-test
- Add moduleNameMapper configuration in packages/pg-codegen/jest.config.js - Maps @launchql/* packages to packages/*/dist directories - Maps pgsql-test, graphile-test, pg-env, and pg-cache to their dist directories - Fixes Jest module resolution issues with pnpm's isolated structure - Required for tests that import pgsql-test which depends on @launchql/logger
- Replace version-based dependencies with workspace:^ protocol in packages/cli/package.json - Remove moduleNameMapper from Jest config as pnpm workspace symlinks handle module resolution - Add .sort() to test file lists for deterministic snapshot ordering This leverages pnpm's native workspace protocol for better dependency resolution and simplifies Jest configuration by relying on pnpm's symlink structure.
…ve Jest moduleNameMapper - Replace workspace dependencies with workspace:^ protocol in: * packages/pgsql-test (dependencies) * packages/launchql-gen (devDependencies) * packages/pg-codegen (dependencies) * packages/introspectron (devDependencies) - Remove moduleNameMapper from Jest configs: * packages/pgsql-test/jest.config.js * packages/launchql-gen/jest.config.js * packages/pg-codegen/jest.config.js * packages/introspectron/jest.config.js This completes the migration to pnpm's native workspace protocol, allowing Jest to resolve workspace packages via pnpm's symlink structure without manual module path mappings.
- Remove symlink and postinstall scripts from root package.json - Remove symlink-workspace dependency from all package.json files - Replace lerna commands with pnpm commands in templates - Delete scripts/symlink-workspace.js (no longer needed) - Update workspace templates to use pnpm native workspace commands pnpm automatically handles workspace package linking via workspace:^ protocol, so manual symlink scripts are unnecessary.
Make the chmod step for lql binary conditional to prevent CI failures when node_modules/.bin/lql doesn't exist. pnpm workspace linking may not always create the binary symlink in node_modules/.bin, but pnpm exec still works correctly for workspace packages
…ency - Add explicit DocumentNode type annotation to IntrospectionQuery export in introspectGql.ts to ensure TypeScript generates proper type definitions - Add graphql dependency to package.json to resolve missing module errors during type checking This fixes the build error where launchql-gen tests failed because IntrospectionQuery type was not exported in the generated .d.ts files. Fixes: TS2724 error in launchql-gen package tests
…solution Replace direct node execution with `pnpm --filter @launchql/cli exec lql` to ensure workspace dependencies are properly resolved. Direct node execution cannot resolve pnpm workspace dependencies like 'pg-env', causing MODULE_NOT_FOUND errors. Using pnpm filter ensures the command runs in the correct workspace context with all dependencies available.
Executing the CLI through `pnpm --filter @launchql/cli exec node dist/index.js` keeps the command inside the CLI package so pnpm wires up workspace dependencies like `pg-env` correctly. This replaces the previous attempts (`pnpm exec lql`, `pnpm --filter … exec lql`, and direct `node …`) that failed to resolve those dependencies in CI.
… in CI Add pg-env as a workspace dependency to @launchql/explorer package.json. The explorer package imports pg-env but it was not declared as a dependency, causing MODULE_NOT_FOUND errors when running CLI commands in CI. Also update CI workflow to use `pnpm --filter @launchql/cli exec node dist/index.js` instead of `pnpm exec lql` to ensure proper workspace module resolution. This ensures all workspace dependencies like pg-env are correctly resolved when executing CLI commands in the CI environment.
…pace dependency Add index.js and index.d.ts barrel files to templatizer package to re-export dist exports, ensuring TypeScript can resolve loadTemplates and TemplateSource when imported from @launchql/templatizer in workspace packages. Change @launchql/core to use workspace:^ protocol for templatizer dependency instead of version pin, allowing Jest/TypeScript to resolve the local built package correctly during development and testing. This fixes TS2305 errors where TypeScript couldn't find exported members loadTemplates and TemplateSource from @launchql/templatizer module.
The server package imports Logger from @launchql/logger but the dependency was not declared in package.json, causing MODULE_NOT_FOUND errors in CI when running the CLI commands. Add workspace dependency to ensure proper module resolution.
…ning Add GIT_TERMINAL_PROMPT=0 and GIT_ASKPASS=echo environment variables to git clone command to prevent authentication popups when cloning public GitHub repositories during template loading. This fixes the issue where running tests locally would trigger GitHub login prompts even for public repositories like launchql/launchql.
- Add @jest/test-sequencer to root devDependencies to fix Jest test sequencer resolution errors - Convert internal package dependencies to workspace:^ protocol: - @launchql/server-utils: workspace:^ in server, core, explorer - pg-cache: workspace:^ in server, explorer, core - @launchql/logger: workspace:^ in server-utils - Add re-export files (index.js, index.d.ts) to pg-cache package for proper module resolution in workspace context - Update init.install test snapshot to match pnpm-based template output (license field and pnpm scripts) This ensures Jest and TypeScript can correctly resolve internal packages when running tests in the monorepo workspace, fixing MODULE_NOT_FOUND errors that were preventing tests from passing.
Update lockfile to match packages/explorer/package.json changes where pg-cache dependency was changed from version pin to workspace:^ protocol. This fixes the CI error where pnpm install --frozen-lockfile failed due to outdated lockfile.
The server package imports graphile-utils in PublicKeySignature plugin but the dependency was not declared in package.json, causing MODULE_NOT_FOUND errors when running CLI commands that load the server module (e.g., init command). Add graphile-utils dependency to ensure proper module resolution.
- Add @jest/test-sequencer to introspectron devDependencies - Add pgsql-test workspace dependency to graphile-test - Configure pnpm to hoist @jest/test-sequencer for Jest resolution - Update pnpm-lock.yaml with new dependencies Fixes test failures where @jest/test-sequencer and pgsql-test could not be resolved during test execution.
Add index.js and index.d.ts files to re-export the built dist bundle, ensuring TypeScript and Jest can correctly resolve pgsql-test imports in graphile-test tests. Fixes CI failures where graphile-test tests could not find pgsql-test module during TypeScript compilation.
Change list markers from dash (-) to asterisk (*) in "Related LaunchQL Tooling" section to match project style preference. Fixes inconsistency where external link lists were using dashes instead of asterisks.
Add csv-parse and @types/csv-parse to dependencies to fix TypeScript compilation errors in src/seed/csv.ts. The csv seed functionality requires csv-parse but it was not declared in package.json. Fixes test failures where all test suites failed with "Cannot find module 'csv-parse'" errors.
pyramation
commented
Nov 12, 2025
| }, | ||
| "packageManager": "[email protected]+sha512.a6b2f7906b721bba3d67d4aff083df04dad64c399707841b7acf00f6b133b7ac24255f2652fa22ae3534329dc6180534e98d17432037ff6fd140556e2bb3137e" | ||
| "packageManager": "[email protected]", | ||
| "pnpm": { |
Contributor
Author
There was a problem hiding this comment.
why are these needed? is it absolutely necessary?
Contributor
There was a problem hiding this comment.
These were automatically added by Corepack when using pnpm.
They help lock the package manager version for consistency, but are not strictly required.
Would you prefer me to remove all packageManager fields from the code?
pyramation
commented
Nov 12, 2025
| "@types/pg": "^8.15.2", | ||
| "graphql": "15.10.1", | ||
| "graphql-upload": "^13.0.0" | ||
| }, |
Contributor
Author
There was a problem hiding this comment.
they just don't need this?
Contributor
There was a problem hiding this comment.
Root doesn’t build or run code directly, so I removed @types/pg, graphql, and graphql-upload there. The packages that actually need them (server, explorer, graphile-settings, cli, etc.) still declare them in their own package.json files. Via pnpm overrides we still pin [email protected] and graphql-upload@^13.0.0, so we don’t end up with multiple versions when installing
pyramation
commented
Nov 12, 2025
| "lint": "pnpm -r run lint" | ||
| }, | ||
| "devDependencies": { | ||
| "@jest/test-sequencer": "^29.7.0", |
Contributor
Author
There was a problem hiding this comment.
why install this?
Contributor
There was a problem hiding this comment.
jest-config (pulled in by several of our packages) requires @jest/test-sequencer at runtime but doesn’t declare it. With pnpm’s strict dependency graph that shows up as a “module not found” when we run the suite. Adding @jest/test-sequencer at the root and hoisting it fixes the missing-module error so the Jest commands in CI keep working
pyramation
commented
Nov 12, 2025
packages/core/package.json
Outdated
| "@types/pg": "^8.15.2" | ||
| }, | ||
| "dependencies": { | ||
| "@launchql/env": "^2.4.3", |
Contributor
Author
There was a problem hiding this comment.
shouldn't launchql/env also be workspace?
Contributor
There was a problem hiding this comment.
I missed that one when switching everything to workspace refs, I will check again for all workspace refs
Contributor
There was a problem hiding this comment.
I've fixed all of these issue, and checked all tests passed locally.
will push later after other issues fixed
pyramation
commented
Nov 12, 2025
packages/core/package.json
Outdated
| "@launchql/logger": "workspace:^", | ||
| "@launchql/server-utils": "workspace:^", | ||
| "@launchql/templatizer": "workspace:^", | ||
| "@launchql/types": "^2.6.2", |
Contributor
Author
There was a problem hiding this comment.
shouldn't launchql/types also be workspace:*?
Contributor
There was a problem hiding this comment.
Same issue as the previous comment, it’s already fixed now
pyramation
commented
Nov 12, 2025
packages/core/package.json
Outdated
| "pg-cache": "^1.3.4", | ||
| "pg": "^8.16.0", | ||
| "pg-cache": "workspace:^", | ||
| "pg-env": "^1.1.0", |
Contributor
Author
There was a problem hiding this comment.
another workspace package
Contributor
There was a problem hiding this comment.
Same issue as the previous comment, it’s already fixed now
pyramation
commented
Nov 12, 2025
packages/explorer/package.json
Outdated
| "@launchql/s3-streamer": "^2.5.3", | ||
| "@launchql/server-utils": "^2.4.3", | ||
| "@launchql/server-utils": "workspace:^", | ||
| "@launchql/types": "^2.6.2", |
Contributor
Author
There was a problem hiding this comment.
more workspace:*:
launchql/env
launchql/s3-streamer
launchql/types
launchql/upload-names
launchql/url-domains
graphile-cache
graphile-settings
graphql-upload
Contributor
There was a problem hiding this comment.
Same issue as the previous comment, it’s already fixed now
pyramation
commented
Nov 12, 2025
packages/graphile-test/package.json
Outdated
| }, | ||
| "dependencies": { | ||
| "@launchql/types": "^2.6.2", | ||
| "graphile-settings": "^2.5.3", |
Contributor
Author
There was a problem hiding this comment.
more workspace packages
Contributor
There was a problem hiding this comment.
Same issue as the previous comment, it’s already fixed now
pyramation
commented
Nov 12, 2025
packages/server/package.json
Outdated
| "@launchql/server-utils": "workspace:^", | ||
| "@launchql/types": "^2.6.2", | ||
| "@launchql/upload-names": "^2.2.1", | ||
| "@launchql/url-domains": "^2.2.1", |
Contributor
Author
There was a problem hiding this comment.
more workspace packages
Contributor
There was a problem hiding this comment.
Same issue as the previous comment, it’s already fixed now
pyramation
commented
Nov 12, 2025
| fi | ||
| echo -e "\${GREEN}Installing SQL modules from:\${RESET} \${CYAN}\${base_dir}\${RESET}" | ||
| find "$base_dir" -type f -name "sqitch.plan" | while read -r plan_file; do |
Contributor
Author
There was a problem hiding this comment.
this code looks wrong, let's figure out why it's not referencing launchql.plan like the old one...
Contributor
There was a problem hiding this comment.
Fixed — updated to search for launchql.plan files
Convert all internal package dependencies to use `workspace:^` protocol instead of version ranges. This ensures pnpm always links to local workspace packages during development rather than installing from npm registry. - Updated all @launchql/* package dependencies - Updated graphile-* package dependencies (cache, query, settings) - Updated pg-* package dependencies (cache, env, query-context) - Updated utility packages (etag-hash, uuid-hash, mime-bytes, etc.) - Updated sandbox project dependencies
Remove test-generated CSV files from git tracking and add output directory to .gitignore. These files are generated during test runs and should not be committed. - Remove packages/pgsql-test/output/users.csv - Remove packages/pgsql-test/output/posts.csv - Add packages/pgsql-test/output/ to .gitignore
…ce install script The generated install.sh script was searching for sqitch.plan files, but LaunchQL modules use launchql.plan. This caused the script to fail to find any modules during installation. - Change find command to search for launchql.plan files - Fixes module discovery in workspace installation workflow Verified by: - Generating test workspace and module - Confirming install.sh successfully finds and installs modules
Update lerna from version 6 to 8.2.3 across the repository and modernize lerna.json configuration to match Lerna 8.x schema. Changes: - Upgrade lerna dependency to ^8.2.3 in root and workspace boilerplate - Update lerna.json to Lerna 8.x format: * Add $schema reference for editor validation * Remove deprecated "lerna" and "useWorkspaces" fields * Move conventionalCommits under command.publish * Keep existing npmClient, npmClientArgs, and registry settings - Update generated workspace template to use new lerna.json format Files modified: - package.json (root) - lerna.json (root) - boilerplates/workspace/package.json - boilerplates/workspace/lerna.json - packages/templatizer/src/generated/workspace.ts
Contributor
|
Quick question. Shouldnt the boilerplate also have pnpm workspace file? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.