diff --git a/contribs/gnodev/README.md b/contribs/gnodev/README.md index 2412d4c4d5a..d537231df10 100644 --- a/contribs/gnodev/README.md +++ b/contribs/gnodev/README.md @@ -11,12 +11,13 @@ Please note that this is a quick overview. For a more detailed guide, refer to t **gnodev** [**options**] [**PKG_PATH ...**] ## Features -- **In-Memory Node**: Gnodev starts an in-memory node, automatically loading the **examples** folder and any - user-specified paths. +- **In-Memory Node**: Gnodev starts an in-memory node with automatic package discovery. +- **Package Discovery**: Automatically detects packages via `gnomod.toml` files and workspaces via `gnowork.toml`. +- **Flexible Loading Modes**: Three loading modes (`auto`, `lazy`, `full`) to balance startup time and convenience. - **Web Interface Server**: Gnodev starts a `gnoweb` server on [`localhost:8888`](https://localhost:8888). - **Balances and Keybase Customization**: Set account balances, load them from a file, or add new accounts via a flag. -- **Hot Reload**: Monitors the **examples** folder and specified directories for file changes, reloading the - package and automatically restarting the node as needed. +- **Hot Reload**: Monitors package directories for file changes, reloading the package and automatically + restarting the node as needed. - **State Maintenance**: Ensures the previous node state is preserved by replaying all transactions. - **Transaction Manipulation**: Allows for interactive cancellation and redoing of transactions. - **State Export**: Export the current state at any time in a genesis doc format. @@ -35,108 +36,86 @@ While `gnodev` is running, trigger specific actions by pressing the following co - **Cmd+C**: Exit `gnodev`. ## Usage -Run `gnodev` followed by any specific options and/or package paths. The **examples** directory is loaded -automatically. Use `--minimal` to prevent this. +Run `gnodev` from a directory containing a `gnomod.toml` file, and the package will be automatically detected +and loaded. You can also pass package directories as arguments. + +### Load Modes +Use the `-load` flag to control how packages are loaded: +- **auto** (default for local): Pre-loads current workspace/package only +- **lazy**: Loads all packages on-demand as they are accessed +- **full** (default for staging): Pre-loads all discovered packages Example: ``` -gnodev --add-account [:] ./myrealm +gnodev # Auto-detect and pre-load current package +gnodev -load=lazy # Load packages on-demand only +gnodev -load=full # Pre-load all packages ``` ### `gnodev -h` -[embedmd]:# (.tmp/gnodev-usage.txt) ```txt USAGE - gnodev [flags] + gnodev [flags] The gnodev command starts an in-memory node and a gno.land web interface, primarily for realm package development. -Currently gnodev comes with two mode and , those command mostly -differ by there default values, while gnodev local as default for working -locally, satging default are oriented to be use on server. - -gnodev uses its own package loader and resolver system to support multiple -scenarios and use cases. It currently supports three types of resolvers, each -taking a location as an argument. -- root: This resolver takes a as its location. It attempts to resolve - packages based on your file system structure and the package path. - For example, if 'root=/user/gnome/myproject' and you try to resolve - 'gno.land/r/bar/buzz' as a package, the resolver will attempt to - resolve it to /user/gnome/myproject/gno.land/r/bar/buzz. -- local: This resolver also takes a as its location. It is designed to - load a single package, using the module name from 'gnomod.toml' within this - package to resolve the package. -- remote: This resolver takes a RPC address as its location. It is - meant to use a remote node as a resolver, primarily for testing a local - package against a remote node. - -Resolvers can be chained, and gnodev will attempt to use them in the order they -are declared. - -For example: - gnodev -resolver root=/user/gnome/myproject -resolver remote=https://rpc.gno.lands - -If no resolvers can resolve a given package path, the loader will return a -"package not found" error. +gnodev comes with two modes: and . These commands +differ mainly by their default values - local mode is optimized for +development, while staging mode is oriented for server usage. -If no command is provided, gnodev will automatically start in mode. +Package discovery is automatic via gnomod.toml and gnowork.toml files. +Use the -load flag to control loading behavior: + - auto: Pre-load current workspace/package (default for local) + - lazy: Load packages on-demand as accessed + - full: Pre-load all discovered packages (default for staging) -For more information and flags usage description, use 'gnodev local -h'. +If no command is provided, gnodev will automatically start in mode. SUBCOMMANDS local Start gnodev in local development mode (default) staging Start gnodev in staging mode - ``` ### `gnodev local -h` -[embedmd]:# (.tmp/gnodev-local-usage.txt) ```txt USAGE gnodev local [flags] [package_dir...] LOCAL: Local mode configures the node for local development usage. This mode is optimized for realm development, providing an interactive and flexible environment. -It enables features such as interactive mode, unsafe API access for testing, and lazy loading to improve performance. -The log format is set to console for easier readability, and the web interface is accessible locally, making it ideal for iterative development and testing. - -By default, the current directory and the "example" folder from "gnoroot" will be used as the root resolver. +It enables features such as interactive mode, unsafe API access for testing, and auto loading mode. +The log format is set to console for easier readability, and the web interface is accessible locally. +Package discovery is automatic via gnomod.toml and gnowork.toml files. FLAGS - -C ... change directory context before running gnodev - -add-account ... add (or set) a premine account in the form `[=]`, can be used multiple time - -balance-file ... load the provided balance file (refer to the documentation for format) - -chain-domain gno.land set node ChainDomain - -chain-id dev set node ChainID - -deploy-key g1jg8mtutu9khhfwc4nxmuhcpftf0pajdhfvsqf5 default key name or Bech32 address for deploying packages - -empty-blocks=false enable creation of empty blocks (default: ~1s interval) - -empty-blocks-interval 1 set the interval for creating empty blocks (in seconds) - -genesis ... load the given genesis file - -interactive=false enable gnodev interactive mode - -lazy-loader=true enable lazy loader - -log-format console log output format, can be `json` or `console` - -max-gas 10000000000 set the maximum gas per block - -no-replay=false do not replay previous transactions upon reload - -no-watch=false do not watch for file changes - -no-web=false disable gnoweb - -node-rpc-listener 127.0.0.1:26657 listening address for GnoLand RPC node - -paths ... additional paths to preload in the form of "gno.land/r/my/realm", separated by commas; glob is supported - -resolver ... list of additional resolvers (`root`, `local`, or `remote`) in the form of = will be executed in the given order - -txs-file ... load the provided transactions file (refer to the documentation for format) - -unsafe-api=true enable /reset and /reload endpoints which are not safe to expose publicly - -v=false enable verbose output for development - -web-help-remote ... gnoweb: web server help page's remote addr (default to ) - -web-home ... gnoweb: set default home page, use `/` or `:none:` to use default web home redirect - -web-html=false gnoweb: enable unsafe HTML parsing in markdown rendering - -web-listener 127.0.0.1:8888 gnoweb: web server listener address - -web-with-html=false gnoweb: enable HTML parsing in markdown rendering - + -C ... change directory context before running gnodev + -add-account ... add (or set) a premine account in the form `[=]` + -balance-file ... load the provided balance file + -chain-domain gno.land set node ChainDomain + -chain-id dev set node ChainID + -deploy-key ... default key name or Bech32 address for deploying packages + -empty-blocks=false enable creation of empty blocks + -empty-blocks-interval 1 set the interval for creating empty blocks (in seconds) + -genesis ... load the given genesis file + -interactive=false enable gnodev interactive mode + -load auto package loading mode: auto, lazy, or full + -log-format console log output format: json or console + -max-gas 10000000000 set the maximum gas per block + -no-replay=false do not replay previous transactions upon reload + -no-watch=false do not watch for file changes + -no-web=false disable gnoweb + -node-rpc-listener ... listening address for GnoLand RPC node + -paths ... additional paths to preload (glob supported) + -txs-file ... load the provided transactions file + -unsafe-api=true enable /reset and /reload endpoints + -v=false enable verbose output + -web-home ... set default home page + -web-listener 127.0.0.1:8888 gnoweb: web server listener address ``` ### `gnodev staging -h` -[embedmd]:# (.tmp/gnodev-staging-usage.txt) ```txt USAGE gnodev staging [flags] [package_dir...] @@ -145,39 +124,31 @@ STAGING: Staging mode configures the node for server usage. This mode is designed for stability and security, suitable for pre-deployment testing. Interactive mode and unsafe API access are disabled to ensure a secure environment. The log format is set to JSON, facilitating integration with logging systems. -Since lazy-load is disabled in this mode, the entire example folder from "gnoroot" is loaded by default. - -Additionally, you can specify an additional package directory to load. - +Full loading mode is used by default, pre-loading all discovered packages. FLAGS - -add-account ... add (or set) a premine account in the form `[=]`, can be used multiple time - -balance-file ... load the provided balance file (refer to the documentation for format) - -chain-domain gno.land set node ChainDomain - -chain-id dev set node ChainID - -deploy-key g1jg8mtutu9khhfwc4nxmuhcpftf0pajdhfvsqf5 default key name or Bech32 address for deploying packages - -empty-blocks=false enable creation of empty blocks (default: ~1s interval) - -empty-blocks-interval 1 set the interval for creating empty blocks (in seconds) - -genesis ... load the given genesis file - -interactive=false enable gnodev interactive mode - -lazy-loader=false enable lazy loader - -log-format json log output format, can be `json` or `console` - -max-gas 10000000000 set the maximum gas per block - -no-replay=false do not replay previous transactions upon reload - -no-watch=false do not watch for file changes - -no-web=false disable gnoweb - -node-rpc-listener 127.0.0.1:26657 listening address for GnoLand RPC node - -paths gno.land/** additional paths to preload in the form of "gno.land/r/my/realm", separated by commas; glob is supported - -resolver ... list of additional resolvers (`root`, `local`, or `remote`) in the form of = will be executed in the given order - -txs-file ... load the provided transactions file (refer to the documentation for format) - -unsafe-api=false enable /reset and /reload endpoints which are not safe to expose publicly - -v=false enable verbose output for development - -web-help-remote ... gnoweb: web server help page's remote addr (default to ) - -web-home :none: gnoweb: set default home page, use `/` or `:none:` to use default web home redirect - -web-html=false gnoweb: enable unsafe HTML parsing in markdown rendering - -web-listener 127.0.0.1:8888 gnoweb: web server listener address - -web-with-html=false gnoweb: enable HTML parsing in markdown rendering - + -add-account ... add (or set) a premine account in the form `[=]` + -balance-file ... load the provided balance file + -chain-domain gno.land set node ChainDomain + -chain-id dev set node ChainID + -deploy-key ... default key name or Bech32 address for deploying packages + -empty-blocks=false enable creation of empty blocks + -empty-blocks-interval 1 set the interval for creating empty blocks (in seconds) + -genesis ... load the given genesis file + -interactive=false enable gnodev interactive mode + -load full package loading mode: auto, lazy, or full + -log-format json log output format: json or console + -max-gas 10000000000 set the maximum gas per block + -no-replay=false do not replay previous transactions upon reload + -no-watch=false do not watch for file changes + -no-web=false disable gnoweb + -node-rpc-listener ... listening address for GnoLand RPC node + -paths ... additional paths to preload (glob supported) + -txs-file ... load the provided transactions file + -unsafe-api=false enable /reset and /reload endpoints + -v=false enable verbose output + -web-home :none: set default home page + -web-listener 127.0.0.1:8888 gnoweb: web server listener address ``` ### Transaction file format diff --git a/contribs/gnodev/app.go b/contribs/gnodev/app.go index fca8dc90d8e..f42c85cb7c2 100644 --- a/contribs/gnodev/app.go +++ b/contribs/gnodev/app.go @@ -151,13 +151,10 @@ func (ds *App) Setup(ctx context.Context, dirs ...string) (err error) { loggerEvents := ds.logger.WithGroup(EventServerLogName) ds.emitterServer = emitter.NewServer(loggerEvents) - // XXX: it would be nice to not have this hardcoded - examplesDir := filepath.Join(ds.cfg.root, "examples") - - // Setup loader and resolver + // Setup loader loaderLogger := ds.logger.WithGroup(LoaderLogName) - resolver, localPaths := setupPackagesResolver(loaderLogger, ds.cfg, dirs...) - ds.loader = packages.NewGlobLoader(examplesDir, resolver) + var localPaths []string + ds.loader, localPaths = setupPackagesLoader(loaderLogger, ds.cfg, dirs...) // Get user's address book from local keybase accountLogger := ds.logger.WithGroup(AccountsLogName) @@ -195,8 +192,9 @@ func (ds *App) Setup(ctx context.Context, dirs ...string) (err error) { address := resolveUnixOrTCPAddr(nodeCfg.TMConfig.RPC.ListenAddress) - // Setup lazy proxy - if ds.cfg.lazyLoader { + // Setup lazy proxy (enabled for auto and lazy modes, disabled for full mode) + enableProxy := ds.cfg.loadMode != LoadModeFull + if enableProxy { proxyLogger := ds.logger.WithGroup(ProxyLogName) ds.proxy, err = proxy.NewPathInterceptor(proxyLogger, address) if err != nil { @@ -222,14 +220,32 @@ func (ds *App) Setup(ctx context.Context, dirs ...string) (err error) { } ds.DeferClose(ds.devNode.Close) - // Setup default web home realm, fallback on first local path + // Setup default web home realm, only considering realm paths (/r/) devNodePaths := ds.devNode.Paths() + // Filter to only realm paths + realmPaths := make([]string, 0, len(devNodePaths)) + for _, p := range devNodePaths { + if strings.Contains(p, "/r/") { + realmPaths = append(realmPaths, p) + } + } + switch webHome := ds.cfg.webHome; webHome { case "": - if len(devNodePaths) > 0 { - ds.webHomePath = strings.TrimPrefix(devNodePaths[0], ds.cfg.chainDomain) - ds.logger.WithGroup(WebLogName).Info("using default package", "path", devNodePaths[0]) + // Only set web home if there are realm paths + if len(realmPaths) > 0 { + var homePath string + if ds.cfg.loadMode == LoadModeAuto && len(realmPaths) > 1 { + // Compute highest common root for auto mode with multiple realms + homePath = commonPathPrefix(realmPaths) + ds.logger.WithGroup(WebLogName).Info("using common root", "path", homePath) + } else { + // Single realm or non-auto mode: use first realm path + homePath = realmPaths[0] + ds.logger.WithGroup(WebLogName).Info("using default realm", "path", homePath) + } + ds.webHomePath = strings.TrimPrefix(homePath, ds.cfg.chainDomain) } case "/", ":none:": // skip default: @@ -261,7 +277,7 @@ func (ds *App) setupHandlers(ctx context.Context) (http.Handler, error) { // Generate initial paths initPaths := map[string]struct{}{} for _, pkg := range ds.devNode.ListPkgs() { - initPaths[pkg.Path] = struct{}{} + initPaths[pkg.ImportPath] = struct{}{} } ds.proxy.HandlePath(func(paths ...string) { @@ -274,7 +290,6 @@ func (ds *App) setupHandlers(ctx context.Context) (http.Handler, error) { // Try to resolve the path first. // If we are unable to resolve it, ignore and continue - if _, err := ds.loader.Resolve(path); err != nil { proxyLogger.Debug("unable to resolve path", "error", err, @@ -545,6 +560,37 @@ func (ds *App) handleKeyPress(ctx context.Context, key rawterm.KeyPress) { } } +// commonPathPrefix returns the highest common directory prefix of all paths. +// For example: ["a/b/x", "a/b/y"] -> "a/b" +func commonPathPrefix(paths []string) string { + switch len(paths) { + case 0: + return "" + case 1: + return paths[0] + } + + // Split first path into segments + parts := strings.Split(paths[0], "/") + + // Find common prefix length across all paths + for _, p := range paths[1:] { + otherParts := strings.Split(p, "/") + + // Trim parts to common length + newLen, minLen := 0, min(len(parts), len(otherParts)) + for i := range minLen { + if parts[i] != otherParts[i] { + break + } + newLen = i + 1 + } + parts = parts[:newLen] + } + + return strings.Join(parts, "/") +} + // XXX: packages modifier does not support glob yet func resolvePackagesModifier(cfg *AppConfig, bk *address.Book, qpaths []string) ([]gnodev.QueryPath, []string, error) { modifiers := make([]gnodev.QueryPath, 0, len(qpaths)) diff --git a/contribs/gnodev/app_config.go b/contribs/gnodev/app_config.go index 56b22c8b98a..02713b52cd5 100644 --- a/contribs/gnodev/app_config.go +++ b/contribs/gnodev/app_config.go @@ -1,6 +1,47 @@ package main -import "flag" +import ( + "flag" + "fmt" +) + +// LoadMode defines the package loading strategy +type LoadMode string + +const ( + // LoadModeAuto pre-loads current workspace/package only (not examples). + // If running from examples folder, behaves like lazy mode. + LoadModeAuto LoadMode = "auto" + // LoadModeLazy loads packages on-demand as they're accessed. + LoadModeLazy LoadMode = "lazy" + // LoadModeFull pre-loads all discovered packages. + LoadModeFull LoadMode = "full" +) + +func (m LoadMode) String() string { return string(m) } + +func (m *LoadMode) Set(s string) error { + switch LoadMode(s) { + case LoadModeAuto, LoadModeLazy, LoadModeFull: + *m = LoadMode(s) + return nil + default: + return fmt.Errorf("invalid load mode %q: must be auto, lazy, or full", s) + } +} + +// varResolver is a placeholder for the deprecated resolver flag. +// The new NativeLoader handles package resolution automatically. +type varResolver []string + +func (va varResolver) String() string { + return fmt.Sprintf("%v", []string(va)) +} + +func (va *varResolver) Set(value string) error { + *va = append(*va, value) + return nil +} type AppConfig struct { // Listeners @@ -32,7 +73,7 @@ type AppConfig struct { // Node Configuration logFormat string - lazyLoader bool + loadMode LoadMode verbose bool noWatch bool noReplay bool @@ -115,7 +156,7 @@ func (c *AppConfig) RegisterFlagsWith(fs *flag.FlagSet, defaultCfg AppConfig) { fs.Var( &c.resolvers, "resolver", - "list of additional resolvers (`root`, `local`, or `remote`) in the form of = will be executed in the given order", + "[DEPRECATED] this flag is ignored; package resolution is now handled automatically via gnomod.toml and gnowork.toml discovery", ) fs.StringVar( @@ -187,11 +228,10 @@ func (c *AppConfig) RegisterFlagsWith(fs *flag.FlagSet, defaultCfg AppConfig) { "do not replay previous transactions upon reload", ) - fs.BoolVar( - &c.lazyLoader, - "lazy-loader", - defaultCfg.lazyLoader, - "enable lazy loader", + fs.Var( + &c.loadMode, + "load", + "package loading mode: `auto` (pre-load current workspace/package), `lazy` (load on-demand), `full` (pre-load all discovered packages)", ) fs.Int64Var( diff --git a/contribs/gnodev/command_local.go b/contribs/gnodev/command_local.go index 168e4a9e325..bc68a636993 100644 --- a/contribs/gnodev/command_local.go +++ b/contribs/gnodev/command_local.go @@ -6,10 +6,7 @@ import ( "flag" "fmt" "os" - "path/filepath" - "strings" - "github.com/gnolang/gno/contribs/gnodev/pkg/packages" "github.com/gnolang/gno/gnovm/pkg/gnoenv" "github.com/gnolang/gno/tm2/pkg/commands" "github.com/mattn/go-isatty" @@ -37,7 +34,7 @@ var defaultLocalAppConfig = AppConfig{ root: gnoenv.RootDir(), interactive: isatty.IsTerminal(os.Stdout.Fd()), unsafeAPI: true, - lazyLoader: true, + loadMode: LoadModeAuto, emptyBlocks: false, emptyBlocksInterval: 1, @@ -57,10 +54,25 @@ func NewLocalCmd(io commands.IO) *commands.Command { ShortHelp: "Start gnodev in local development mode (default)", LongHelp: `LOCAL: Local mode configures the node for local development usage. This mode is optimized for realm development, providing an interactive and flexible environment. -It enables features such as interactive mode, unsafe API access for testing, and lazy loading to improve performance. -The log format is set to console for easier readability, and the web interface is accessible locally, making it ideal for iterative development and testing. - -By default, the current directory and the "example" folder from "gnoroot" will be used as the root resolver. +It enables features such as interactive mode, unsafe API access for testing, and on-demand +package loading. The log format is set to console for easier readability, and the web +interface is accessible locally, making it ideal for iterative development and testing. + +LOAD MODES (-load flag): + auto Pre-load current workspace/package only. If running from the examples folder, + uses lazy loading instead. (default) + lazy Load packages on-demand as they are accessed via queries or transactions. + full Pre-load all discovered packages under the chain domain. + +PACKAGE DISCOVERY: + - If the current directory contains a gnomod.toml file, the package is automatically + detected and loaded using the module path defined in the file. + - If the current directory contains a gnowork.toml file, it is treated as a workspace + and all packages within are discovered. + - Additional package directories can be passed as arguments. + - The -paths flag can be used to pre-load additional packages on top of the load mode. + +The examples folder from GNOROOT is always included as a workspace for dependency resolution. `, NoParentFlags: true, }, @@ -94,37 +106,11 @@ func execLocalApp(cfg *LocalAppConfig, args []string, cio commands.IO) error { return fmt.Errorf("unable to guess current dir: %w", err) } - // If no resolvers is defined, use gno example as root resolver - var baseResolvers []packages.Resolver - - if len(cfg.resolvers) == 0 { - // Check if we are not in gnoroot - if !strings.HasPrefix(dir, filepath.Clean(cfg.root)+"/") { - // Add current dir as root resolvers - baseResolvers = append(baseResolvers, packages.NewRootResolver(dir)) - } - - // Add examples as root resolver - gnoroot, err := gnoenv.GuessRootDir() - if err != nil { - return err - } - exampleRoot := filepath.Join(gnoroot, "examples") - baseResolvers = append(baseResolvers, packages.NewRootResolver(exampleRoot)) - } - - // Check if current directory is a valid gno package - path := guessPath(&cfg.AppConfig, dir) - resolver := packages.NewLocalResolver(path, dir) - if resolver.IsValid() { - // Add current directory as local resolver - baseResolvers = append([]packages.Resolver{resolver}, baseResolvers...) - if len(cfg.paths) > 0 { - cfg.paths += "," - } - cfg.paths += resolver.Path - } - cfg.resolvers = append(baseResolvers, cfg.resolvers...) + // Always add current directory as workspace root for discovery + // (even if it's not itself a gno package, it may contain packages in subdirs) + // The load mode will determine what gets pre-loaded from these directories + args = append([]string{dir}, args...) - return runApp(&cfg.AppConfig, cio) // else run app without any dir + // If args are provided, they are directories to add + return runApp(&cfg.AppConfig, cio, args...) } diff --git a/contribs/gnodev/command_staging.go b/contribs/gnodev/command_staging.go index 6e593f517e0..acc03e539dd 100644 --- a/contribs/gnodev/command_staging.go +++ b/contribs/gnodev/command_staging.go @@ -3,10 +3,7 @@ package main import ( "context" "flag" - "path" - "path/filepath" - "github.com/gnolang/gno/contribs/gnodev/pkg/packages" "github.com/gnolang/gno/gnovm/pkg/gnoenv" "github.com/gnolang/gno/tm2/pkg/commands" ) @@ -28,11 +25,9 @@ var defaultStagingOptions = AppConfig{ root: gnoenv.RootDir(), interactive: false, unsafeAPI: false, - lazyLoader: false, - paths: path.Join(DefaultDomain, "/**"), // Load every package under the main domain}, + loadMode: LoadModeFull, // Pre-load all packages emptyBlocks: false, emptyBlocksInterval: 1, - // As we have no reason to configure this yet, set this to random port // to avoid potential conflict with other app nodeP2PListenerAddr: "tcp://127.0.0.1:0", @@ -51,9 +46,13 @@ func NewStagingCmd(io commands.IO) *commands.Command { This mode is designed for stability and security, suitable for pre-deployment testing. Interactive mode and unsafe API access are disabled to ensure a secure environment. The log format is set to JSON, facilitating integration with logging systems. -Since lazy-load is disabled in this mode, the entire example folder from "gnoroot" is loaded by default. -Additionally, you can specify an additional package directory to load. +PACKAGE LOADING: +This mode uses -load=full by default, which pre-loads all discovered packages. +The lazy loading proxy is disabled in this mode. + +Additional package directories can be passed as arguments. +Use -load=auto or -load=lazy to change the loading behavior. `, NoParentFlags: true, }, @@ -69,16 +68,5 @@ func (c *StagingAppConfig) RegisterFlags(fs *flag.FlagSet) { } func execStagingCmd(cfg *StagingAppConfig, args []string, io commands.IO) error { - // If no resolvers is defined, use gno example as root resolver - if len(cfg.AppConfig.resolvers) == 0 { - gnoroot, err := gnoenv.GuessRootDir() - if err != nil { - return err - } - - exampleRoot := filepath.Join(gnoroot, "examples") - cfg.AppConfig.resolvers = append(cfg.AppConfig.resolvers, packages.NewRootResolver(exampleRoot)) - } - return runApp(&cfg.AppConfig, io, args...) } diff --git a/contribs/gnodev/main.go b/contribs/gnodev/main.go index 78dc5b7e978..e93ebf1d9cc 100644 --- a/contribs/gnodev/main.go +++ b/contribs/gnodev/main.go @@ -21,40 +21,28 @@ func main() { Name: "gnodev", ShortUsage: "gnodev [flags] ", ShortHelp: "Runs an in-memory node and gno.land web server for development purposes.", - LongHelp: `The gnodev command starts an in-memory node and a gno.land -web interface, primarily for realm package development. + LongHelp: `The gnodev command starts an in-memory node and a gno.land web interface +for realm package development. -Currently gnodev comes with two mode and , those command mostly -differ by there default values, while gnodev local as default for working -locally, satging default are oriented to be use on server. +LOAD MODES (-load flag): + auto Pre-load current workspace/package only (default for local) + lazy Load all packages on-demand + full Pre-load all discovered packages (default for staging) -gnodev uses its own package loader and resolver system to support multiple -scenarios and use cases. It currently supports three types of resolvers, each -taking a location as an argument. -- root: This resolver takes a as its location. It attempts to resolve - packages based on your file system structure and the package path. - For example, if 'root=/user/gnome/myproject' and you try to resolve - 'gno.land/r/bar/buzz' as a package, the resolver will attempt to - resolve it to /user/gnome/myproject/gno.land/r/bar/buzz. -- local: This resolver also takes a as its location. It is designed to - load a single package, using the module name from 'gnomod.toml' within this - package to resolve the package. -- remote: This resolver takes a RPC address as its location. It is - meant to use a remote node as a resolver, primarily for testing a local - package against a remote node. +PACKAGE DISCOVERY: +gnodev automatically discovers packages based on your project structure: -Resolvers can be chained, and gnodev will attempt to use them in the order they -are declared. + - gnomod.toml: Marks a directory as a Gno package + - gnowork.toml: Marks a directory as a workspace containing multiple packages -For example: - gnodev -resolver root=/user/gnome/myproject -resolver remote=https://rpc.gno.lands +EXAMPLES: + gnodev Start with auto-detection (pre-loads current package) + gnodev -load=lazy Start with on-demand loading only + gnodev -load=full Pre-load all discovered packages + gnodev ./myrealm Load package from ./myrealm directory + gnodev -paths "gno.land/r/demo/**" Pre-load additional packages -If no resolvers can resolve a given package path, the loader will return a -"package not found" error. - -If no command is provided, gnodev will automatically start in mode. - -For more information and flags usage description, use 'gnodev local -h'.`, +For detailed flags, use 'gnodev local -h' or 'gnodev staging -h'.`, }, nil, func(ctx context.Context, _ []string) error { diff --git a/contribs/gnodev/pkg/dev/node.go b/contribs/gnodev/pkg/dev/node.go index b0dd4a22e98..f612c991e12 100644 --- a/contribs/gnodev/pkg/dev/node.go +++ b/contribs/gnodev/pkg/dev/node.go @@ -96,8 +96,10 @@ func DefaultNodeConfig(rootdir, domain string) *NodeConfig { }, } - exampleFolder := filepath.Join(gnoenv.RootDir(), "example") // XXX: we should avoid having to hardcoding this here - defaultLoader := packages.NewLoader(packages.NewRootResolver(exampleFolder)) + examplesDir := filepath.Join(gnoenv.RootDir(), "examples") + defaultLoader := packages.NewNativeLoader(packages.NativeLoaderConfig{ + ExtraWorkspaces: []string{examplesDir}, + }) return &NodeConfig{ Logger: log.NewNoopLogger(), @@ -124,7 +126,7 @@ type Node struct { client client.Client logger *slog.Logger loader packages.Loader - pkgs []packages.Package + pkgs []*packages.Package pkgsModifier map[string]QueryPath // path -> QueryPath paths []string @@ -185,7 +187,7 @@ func (n *Node) Close() error { return n.Node.Stop() } -func (n *Node) ListPkgs() []packages.Package { +func (n *Node) ListPkgs() []*packages.Package { n.muNode.RLock() defer n.muNode.RUnlock() @@ -225,7 +227,7 @@ func (n *Node) HasPackageLoaded(path string) bool { defer n.muNode.RUnlock() for _, pkg := range n.pkgs { - if pkg.MemPackage.Path == path { + if pkg.ImportPath == path { return true } } @@ -425,16 +427,23 @@ func (n *Node) getBlockStoreState(ctx context.Context) ([]gnoland.TxWithMetadata return state, nil } -func (n *Node) generateTxs(fee std.Fee, pkgs []packages.Package) []gnoland.TxWithMetadata { +func (n *Node) generateTxs(fee std.Fee, pkgs []*packages.Package) []gnoland.TxWithMetadata { metatxs := make([]gnoland.TxWithMetadata, 0, len(pkgs)) for _, pkg := range pkgs { + // Read full MemPackage content on demand + mempkg, err := pkg.ToMemPackage() + if err != nil { + n.logger.Error("failed to read package", "path", pkg.ImportPath, "err", err) + continue + } + msg := vm.MsgAddPackage{ Creator: n.config.DefaultCreator, MaxDeposit: n.config.DefaultDeposit, - Package: &pkg.MemPackage, + Package: mempkg, } - if m, ok := n.pkgsModifier[pkg.Path]; ok { + if m, ok := n.pkgsModifier[pkg.ImportPath]; ok { if !m.Creator.IsZero() { msg.Creator = m.Creator } @@ -444,7 +453,7 @@ func (n *Node) generateTxs(fee std.Fee, pkgs []packages.Package) []gnoland.TxWit } n.logger.Debug("applying pkgs modifier", - "path", pkg.Path, + "path", pkg.ImportPath, "creator", msg.Creator, "deposit", msg.MaxDeposit, ) @@ -601,6 +610,8 @@ func (n *Node) rebuildNode(ctx context.Context, genesis gnoland.GnoGenesisState) // Execute node creation and handle any errors. defer recoverFromError() + n.logger.Info("starting node", "pkgs", len(genesis.Txs)) + // XXX: Redirect the node log somewhere else node, nodeErr := gnoland.NewInMemoryNode(noopLogger, nodeConfig) if nodeErr != nil { @@ -621,6 +632,7 @@ func (n *Node) rebuildNode(ctx context.Context, genesis gnoland.GnoGenesisState) return ctx.Err() } + n.logger.Info("node ready") return nil } diff --git a/contribs/gnodev/pkg/dev/node_test.go b/contribs/gnodev/pkg/dev/node_test.go index d48322bc6cd..bf808430033 100644 --- a/contribs/gnodev/pkg/dev/node_test.go +++ b/contribs/gnodev/pkg/dev/node_test.go @@ -35,6 +35,8 @@ func TestNewNode_NoPackages(t *testing.T) { // Call NewDevNode with no package should work cfg := DefaultNodeConfig(gnoenv.RootDir(), "gno.land") cfg.Logger = logger + // Use an empty mock loader for testing without packages + cfg.Loader = packages.NewMockLoader() node, err := NewDevNode(ctx, cfg) require.NoError(t, err) @@ -68,7 +70,7 @@ func Render(_ string) string { return "foo" } logger := log.NewTestingLogger(t) cfg := DefaultNodeConfig(gnoenv.RootDir(), "gno.land") - cfg.Loader = packages.NewLoader(packages.NewMockResolver(&pkg)) + cfg.Loader = packages.NewMockLoader(&pkg) cfg.Logger = logger node, err := NewDevNode(ctx, cfg, pkg.Path) @@ -596,7 +598,6 @@ func testingCallRealmWithConfig(t *testing.T, node *Node, bcfg gnoclient.BaseTxC } func newTestingNodeConfig(pkgs ...*std.MemPackage) *NodeConfig { - var loader packages.BaseLoader gnoroot := gnoenv.RootDir() // Ensure that a gnomod.toml exists @@ -609,12 +610,9 @@ func newTestingNodeConfig(pkgs ...*std.MemPackage) *NodeConfig { pkg.Sort() } - loader.Resolver = packages.MiddlewareResolver( - packages.NewMockResolver(pkgs...), - packages.FilterStdlibs) cfg := DefaultNodeConfig(gnoenv.RootDir(), "gno.land") cfg.TMConfig = integration.DefaultTestingTMConfig(gnoroot) - cfg.Loader = &loader + cfg.Loader = packages.NewMockLoader(pkgs...) return cfg } diff --git a/contribs/gnodev/pkg/packages/glob.go b/contribs/gnodev/pkg/packages/glob.go deleted file mode 100644 index 1b76425deb4..00000000000 --- a/contribs/gnodev/pkg/packages/glob.go +++ /dev/null @@ -1,214 +0,0 @@ -// Inspired by: https://cs.opensource.google/go/x/tools/+/master:gopls/internal/test/integration/fake/glob/glob.go - -package packages - -import ( - "errors" - "fmt" - "strings" -) - -var ErrAdjacentSlash = errors.New("** may only be adjacent to '/'") - -// Glob patterns can have the following syntax: -// - `*` to match one or more characters in a path segment -// - `**` to match any number of path segments, including none -// -// Expanding on this: -// - '/' matches one or more literal slashes. -// - any other character matches itself literally. -type Glob struct { - elems []element // pattern elements -} - -// Parse builds a Glob for the given pattern, returning an error if the pattern -// is invalid. -func Parse(pattern string) (*Glob, error) { - g, _, err := parse(pattern) - return g, err -} - -func parse(pattern string) (*Glob, string, error) { - g := new(Glob) - for len(pattern) > 0 { - switch pattern[0] { - case '/': - // Skip consecutive slashes - for len(pattern) > 0 && pattern[0] == '/' { - pattern = pattern[1:] - } - g.elems = append(g.elems, slash{}) - - case '*': - if len(pattern) > 1 && pattern[1] == '*' { - if (len(g.elems) > 0 && g.elems[len(g.elems)-1] != slash{}) || (len(pattern) > 2 && pattern[2] != '/') { - return nil, "", ErrAdjacentSlash - } - pattern = pattern[2:] - g.elems = append(g.elems, starStar{}) - break - } - pattern = pattern[1:] - g.elems = append(g.elems, star{}) - - default: - pattern = g.parseLiteral(pattern) - } - } - return g, "", nil -} - -func (g *Glob) parseLiteral(pattern string) string { - end := strings.IndexAny(pattern, "*/") - if end == -1 { - end = len(pattern) - } - g.elems = append(g.elems, literal(pattern[:end])) - return pattern[end:] -} - -func (g *Glob) String() string { - var b strings.Builder - for _, e := range g.elems { - fmt.Fprint(&b, e) - } - return b.String() -} - -func (g *Glob) StarFreeBase() string { - var b strings.Builder - for _, e := range g.elems { - if e == (star{}) || e == (starStar{}) { - break - } - fmt.Fprint(&b, e) - } - return b.String() -} - -// element holds a glob pattern element, as defined below. -type element fmt.Stringer - -// element types. -type ( - slash struct{} // One or more '/' separators - literal string // string literal, not containing / or * - star struct{} // * - starStar struct{} // ** -) - -func (s slash) String() string { return "/" } -func (l literal) String() string { return string(l) } -func (s star) String() string { return "*" } -func (s starStar) String() string { return "**" } - -// Match reports whether the input string matches the glob pattern. -func (g *Glob) Match(input string) bool { - return match(g.elems, input) -} - -func match(elems []element, input string) (ok bool) { - var elem interface{} - for len(elems) > 0 { - elem, elems = elems[0], elems[1:] - switch elem := elem.(type) { - case slash: - // Skip consecutive slashes in the input - if len(input) == 0 || input[0] != '/' { - return false - } - for len(input) > 0 && input[0] == '/' { - input = input[1:] - } - - case starStar: - // Special cases: - // - **/a matches "a" - // - **/ matches everything - // - // Note that if ** is followed by anything, it must be '/' (this is - // enforced by Parse). - if len(elems) > 0 { - elems = elems[1:] - } - - // A trailing ** matches anything. - if len(elems) == 0 { - return true - } - - // Backtracking: advance pattern segments until the remaining pattern - // elements match. - for len(input) != 0 { - if match(elems, input) { - return true - } - _, input = split(input) - } - return false - - case literal: - if !strings.HasPrefix(input, string(elem)) { - return false - } - input = input[len(elem):] - - case star: - var segInput string - segInput, input = split(input) - - elemEnd := len(elems) - for i, e := range elems { - if e == (slash{}) { - elemEnd = i - break - } - } - segElems := elems[:elemEnd] - elems = elems[elemEnd:] - - // A trailing * matches the entire segment. - if len(segElems) == 0 { - if len(elems) > 0 && elems[0] == (slash{}) { - elems = elems[1:] // shift elems - } - break - } - - // Backtracking: advance characters until remaining subpattern elements - // match. - matched := false - for i := range segInput { - if match(segElems, segInput[i:]) { - matched = true - break - } - } - if !matched { - return false - } - - default: - panic(fmt.Sprintf("segment type %T not implemented", elem)) - } - } - - return len(input) == 0 -} - -// split returns the portion before and after the first slash -// (or sequence of consecutive slashes). If there is no slash -// it returns (input, nil). -func split(input string) (first, rest string) { - i := strings.IndexByte(input, '/') - if i < 0 { - return input, "" - } - first = input[:i] - for j := i; j < len(input); j++ { - if input[j] != '/' { - return first, input[j:] - } - } - return first, "" -} diff --git a/contribs/gnodev/pkg/packages/glob_test.go b/contribs/gnodev/pkg/packages/glob_test.go deleted file mode 100644 index 7fad4eb2fe1..00000000000 --- a/contribs/gnodev/pkg/packages/glob_test.go +++ /dev/null @@ -1,93 +0,0 @@ -// Inspired by: https://cs.opensource.google/go/x/tools/+/master:gopls/internal/test/integration/fake/glob/glob_test.go - -package packages - -import ( - "testing" - - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestMatch(t *testing.T) { - t.Parallel() - - tests := []struct { - pattern, input string - want bool - }{ - // Basic cases. - {"", "", true}, - {"", "a", false}, - {"", "/", false}, - {"abc", "abc", true}, - - // ** behavior - {"**", "abc", true}, - {"**/abc", "abc", true}, - {"**", "abc/def", true}, - - // * behavior - {"/*", "/a", true}, - {"*", "foo", true}, - {"*o", "foo", true}, - {"*o", "foox", false}, - {"f*o", "foo", true}, - {"f*o", "fo", true}, - - // Dirs cases - {"**/", "path/to/foo/", true}, - {"**/", "path/to/foo", true}, - - {"path/to/foo", "path/to/foo", true}, - {"path/to/foo", "path/to/bar", false}, - {"path/*/foo", "path/to/foo", true}, - {"path/*/1/*/3/*/5*/foo", "path/to/1/2/3/4/522/foo", true}, - {"path/*/1/*/3/*/5*/foo", "path/to/1/2/3/4/722/foo", false}, - {"path/*/1/*/3/*/5*/foo", "path/to/1/2/3/4/522/bar", false}, - {"path/*/foo", "path/to/to/foo", false}, - {"path/**/foo", "path/to/to/foo", true}, - {"path/**/foo", "path/to/to/bar", false}, - {"path/**/foo", "path/foo", true}, - {"**/abc/**", "foo/r/x/abc/bar", true}, - - // Realistic examples. - {"**/*.ts", "path/to/foo.ts", true}, - {"**/*.js", "path/to/foo.js", true}, - {"**/*.go", "path/to/foo.go", true}, - } - - for _, test := range tests { - g, err := Parse(test.pattern) - require.NoErrorf(t, err, "Parse(%q) failed unexpectedly: %v", test.pattern, err) - assert.Equalf(t, test.want, g.Match(test.input), - "Parse(%q).Match(%q) = %t, want %t", test.pattern, test.input, !test.want, test.want) - } -} - -func TestBaseFreeStar(t *testing.T) { - t.Parallel() - - tests := []struct { - pattern, baseFree string - }{ - // Basic cases. - {"", ""}, - {"foo", "foo"}, - {"foo/bar", "foo/bar"}, - {"foo///bar", "foo/bar"}, - {"foo/bar/", "foo/bar/"}, - {"foo/bar/*/*/z", "foo/bar/"}, - {"foo/bar/**", "foo/bar/"}, - {"**", ""}, - {"/**", "/"}, - } - - for _, test := range tests { - g, err := Parse(test.pattern) - require.NoErrorf(t, err, "Parse(%q) failed unexpectedly: %v", test.pattern, err) - got := g.StarFreeBase() - assert.Equalf(t, test.baseFree, got, - "Parse(%q).Match(%q) = %q, want %q", test.pattern, test.baseFree, got, test.baseFree) - } -} diff --git a/contribs/gnodev/pkg/packages/index.go b/contribs/gnodev/pkg/packages/index.go new file mode 100644 index 00000000000..c85af5f9b8e --- /dev/null +++ b/contribs/gnodev/pkg/packages/index.go @@ -0,0 +1,73 @@ +package packages + +import ( + "sync" +) + +// PathIndex maintains a mapping between import paths and filesystem directories. +// This is needed for lazy loading where package paths are returned but we need +// to resolve them to filesystem directories. +type PathIndex struct { + mu sync.RWMutex + byPath map[string]*Package // ImportPath -> Package + byDir map[string]*Package // Dir -> Package +} + +func NewPathIndex() *PathIndex { + return &PathIndex{ + byPath: make(map[string]*Package), + byDir: make(map[string]*Package), + } +} + +func (idx *PathIndex) Add(pkg *Package) { + idx.mu.Lock() + defer idx.mu.Unlock() + + idx.byPath[pkg.ImportPath] = pkg + if pkg.Dir != "" { + idx.byDir[pkg.Dir] = pkg + } +} + +func (idx *PathIndex) GetByPath(importPath string) (*Package, bool) { + idx.mu.RLock() + defer idx.mu.RUnlock() + + pkg, ok := idx.byPath[importPath] + return pkg, ok +} + +func (idx *PathIndex) GetByDir(dir string) (*Package, bool) { + idx.mu.RLock() + defer idx.mu.RUnlock() + + pkg, ok := idx.byDir[dir] + return pkg, ok +} + +func (idx *PathIndex) List() []*Package { + idx.mu.RLock() + defer idx.mu.RUnlock() + + pkgs := make([]*Package, 0, len(idx.byPath)) + for _, pkg := range idx.byPath { + pkgs = append(pkgs, pkg) + } + return pkgs +} + +func (idx *PathIndex) Clear() { + idx.mu.Lock() + defer idx.mu.Unlock() + + idx.byPath = make(map[string]*Package) + idx.byDir = make(map[string]*Package) +} + +func (idx *PathIndex) Len() int { + idx.mu.RLock() + defer idx.mu.RUnlock() + + return len(idx.byPath) +} diff --git a/contribs/gnodev/pkg/packages/loader.go b/contribs/gnodev/pkg/packages/loader.go index 3bc978721e6..87433c7658f 100644 --- a/contribs/gnodev/pkg/packages/loader.go +++ b/contribs/gnodev/pkg/packages/loader.go @@ -1,8 +1,8 @@ package packages type Loader interface { - // Load resolves package package paths and all their dependencies in the correct order. - Load(paths ...string) ([]Package, error) + // Load resolves package paths and all their dependencies in the correct order. + Load(paths ...string) ([]*Package, error) // Resolve processes a single package path and returns the corresponding Package. Resolve(path string) (*Package, error) diff --git a/contribs/gnodev/pkg/packages/loader_base.go b/contribs/gnodev/pkg/packages/loader_base.go deleted file mode 100644 index f931f3be488..00000000000 --- a/contribs/gnodev/pkg/packages/loader_base.go +++ /dev/null @@ -1,110 +0,0 @@ -package packages - -import ( - "errors" - "fmt" - "go/parser" - "go/token" - "strings" -) - -type BaseLoader struct { - Resolver -} - -func NewLoader(res ...Resolver) *BaseLoader { - return &BaseLoader{ChainResolvers(res...)} -} - -func (l BaseLoader) Name() string { - return l.Resolver.Name() -} - -func (l BaseLoader) Load(paths ...string) ([]Package, error) { - fset := token.NewFileSet() - visited, stack := map[string]bool{}, map[string]bool{} - pkgs := make([]Package, 0) - for _, root := range paths { - deps, err := load(root, fset, l.Resolver, visited, stack) - if err != nil { - return nil, err - } - pkgs = append(pkgs, deps...) - } - - return pkgs, nil -} - -func (l BaseLoader) Resolve(path string) (*Package, error) { - fset := token.NewFileSet() - return l.Resolver.Resolve(fset, path) -} - -func load(path string, fset *token.FileSet, resolver Resolver, visited, stack map[string]bool) ([]Package, error) { - if stack[path] { - return nil, fmt.Errorf("cycle detected: %s", path) - } - if visited[path] { - return nil, nil - } - - visited[path] = true - - mempkg, err := resolver.Resolve(fset, path) - if err != nil { - if errors.Is(err, ErrResolverPackageSkip) { - return nil, nil - } - - return nil, fmt.Errorf("unable to resolve package %q: %w", path, err) - } - - var name string - imports := map[string]struct{}{} - for _, file := range mempkg.Files { - fname := file.Name - if !isGnoFile(fname) { - continue - } - - f, err := parser.ParseFile(fset, fname, file.Body, parser.ImportsOnly) - if err != nil { - return nil, fmt.Errorf("unable to parse file %q: %w", file.Name, err) - } - - for _, imp := range f.Imports { - if len(imp.Path.Value) <= 2 { - continue - } - - val := imp.Path.Value[1 : len(imp.Path.Value)-1] - imports[val] = struct{}{} - } - - if strings.HasSuffix(fname, "_filetest.gno") { - continue - } - - pname := strings.TrimSuffix(f.Name.Name, "_test") - if name != "" && name != pname { - return nil, fmt.Errorf("conflict package name between %q and %q", name, f.Name.Name) - } - - name = pname - } - - pkgs := []Package{} - for imp := range imports { - subDeps, err := load(imp, fset, resolver, visited, stack) - if err != nil { - return nil, fmt.Errorf("importing %q: %w", imp, err) - } - - pkgs = append(pkgs, subDeps...) - } - pkgs = append(pkgs, *mempkg) - - stack[path] = false - - return pkgs, nil -} diff --git a/contribs/gnodev/pkg/packages/loader_glob.go b/contribs/gnodev/pkg/packages/loader_glob.go deleted file mode 100644 index 6e3d1158cb5..00000000000 --- a/contribs/gnodev/pkg/packages/loader_glob.go +++ /dev/null @@ -1,97 +0,0 @@ -package packages - -import ( - "fmt" - "go/token" - "io/fs" - "os" - "path" - "path/filepath" - "strings" -) - -type GlobLoader struct { - Root string - Resolver Resolver -} - -func NewGlobLoader(rootpath string, res ...Resolver) *GlobLoader { - return &GlobLoader{rootpath, ChainResolvers(res...)} -} - -func (l GlobLoader) Name() string { - return l.Resolver.Name() -} - -func (l GlobLoader) MatchPaths(globs ...string) ([]string, error) { - if l.Root == "" { - return globs, nil - } - - if _, err := os.Stat(l.Root); err != nil { - return nil, fmt.Errorf("unable to stat root: %w", err) - } - - mpaths := []string{} - for _, input := range globs { - cleanInput := path.Clean(input) - gpath, err := Parse(cleanInput) - if err != nil { - return nil, fmt.Errorf("invalid glob path %q: %w", input, err) - } - - base := gpath.StarFreeBase() - if base == cleanInput { - mpaths = append(mpaths, base) - continue - } - - root := l.Root - err = filepath.WalkDir(root, func(dirpath string, d fs.DirEntry, err error) error { - if err != nil { - return err - } - - path, err := filepath.Rel(root, dirpath) - if err != nil { - return err - } - - // normalize filepath to path - path = NormalizeFilepathToPath(path) - - if !d.IsDir() { - return nil - } - - if strings.HasPrefix(d.Name(), ".") { - return fs.SkipDir - } - - if gpath.Match(path) { - mpaths = append(mpaths, path) - } - - return nil - }) - if err != nil { - return nil, fmt.Errorf("walking directory %q: %w", root, err) - } - } - - return mpaths, nil -} - -func (l GlobLoader) Load(gpaths ...string) ([]Package, error) { - paths, err := l.MatchPaths(gpaths...) - if err != nil { - return nil, fmt.Errorf("match glob pattern error: %w", err) - } - - loader := &BaseLoader{Resolver: l.Resolver} - return loader.Load(paths...) -} - -func (l GlobLoader) Resolve(path string) (*Package, error) { - return l.Resolver.Resolve(token.NewFileSet(), path) -} diff --git a/contribs/gnodev/pkg/packages/loader_native.go b/contribs/gnodev/pkg/packages/loader_native.go new file mode 100644 index 00000000000..1dd22943b4e --- /dev/null +++ b/contribs/gnodev/pkg/packages/loader_native.go @@ -0,0 +1,240 @@ +package packages + +import ( + "fmt" + "io" + "io/fs" + "log/slog" + "os" + "path/filepath" + "strings" + + "github.com/gnolang/gno/gnovm/pkg/gnoenv" + "github.com/gnolang/gno/gnovm/pkg/gnolang" + "github.com/gnolang/gno/gnovm/pkg/gnomod" + vmpackages "github.com/gnolang/gno/gnovm/pkg/packages" + "github.com/gnolang/gno/gnovm/pkg/packages/pkgdownload/rpcpkgfetcher" +) + +type NativeLoader struct { + index *PathIndex + logger *slog.Logger + gnoRoot string + extraWorkspaces []string + remoteOverrides map[string]string // domain -> rpc URL +} + +type NativeLoaderConfig struct { + Logger *slog.Logger + GnoRoot string + ExtraWorkspaces []string + RemoteOverrides map[string]string +} + +// logWriter wraps a logger as an io.Writer +type logWriter struct { + logger *slog.Logger +} + +func (w *logWriter) Write(p []byte) (n int, err error) { + w.logger.Info(strings.TrimSpace(string(p))) + return len(p), nil +} + +func NewNativeLoader(cfg NativeLoaderConfig) *NativeLoader { + logger := cfg.Logger + if logger == nil { + logger = slog.New(slog.NewTextHandler(io.Discard, nil)) + } + gnoRoot := cfg.GnoRoot + if gnoRoot == "" { + gnoRoot = gnoenv.RootDir() + } + remoteOverrides := cfg.RemoteOverrides + if remoteOverrides == nil { + remoteOverrides = make(map[string]string) + } + return &NativeLoader{ + index: NewPathIndex(), + logger: logger, + gnoRoot: gnoRoot, + extraWorkspaces: cfg.ExtraWorkspaces, + remoteOverrides: remoteOverrides, + } +} + +func (l *NativeLoader) Name() string { + return "native" +} + +func (l *NativeLoader) Load(patterns ...string) ([]*Package, error) { + // Configure gnovm packages loader + cfg := vmpackages.LoadConfig{ + Deps: true, + AllowEmpty: true, + Test: true, // Load test file dependencies + GnoRoot: l.gnoRoot, + ExtraWorkspaceRoots: l.extraWorkspaces, + Out: &logWriter{l.logger}, + Fetcher: rpcpkgfetcher.New(l.remoteOverrides), + } + + // Call native loader + pkgList, err := vmpackages.Load(cfg, patterns...) + if err != nil { + return nil, fmt.Errorf("native load failed: %w", err) + } + + // Sort packages by dependencies + sortedPkgs, err := pkgList.Sort() + if err != nil { + return nil, fmt.Errorf("failed to sort packages: %w", err) + } + + // Filter out ignored packages + sortedPkgs = sortedPkgs.GetNonIgnoredPkgs() + + // Convert to gnodev packages and populate index + result := make([]*Package, 0, len(sortedPkgs)) + for _, vmPkg := range sortedPkgs { + // Skip packages with errors + if len(vmPkg.Errors) > 0 { + for _, e := range vmPkg.Errors { + l.logger.Warn("package error", "path", vmPkg.ImportPath, "error", e.Error()) + } + continue + } + + pkg := FromGnoVMPackage(vmPkg) + l.index.Add(pkg) + result = append(result, pkg) + } + + l.logger.Info("packages loaded", "count", len(result)) + return result, nil +} + +func (l *NativeLoader) Resolve(importPath string) (*Package, error) { + // First check the pre-populated index + if pkg, ok := l.index.GetByPath(importPath); ok { + l.logger.Debug("resolved from index", "path", importPath) + return pkg, nil + } + + // Check if it's a stdlib + if gnolang.IsStdlib(importPath) { + dir := filepath.Join(l.gnoRoot, "gnovm", "stdlibs", filepath.FromSlash(importPath)) + if _, err := os.Stat(dir); err == nil { + pkg := &Package{ + ImportPath: importPath, + Dir: dir, + Kind: PackageKindFS, + } + l.index.Add(pkg) + l.logger.Debug("resolved stdlib", "path", importPath) + return pkg, nil + } + return nil, ErrResolverPackageNotFound + } + + // Not found in index - return error + // Note: We don't fall back to Load() here because it requires workspace context + // The index should be pre-populated via DiscoverPackages() for lazy loading to work + l.logger.Debug("package not found in index", "path", importPath) + return nil, ErrResolverPackageNotFound +} + +// DiscoverPackages scans workspace roots and populates the index with all +// discoverable packages. This enables Resolve() to work for lazy loading +// by pre-mapping import paths to filesystem directories. +func (l *NativeLoader) DiscoverPackages() error { + roots := l.extraWorkspaces + + l.logger.Debug("starting package discovery", "roots", roots) + + for _, root := range roots { + root = filepath.Clean(root) + if _, err := os.Stat(root); os.IsNotExist(err) { + l.logger.Debug("workspace root does not exist, skipping", "root", root) + continue + } + + l.logger.Debug("walking workspace root", "root", root) + + err := filepath.WalkDir(root, func(path string, d fs.DirEntry, err error) error { + if err != nil { + return nil // skip errors + } + + // Skip non-files + if d.IsDir() { + // Skip sub-workspaces (they have their own gnowork.toml) + if path != root { + subwork := filepath.Join(path, "gnowork.toml") + if _, err := os.Stat(subwork); err == nil { + return fs.SkipDir + } + } + return nil + } + + // Look for gnomod.toml or gno.mod files + name := d.Name() + if name != "gnomod.toml" && name != "gno.mod" { + return nil + } + + dir := filepath.Dir(path) + + // Skip if we already have this directory indexed + if _, ok := l.index.GetByDir(dir); ok { + return nil + } + + // Parse the gnomod to get the module path + gm, err := gnomod.ParseDir(dir) + if err != nil { + l.logger.Debug("failed to parse gnomod", "dir", dir, "err", err) + return nil // skip invalid + } + + pkg := &Package{ + ImportPath: gm.Module, + Dir: dir, + Kind: PackageKindFS, + } + l.index.Add(pkg) + + return nil + }) + if err != nil { + l.logger.Warn("error walking workspace", "root", root, "err", err) + } + } + + l.logger.Info("packages discovered", "count", l.index.Len()) + return nil +} + +// GetIndex returns the path index for external use (e.g., file watching) +func (l *NativeLoader) GetIndex() *PathIndex { + return l.index +} + +// FromGnoVMPackage converts a gnovm package to gnodev package +func FromGnoVMPackage(pkg *vmpackages.Package) *Package { + kind := PackageKindFS + + // Determine if remote by checking if it's in modcache + modCachePath := gnomod.ModCachePath() + if strings.HasPrefix(filepath.Clean(pkg.Dir), modCachePath) { + kind = PackageKindRemote + } + + return &Package{ + ImportPath: pkg.ImportPath, + Dir: pkg.Dir, + Kind: kind, + Name: pkg.Name, + } +} diff --git a/contribs/gnodev/pkg/packages/loader_test.go b/contribs/gnodev/pkg/packages/loader_test.go deleted file mode 100644 index 521cc9539a9..00000000000 --- a/contribs/gnodev/pkg/packages/loader_test.go +++ /dev/null @@ -1,84 +0,0 @@ -package packages - -import ( - "testing" - - "github.com/gnolang/gno/tm2/pkg/std" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestLoader_LoadWithDeps(t *testing.T) { - t.Parallel() - - fsresolver := NewRootResolver("./testdata") - loader := NewLoader(fsresolver) - - // package c depend on package b - pkgs, err := loader.Load(TestdataPkgC) - require.NoError(t, err) - require.Len(t, pkgs, 3) - for i, path := range []string{TestdataPkgA, TestdataPkgB, TestdataPkgC} { - assert.Equal(t, path, pkgs[i].Path) - } -} - -func TestLoader_ResolverPriority(t *testing.T) { - t.Parallel() - - const commonPath = "abc.yz/t/a" - - pkgA := std.MemPackage{Name: "pkga", Path: commonPath} - resolverA := NewMockResolver(&pkgA) - - pkgB := std.MemPackage{Name: "pkgb", Path: commonPath} - resolverB := NewMockResolver(&pkgB) - - t.Run("pkgA then pkgB", func(t *testing.T) { - t.Parallel() - - loader := NewLoader(resolverA, resolverB) - pkg, err := loader.Resolve(commonPath) - require.NoError(t, err) - require.Equal(t, pkgA.Name, pkg.Name) - require.Equal(t, commonPath, pkg.Path) - }) - - t.Run("pkgB then pkgA", func(t *testing.T) { - t.Parallel() - - loader := NewLoader(resolverB, resolverA) - pkg, err := loader.Resolve(commonPath) - require.NoError(t, err) - require.Equal(t, pkgB.Name, pkg.Name) - require.Equal(t, commonPath, pkg.Path) - }) -} - -func TestLoader_Glob(t *testing.T) { - const root = "./testdata" - cases := []struct { - GlobPath string - PkgResults []string - }{ - {"abc.xy/t/**", append(testdataPkgs, testdataNested...)}, - {"abc.xy/t/nested/*", []string{TestdataNestedA}}, - {"abc.xy/t/**/cc", []string{TestdataPkgA, TestdataPkgB, TestdataPkgC, TestdataNestedC}}, - {"abc.xy/t/*/aa", []string{TestdataNestedA}}, - } - - fsresolver := NewRootResolver("./testdata") - globloader := NewGlobLoader("./testdata", fsresolver) - - for _, tc := range cases { - t.Run(tc.GlobPath, func(t *testing.T) { - pkgs, err := globloader.Load(tc.GlobPath) - require.NoError(t, err) - actuals := make([]string, 0, len(pkgs)) - for _, pkg := range pkgs { - actuals = append(actuals, pkg.Path) - } - assert.Equal(t, tc.PkgResults, actuals) - }) - } -} diff --git a/contribs/gnodev/pkg/packages/mock_loader.go b/contribs/gnodev/pkg/packages/mock_loader.go new file mode 100644 index 00000000000..da73f0fce1f --- /dev/null +++ b/contribs/gnodev/pkg/packages/mock_loader.go @@ -0,0 +1,46 @@ +package packages + +import ( + "github.com/gnolang/gno/tm2/pkg/std" +) + +// MockLoader is a simple loader for testing that uses in-memory packages. +type MockLoader struct { + packages map[string]*std.MemPackage +} + +// NewMockLoader creates a loader from a list of in-memory packages. +func NewMockLoader(pkgs ...*std.MemPackage) *MockLoader { + m := &MockLoader{ + packages: make(map[string]*std.MemPackage, len(pkgs)), + } + for _, pkg := range pkgs { + m.packages[pkg.Path] = pkg + } + return m +} + +func (l *MockLoader) Name() string { + return "mock" +} + +func (l *MockLoader) Load(paths ...string) ([]*Package, error) { + result := make([]*Package, 0, len(paths)) + for _, path := range paths { + pkg, err := l.Resolve(path) + if err != nil { + continue // Skip packages that don't exist + } + result = append(result, pkg) + } + return result, nil +} + +func (l *MockLoader) Resolve(path string) (*Package, error) { + mempkg, ok := l.packages[path] + if !ok { + return nil, ErrResolverPackageNotFound + } + + return NewPackageFromMemPackage(mempkg), nil +} diff --git a/contribs/gnodev/pkg/packages/package.go b/contribs/gnodev/pkg/packages/package.go index 24fd45bb676..637a55f71cd 100644 --- a/contribs/gnodev/pkg/packages/package.go +++ b/contribs/gnodev/pkg/packages/package.go @@ -3,112 +3,71 @@ package packages import ( "errors" "fmt" - "go/parser" - "go/token" - "os" - "strings" "github.com/gnolang/gno/gnovm/pkg/gnolang" - "github.com/gnolang/gno/gnovm/pkg/gnomod" "github.com/gnolang/gno/tm2/pkg/std" ) +// Common errors +var ( + ErrResolverPackageNotFound = errors.New("package not found") + ErrResolverPackageSkip = errors.New("package should be skipped") +) + type PackageKind int const ( - PackageKindOther = iota - PackageKindRemote = iota + PackageKindOther PackageKind = iota + PackageKindRemote PackageKindFS ) +// Package represents a Gno package with its location information. +// Unlike the old design, MemPackage content is loaded on-demand via ToMemPackage(). type Package struct { - std.MemPackage - Kind PackageKind - Location string + ImportPath string // Import path (e.g., gno.land/r/demo/boards) + Dir string // Filesystem directory + Kind PackageKind // FS or Remote + Name string // Package name + + // memPkg is used for mock/test packages that don't have a filesystem directory. + // When set, ToMemPackage returns this directly instead of reading from disk. + memPkg *std.MemPackage } -func ReadPackageFromDir(fset *token.FileSet, path, dir string) (*Package, error) { - if !gnolang.IsUserlib(path) && !gnolang.IsStdlib(path) { - return nil, ErrResolverPackageSkip - } - - mod, err := gnomod.ParseDir(dir) - switch { - case err == nil: - if mod.Ignore { - // Skip ignored package - // XXX: We could potentially do that in a middleware, but doing this - // here avoid to potentially parse broken files - return nil, ErrResolverPackageSkip - } - case errors.Is(err, os.ErrNotExist) || errors.Is(err, gnomod.ErrNoModFile): - // gnomod.toml is not present, continue anyway - default: - return nil, err +// ToMemPackage reads the full package content from disk, +// or returns the mock package if set. +func (p *Package) ToMemPackage() (*std.MemPackage, error) { + // If we have a mock package, return it directly + if p.memPkg != nil { + return p.memPkg, nil } - mempkg, err := gnolang.ReadMemPackage(dir, path, gnolang.MPAnyAll) - switch { - case err == nil: // ok - case os.IsNotExist(err): - return nil, ErrResolverPackageNotFound - case mempkg == nil || mempkg.IsEmpty(): // XXX: should check an internal error instead - return nil, ErrResolverPackageSkip - default: - return nil, fmt.Errorf("unable to read package %q: %w", dir, err) + if p.Dir == "" { + return nil, fmt.Errorf("package %s has no directory", p.ImportPath) } - if err := validateMemPackage(fset, mempkg); err != nil { - return nil, err - } - - return &Package{ - MemPackage: *mempkg, - Location: dir, - Kind: PackageKindFS, - }, nil -} - -func validateMemPackage(fset *token.FileSet, mempkg *std.MemPackage) error { - if isMemPackageEmpty(mempkg) { - return fmt.Errorf("empty package: %w", ErrResolverPackageSkip) + mptype := gnolang.MPUserAll + if gnolang.IsStdlib(p.ImportPath) { + mptype = gnolang.MPStdlibAll } - // Validate package name - for _, file := range mempkg.Files { - if !isGnoFile(file.Name) { - continue - } - - f, err := parser.ParseFile(fset, file.Name, file.Body, parser.PackageClauseOnly) - if err != nil { - return fmt.Errorf("unable to parse file %q: %w", file.Name, err) - } - - if strings.HasSuffix(file.Name, "_filetest.gno") { - continue - } - - pname := strings.TrimSuffix(f.Name.Name, "_test") - if pname != mempkg.Name { - return fmt.Errorf("%q package name conflict, expected %q found %q", - mempkg.Path, mempkg.Name, f.Name.Name) - } + mempkg, err := gnolang.ReadMemPackage(p.Dir, p.ImportPath, mptype) + if err != nil { + return nil, fmt.Errorf("failed to read package %s from %s: %w", p.ImportPath, p.Dir, err) } - return nil + return mempkg, nil } -func isMemPackageEmpty(mempkg *std.MemPackage) bool { - if mempkg.IsEmpty() { - return true - } - - for _, file := range mempkg.Files { - if isGnoFile(file.Name) || file.Name == "gnomod.toml" { - return false - } +// NewPackageFromMemPackage creates a Package from an in-memory MemPackage. +// This is primarily used for testing. +func NewPackageFromMemPackage(mempkg *std.MemPackage) *Package { + return &Package{ + ImportPath: mempkg.Path, + Name: mempkg.Name, + Dir: "", + Kind: PackageKindOther, + memPkg: mempkg, } - - return true } diff --git a/contribs/gnodev/pkg/packages/resolver.go b/contribs/gnodev/pkg/packages/resolver.go deleted file mode 100644 index 3f9a8eea085..00000000000 --- a/contribs/gnodev/pkg/packages/resolver.go +++ /dev/null @@ -1,230 +0,0 @@ -package packages - -import ( - "errors" - "fmt" - "go/parser" - "go/scanner" - "go/token" - "log/slog" - "strings" - "time" -) - -var ( - ErrResolverPackageNotFound = errors.New("package not found") - ErrResolverPackageSkip = errors.New("package has been skip") -) - -type Resolver interface { - Name() string - Resolve(fset *token.FileSet, path string) (*Package, error) -} - -type NoopResolver struct{} - -func (NoopResolver) Name() string { return "" } -func (NoopResolver) Resolve(fset *token.FileSet, path string) (*Package, error) { - return nil, ErrResolverPackageNotFound -} - -// Chain Resolver - -type ChainedResolver []Resolver - -func ChainResolvers(rs ...Resolver) Resolver { - switch len(rs) { - case 0: - return &NoopResolver{} - case 1: - return rs[0] - default: - return ChainedResolver(rs) - } -} - -func (cr ChainedResolver) Name() string { - names := make([]string, 0, len(cr)) - for _, r := range cr { - rname := r.Name() - if rname == "" { - continue - } - - names = append(names, rname) - } - - return strings.Join(names, "/") -} - -func (cr ChainedResolver) Resolve(fset *token.FileSet, path string) (*Package, error) { - for _, resolver := range cr { - pkg, err := resolver.Resolve(fset, path) - if err == nil { - return pkg, nil - } else if errors.Is(err, ErrResolverPackageNotFound) { - continue - } - - return nil, fmt.Errorf("resolver %q error: %w", resolver.Name(), err) - } - - return nil, ErrResolverPackageNotFound -} - -type MiddlewareHandler func(fset *token.FileSet, path string, next Resolver) (*Package, error) - -type middlewareResolver struct { - Handler MiddlewareHandler - Next Resolver -} - -func MiddlewareResolver(r Resolver, handlers ...MiddlewareHandler) Resolver { - // Start with the final resolver - start := r - - // Wrap each handler around the previous one - for _, handler := range handlers { - start = &middlewareResolver{ - Next: start, - Handler: handler, - } - } - - return start -} - -func (r middlewareResolver) Name() string { - return r.Next.Name() -} - -func (r *middlewareResolver) Resolve(fset *token.FileSet, path string) (*Package, error) { - if r.Handler != nil { - return r.Handler(fset, path, r.Next) - } - - return r.Next.Resolve(fset, path) -} - -// LogMiddleware creates a logging middleware handler. -func LogMiddleware(logger *slog.Logger) MiddlewareHandler { - return func(fset *token.FileSet, path string, next Resolver) (*Package, error) { - start := time.Now() - pkg, err := next.Resolve(fset, path) - switch { - case err == nil: - logger.Debug("path resolved", - "resolver", next.Name(), - "path", path, - "name", pkg.Name, - "took", time.Since(start).String(), - "location", pkg.Location, - ) - case errors.Is(err, ErrResolverPackageSkip): - logger.Debug(err.Error(), - "resolver", next.Name(), - "path", path, - "took", time.Since(start).String(), - ) - - case errors.Is(err, ErrResolverPackageNotFound): - logger.Warn(err.Error(), - "resolver", next.Name(), - "path", path, - "took", time.Since(start).String()) - - default: - logger.Error(err.Error(), - "resolver", next.Name(), - "path", path, - "took", time.Since(start).String()) - } - - return pkg, err - } -} - -type ShouldCacheFunc func(pkg *Package) bool - -func CacheAll(_ *Package) bool { return true } - -// CacheMiddleware creates a caching middleware handler. -func CacheMiddleware(shouldCache ShouldCacheFunc) MiddlewareHandler { - cacheMap := make(map[string]*Package) - return func(fset *token.FileSet, path string, next Resolver) (*Package, error) { - if pkg, ok := cacheMap[path]; ok { - return pkg, nil - } - - pkg, err := next.Resolve(fset, path) - if pkg != nil && shouldCache(pkg) { - cacheMap[path] = pkg - } - - return pkg, err - } -} - -// FilterPathHandler defines the function signature for filter handlers. -type FilterPathHandler func(path string) bool - -func FilterPathMiddleware(name string, filter FilterPathHandler) MiddlewareHandler { - return func(fset *token.FileSet, path string, next Resolver) (*Package, error) { - if filter(path) { - return nil, fmt.Errorf("filter %q: %w", name, ErrResolverPackageSkip) - } - - return next.Resolve(fset, path) - } -} - -var FilterStdlibs = FilterPathMiddleware("stdlibs", isStdPath) - -func isStdPath(path string) bool { - if i := strings.IndexRune(path, '/'); i > 0 { - if j := strings.IndexRune(path[:i], '.'); j >= 0 { - return false - } - } - - return true -} - -// PackageCheckerMiddleware creates a middleware handler for post-processing syntax. -func PackageCheckerMiddleware(logger *slog.Logger) MiddlewareHandler { - return func(fset *token.FileSet, path string, next Resolver) (*Package, error) { - // First, resolve the package using the next resolver in the chain. - pkg, err := next.Resolve(fset, path) - if err != nil { - return nil, err - } - - // Post-process each file in the package. - for _, file := range pkg.Files { - fname := file.Name - if !isGnoFile(fname) { - continue - } - - logger.Debug("checking syntax", "path", path, "filename", fname) - _, err := parser.ParseFile(fset, file.Name, file.Body, parser.AllErrors) - if err == nil { - continue - } - - if el, ok := err.(scanner.ErrorList); ok { - for _, e := range el { - logger.Error("syntax error", - "path", path, - "filename", fname, - "err", e.Error(), - ) - } - } - - return nil, fmt.Errorf("file %q have error(s)", file.Name) - } - - return pkg, nil - } -} diff --git a/contribs/gnodev/pkg/packages/resolver_local.go b/contribs/gnodev/pkg/packages/resolver_local.go deleted file mode 100644 index 0312b262403..00000000000 --- a/contribs/gnodev/pkg/packages/resolver_local.go +++ /dev/null @@ -1,40 +0,0 @@ -package packages - -import ( - "fmt" - "go/token" - "path" - "path/filepath" - "strings" -) - -type LocalResolver struct { - Path string - Dir string -} - -func NewLocalResolver(path, dir string) *LocalResolver { - return &LocalResolver{ - Path: path, - Dir: dir, - } -} - -func (r *LocalResolver) Name() string { - return fmt.Sprintf("local<%s>", path.Base(r.Dir)) -} - -func (r LocalResolver) IsValid() bool { - pkg, err := r.Resolve(token.NewFileSet(), r.Path) - return err == nil && pkg != nil -} - -func (r LocalResolver) Resolve(fset *token.FileSet, path string) (*Package, error) { - after, found := strings.CutPrefix(path, r.Path) - if !found { - return nil, ErrResolverPackageNotFound - } - - dir := filepath.Join(r.Dir, after) - return ReadPackageFromDir(fset, path, dir) -} diff --git a/contribs/gnodev/pkg/packages/resolver_mock.go b/contribs/gnodev/pkg/packages/resolver_mock.go deleted file mode 100644 index 8eaf1ba2102..00000000000 --- a/contribs/gnodev/pkg/packages/resolver_mock.go +++ /dev/null @@ -1,40 +0,0 @@ -package packages - -import ( - "go/token" - - "github.com/gnolang/gno/tm2/pkg/std" -) - -type MockResolver struct { - pkgs map[string]*std.MemPackage - resolveCalls map[string]int // Track resolve calls per path -} - -func NewMockResolver(pkgs ...*std.MemPackage) *MockResolver { - mappkgs := make(map[string]*std.MemPackage, len(pkgs)) - for _, pkg := range pkgs { - mappkgs[pkg.Path] = pkg - } - return &MockResolver{ - pkgs: mappkgs, - resolveCalls: make(map[string]int), - } -} - -func (m *MockResolver) ResolveCalls(fset *token.FileSet, path string) int { - count := m.resolveCalls[path] - return count -} - -func (m *MockResolver) Resolve(fset *token.FileSet, path string) (*Package, error) { - m.resolveCalls[path]++ // Increment call count - if mempkg, ok := m.pkgs[path]; ok { - return &Package{MemPackage: *mempkg}, nil - } - return nil, ErrResolverPackageNotFound -} - -func (m *MockResolver) Name() string { - return "mock" -} diff --git a/contribs/gnodev/pkg/packages/resolver_remote.go b/contribs/gnodev/pkg/packages/resolver_remote.go deleted file mode 100644 index 5d3d7563def..00000000000 --- a/contribs/gnodev/pkg/packages/resolver_remote.go +++ /dev/null @@ -1,95 +0,0 @@ -package packages - -import ( - "bytes" - "context" - "errors" - "fmt" - "go/parser" - "go/token" - gopath "path" - - "github.com/gnolang/gno/gno.land/pkg/sdk/vm" - "github.com/gnolang/gno/tm2/pkg/bft/rpc/client" - "github.com/gnolang/gno/tm2/pkg/std" -) - -type remoteResolver struct { - *client.RPCClient - name string - fset *token.FileSet -} - -func NewRemoteResolver(name string, cl *client.RPCClient) Resolver { - return &remoteResolver{ - RPCClient: cl, - name: name, - fset: token.NewFileSet(), - } -} - -func (res *remoteResolver) Name() string { - return fmt.Sprintf("remote<%s>", res.name) -} - -func (res *remoteResolver) Resolve(fset *token.FileSet, path string) (*Package, error) { - const qpath = "vm/qfile" - - // First query files - data := []byte(path) - qres, err := res.RPCClient.ABCIQuery(context.Background(), qpath, data) - if err != nil { - return nil, fmt.Errorf("client unable to query: %w", err) - } - - if err := qres.Response.Error; err != nil { - if errors.Is(err, vm.InvalidFileError{}) || - errors.Is(err, vm.InvalidPkgPathError{}) || - errors.Is(err, vm.InvalidPackageError{}) { - return nil, ErrResolverPackageNotFound - } - - return nil, fmt.Errorf("querying %q error: %w", path, err) - } - - var name string - memFiles := []*std.MemFile{} - files := bytes.Split(qres.Response.Data, []byte{'\n'}) - for _, filename := range files { - fname := string(filename) - fpath := gopath.Join(path, fname) - qres, err := res.RPCClient.ABCIQuery(context.Background(), qpath, []byte(fpath)) - if err != nil { - return nil, fmt.Errorf("unable to query path") - } - - if err := qres.Response.Error; err != nil { - return nil, fmt.Errorf("unable to query file %q on path %q: %w", fname, path, err) - } - body := qres.Response.Data - - // Check package name - if name == "" && isGnoFile(fname) { - // Check package name - f, err := parser.ParseFile(fset, fname, body, parser.PackageClauseOnly) - if err != nil { - return nil, fmt.Errorf("unable to parse file %q: %w", fname, err) - } - name = f.Name.Name - } - - memFiles = append(memFiles, &std.MemFile{ - Name: fname, Body: string(body), - }) - } - - return &Package{ - MemPackage: std.MemPackage{ - Name: name, - Path: path, - Files: memFiles, - }, - Kind: PackageKindRemote, - Location: path, - }, nil -} diff --git a/contribs/gnodev/pkg/packages/resolver_remote_test.go b/contribs/gnodev/pkg/packages/resolver_remote_test.go deleted file mode 100644 index 69347c0ad4d..00000000000 --- a/contribs/gnodev/pkg/packages/resolver_remote_test.go +++ /dev/null @@ -1 +0,0 @@ -package packages diff --git a/contribs/gnodev/pkg/packages/resolver_root.go b/contribs/gnodev/pkg/packages/resolver_root.go deleted file mode 100644 index 4b7b1168433..00000000000 --- a/contribs/gnodev/pkg/packages/resolver_root.go +++ /dev/null @@ -1,35 +0,0 @@ -package packages - -import ( - "fmt" - "go/token" - "os" - "path/filepath" -) - -type rootResolver struct { - root string // Root folder -} - -func NewRootResolver(path string) Resolver { - if abs, err := filepath.Abs(path); err == nil { - return &rootResolver{root: abs} - } - - // fallback on path - return &rootResolver{root: path} -} - -func (r *rootResolver) Name() string { - return fmt.Sprintf("root<%s>", filepath.Base(r.root)) -} - -func (r *rootResolver) Resolve(fset *token.FileSet, path string) (*Package, error) { - dir := filepath.Join(r.root, path) - _, err := os.Stat(dir) - if err != nil { - return nil, ErrResolverPackageNotFound - } - - return ReadPackageFromDir(fset, path, dir) -} diff --git a/contribs/gnodev/pkg/packages/resolver_test.go b/contribs/gnodev/pkg/packages/resolver_test.go deleted file mode 100644 index 2463deec63e..00000000000 --- a/contribs/gnodev/pkg/packages/resolver_test.go +++ /dev/null @@ -1,294 +0,0 @@ -package packages - -import ( - "bytes" - "errors" - "go/token" - "log/slog" - "path/filepath" - "testing" - - "github.com/gnolang/gno/gno.land/pkg/integration" - "github.com/gnolang/gno/gnovm/pkg/gnoenv" - "github.com/gnolang/gno/gnovm/pkg/gnolang" - "github.com/gnolang/gno/tm2/pkg/bft/rpc/client" - "github.com/gnolang/gno/tm2/pkg/crypto/secp256k1" - "github.com/gnolang/gno/tm2/pkg/log" - "github.com/gnolang/gno/tm2/pkg/std" - "github.com/stretchr/testify/assert" - "github.com/stretchr/testify/require" -) - -func TestLogMiddleware(t *testing.T) { - t.Parallel() - - mockResolver := NewMockResolver(&std.MemPackage{ - Path: "abc.xy/test/pkg", - Name: "pkg", - Files: []*std.MemFile{ - {Name: "file.gno", Body: "package pkg"}, - }, - }) - - t.Run("logs package not found", func(t *testing.T) { - t.Parallel() - - var buff bytes.Buffer - - logger := slog.New(slog.NewTextHandler(&buff, &slog.HandlerOptions{})) - middleware := LogMiddleware(logger) - - resolver := MiddlewareResolver(mockResolver, middleware) - pkg, err := resolver.Resolve(token.NewFileSet(), "abc.xy/invalid/pkg") - require.Error(t, err) - require.Nil(t, pkg) - assert.Contains(t, buff.String(), "package not found") - }) - - t.Run("logs package resolution", func(t *testing.T) { - t.Parallel() - - var buff bytes.Buffer - logger := slog.New(slog.NewTextHandler(&buff, &slog.HandlerOptions{ - Level: slog.LevelDebug, - })) - middleware := LogMiddleware(logger) - - resolver := MiddlewareResolver(mockResolver, middleware) - pkg, err := resolver.Resolve(token.NewFileSet(), "abc.xy/test/pkg") - require.NoError(t, err) - require.NotNil(t, pkg) - assert.Contains(t, buff.String(), "path resolved") - }) -} - -func TestCacheMiddleware(t *testing.T) { - t.Parallel() - - pkg := &std.MemPackage{Path: "abc.xy/cached/pkg", Name: "pkg"} - t.Run("caches resolved packages", func(t *testing.T) { - t.Parallel() - - mockResolver := NewMockResolver(pkg) - cacheMiddleware := CacheMiddleware(CacheAll) - cachedResolver := MiddlewareResolver(mockResolver, cacheMiddleware) - - // First call - pkg1, err := cachedResolver.Resolve(token.NewFileSet(), pkg.Path) - require.NoError(t, err) - require.Equal(t, 1, mockResolver.resolveCalls[pkg.Path]) - - // Second call - pkg2, err := cachedResolver.Resolve(token.NewFileSet(), pkg.Path) - require.NoError(t, err) - require.Same(t, pkg1, pkg2) - require.Equal(t, 1, mockResolver.resolveCalls[pkg.Path]) - }) - - t.Run("no cache when shouldCache is false", func(t *testing.T) { - t.Parallel() - - mockResolver := NewMockResolver(pkg) - cacheMiddleware := CacheMiddleware(func(*Package) bool { return false }) - cachedResolver := MiddlewareResolver(mockResolver, cacheMiddleware) - - pkg1, err := cachedResolver.Resolve(token.NewFileSet(), pkg.Path) - require.NoError(t, err) - pkg2, err := cachedResolver.Resolve(token.NewFileSet(), pkg.Path) - require.NoError(t, err) - require.NotSame(t, pkg1, pkg2) - require.Equal(t, 2, mockResolver.resolveCalls[pkg.Path]) - }) -} - -func TestFilterStdlibsMiddleware(t *testing.T) { - t.Parallel() - - middleware := FilterStdlibs - mockResolver := NewMockResolver(&std.MemPackage{ - Path: "abc.xy/t/pkg", - Name: "pkg", - Files: []*std.MemFile{ - {Name: "file.gno", Body: "package pkg"}, - }, - }) - filteredResolver := MiddlewareResolver(mockResolver, middleware) - - t.Run("filters stdlib paths", func(t *testing.T) { - t.Parallel() - - _, err := filteredResolver.Resolve(token.NewFileSet(), "fmt") - require.Error(t, err) - require.True(t, errors.Is(err, ErrResolverPackageSkip)) - require.Equal(t, 0, mockResolver.resolveCalls["fmt"]) - }) - - t.Run("allows non-stdlib paths", func(t *testing.T) { - t.Parallel() - - pkg, err := filteredResolver.Resolve(token.NewFileSet(), "abc.xy/t/pkg") - require.NoError(t, err) - require.NotNil(t, pkg) - require.Equal(t, 1, mockResolver.resolveCalls["abc.xy/t/pkg"]) - }) -} - -func TestPackageCheckerMiddleware(t *testing.T) { - t.Parallel() - - logger := log.NewTestingLogger(t) - t.Run("valid package syntax", func(t *testing.T) { - t.Parallel() - - validPkg := &std.MemPackage{ - Path: "abc.xy/r/valid/pkg", - Name: "valid", - Files: []*std.MemFile{ - {Name: "valid.gno", Body: "package valid; func Foo() {}"}, - }, - } - mockResolver := NewMockResolver(validPkg) - middleware := PackageCheckerMiddleware(logger) - resolver := MiddlewareResolver(mockResolver, middleware) - - pkg, err := resolver.Resolve(token.NewFileSet(), validPkg.Path) - require.NoError(t, err) - require.NotNil(t, pkg) - }) - - t.Run("invalid package syntax", func(t *testing.T) { - t.Parallel() - - invalidPkg := &std.MemPackage{ - Path: "abc.xy/r/invalid/pkg", - Name: "invalid", - Files: []*std.MemFile{ - {Name: "invalid.gno", Body: "package invalid\nfunc Foo() {"}, - }, - } - mockResolver := NewMockResolver(invalidPkg) - middleware := PackageCheckerMiddleware(logger) - resolver := MiddlewareResolver(mockResolver, middleware) - - _, err := resolver.Resolve(token.NewFileSet(), invalidPkg.Path) - require.Error(t, err) - require.Contains(t, err.Error(), `file "invalid.gno" have error(s)`) - }) - - t.Run("ignores non-gno files", func(t *testing.T) { - t.Parallel() - - nonGnoPkg := &std.MemPackage{ - Path: "abc.xy/r/non/gno/pkg", - Name: "pkg", - Files: []*std.MemFile{ - {Name: "README.md", Body: "# Documentation"}, - }, - } - mockResolver := NewMockResolver(nonGnoPkg) - middleware := PackageCheckerMiddleware(logger) - resolver := MiddlewareResolver(mockResolver, middleware) - - _, err := resolver.Resolve(token.NewFileSet(), nonGnoPkg.Path) - require.NoError(t, err) - }) -} - -func TestResolverLocal_Resolve(t *testing.T) { - t.Parallel() - - const anotherPath = "abc.xy/t/another/path" - - localResolver := NewLocalResolver(anotherPath, filepath.Join("./testdata", TestdataPkgA)) - - t.Run("valid package", func(t *testing.T) { - t.Parallel() - - pkg, err := localResolver.Resolve(token.NewFileSet(), anotherPath) - require.NoError(t, err) - require.NotNil(t, pkg) - require.Equal(t, pkg.Name, "aa") - }) - - t.Run("invalid package", func(t *testing.T) { - t.Parallel() - - pkg, err := localResolver.Resolve(token.NewFileSet(), "abc.xy/t/wrong/package") - require.Nil(t, pkg) - require.Error(t, err) - require.ErrorIs(t, err, ErrResolverPackageNotFound) - }) -} - -func TestResolver_ResolveRemote(t *testing.T) { - const targetPath = "gno.land/r/target/path" - - mempkg := std.MemPackage{ - Name: "foo", - Path: targetPath, - Files: []*std.MemFile{ - { - Name: "foo.gno", - Body: `package foo; func Render(_ string) string { return "bar" }`, - }, - }, - } - mempkg.SetFile("gnomod.toml", gnolang.GenGnoModLatest(mempkg.Path)) - mempkg.Sort() - - rootdir := gnoenv.RootDir() - cfg := integration.TestingMinimalNodeConfig(rootdir) - logger := log.NewTestingLogger(t) - - // Setup genesis state - privKey := secp256k1.GenPrivKey() - cfg.Genesis.AppState = integration.GenerateTestingGenesisState(privKey, mempkg) - - _, address := integration.TestingInMemoryNode(t, logger, cfg) - cl, err := client.NewHTTPClient(address) - require.NoError(t, err) - - remoteResolver := NewRemoteResolver(address, cl) - t.Run("valid package", func(t *testing.T) { - pkg, err := remoteResolver.Resolve(token.NewFileSet(), mempkg.Path) - require.NoError(t, err) - require.NotNil(t, pkg) - assert.Equal(t, mempkg, pkg.MemPackage) - }) - - t.Run("invalid package", func(t *testing.T) { - pkg, err := remoteResolver.Resolve(token.NewFileSet(), "gno.land/r/not/a/valid/package") - require.Nil(t, pkg) - require.Error(t, err) - require.ErrorIs(t, err, ErrResolverPackageNotFound) - }) -} - -func TestResolverRoot_Resolve(t *testing.T) { - t.Parallel() - - fsResolver := NewRootResolver("./testdata") - t.Run("valid packages", func(t *testing.T) { - t.Parallel() - - for _, tpkg := range []string{TestdataPkgA, TestdataPkgB, TestdataPkgC} { - t.Run(tpkg, func(t *testing.T) { - t.Logf("resolving %q", tpkg) - pkg, err := fsResolver.Resolve(token.NewFileSet(), tpkg) - require.NoError(t, err) - require.NotNil(t, pkg) - require.Equal(t, tpkg, pkg.Path) - require.Equal(t, filepath.Base(tpkg), pkg.Name) - }) - } - }) - - t.Run("invalid packages", func(t *testing.T) { - t.Parallel() - - pkg, err := fsResolver.Resolve(token.NewFileSet(), "abc.xy/wrong/package") - require.Nil(t, pkg) - require.Error(t, err) - require.ErrorIs(t, err, ErrResolverPackageNotFound) - }) -} diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/aa/file.gno b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/aa/file.gno deleted file mode 100644 index b809785a376..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/aa/file.gno +++ /dev/null @@ -1,3 +0,0 @@ -package aa - -type SA struct{} diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/aa/gnomod.toml b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/aa/gnomod.toml deleted file mode 100644 index c7e4c9a7490..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/aa/gnomod.toml +++ /dev/null @@ -1,2 +0,0 @@ -module = "abc.xy/t/aa" -gno = "0.9" \ No newline at end of file diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/bb/file.gno b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/bb/file.gno deleted file mode 100644 index ba9240d55f9..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/bb/file.gno +++ /dev/null @@ -1,5 +0,0 @@ -package bb - -import "abc.xy/t/aa" - -type SB = aa.SA diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/bb/gnomod.toml b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/bb/gnomod.toml deleted file mode 100644 index 9d83e9468ab..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/bb/gnomod.toml +++ /dev/null @@ -1,2 +0,0 @@ -module = "abc.xy/t/bb" -gno = "0.9" \ No newline at end of file diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/cc/file.gno b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/cc/file.gno deleted file mode 100644 index 4bdf2bbfd8c..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/cc/file.gno +++ /dev/null @@ -1,5 +0,0 @@ -package cc - -import "abc.xy/t/bb" - -type SC = bb.SB diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/cc/gnomod.toml b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/cc/gnomod.toml deleted file mode 100644 index 15f0ed97751..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/cc/gnomod.toml +++ /dev/null @@ -1,2 +0,0 @@ -module = "abc.xy/t/cc" -gno = "0.9" \ No newline at end of file diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/aa/file.gno b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/aa/file.gno deleted file mode 100644 index 14492ef76f3..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/aa/file.gno +++ /dev/null @@ -1 +0,0 @@ -package aa diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/aa/gnomod.toml b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/aa/gnomod.toml deleted file mode 100644 index 4ffbda9c342..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/aa/gnomod.toml +++ /dev/null @@ -1,2 +0,0 @@ -module = "abc.xy/t/nested/aa" -gno = "0.9" \ No newline at end of file diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/bb/file.gno b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/bb/file.gno deleted file mode 100644 index 592f1946da0..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/bb/file.gno +++ /dev/null @@ -1 +0,0 @@ -package bb diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/bb/gnomod.toml b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/bb/gnomod.toml deleted file mode 100644 index ec60b29b7a8..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/bb/gnomod.toml +++ /dev/null @@ -1,2 +0,0 @@ -module = "abc.xy/t/nested/nested/bb" -gno = "0.9" \ No newline at end of file diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/cc/file.gno b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/cc/file.gno deleted file mode 100644 index 10702f6990c..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/cc/file.gno +++ /dev/null @@ -1 +0,0 @@ -package cc diff --git a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/cc/gnomod.toml b/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/cc/gnomod.toml deleted file mode 100644 index 4541d5783da..00000000000 --- a/contribs/gnodev/pkg/packages/testdata/abc.xy/t/nested/nested/cc/gnomod.toml +++ /dev/null @@ -1,2 +0,0 @@ -module = "abc.xy/t/nested/nested/cc" -gno = "0.9" \ No newline at end of file diff --git a/contribs/gnodev/pkg/packages/testdata_test.go b/contribs/gnodev/pkg/packages/testdata_test.go deleted file mode 100644 index 083bb6f7db9..00000000000 --- a/contribs/gnodev/pkg/packages/testdata_test.go +++ /dev/null @@ -1,47 +0,0 @@ -// This test file serves as a reference for the testdata directory tree. - -package packages - -// The structure of the testdata directory is as follows: -// -// testdata -// └── abc.xy -// └── t -// ├── aa -// │   ├── file.gno -// │   └── gnomod.toml -// ├── bb -// │   ├── file.gno -// │   └── gnomod.toml -// ├── cc -// │   ├── file.gno -// │   └── gnomod.toml -// └── nested -// ├── aa -// │   ├── file.gno -// │   └── gnomod.toml -// └── nested -// ├── bb -// │   ├── file.gno -// │   └── gnomod.toml -// └── cc -// ├── file.gno -// └── gnomod.toml - -const ( - TestdataPkgA = "abc.xy/t/aa" - TestdataPkgB = "abc.xy/t/bb" - TestdataPkgC = "abc.xy/t/cc" -) - -// List of testdata package paths -var testdataPkgs = []string{TestdataPkgA, TestdataPkgB, TestdataPkgC} - -const ( - TestdataNestedA = "abc.xy/t/nested/aa" // Path to nested package A - TestdataNestedB = "abc.xy/t/nested/nested/bb" // Path to nested package B - TestdataNestedC = "abc.xy/t/nested/nested/cc" // Path to nested package C -) - -// List of nested package paths -var testdataNested = []string{TestdataNestedA, TestdataNestedB, TestdataNestedC} diff --git a/contribs/gnodev/pkg/packages/utils.go b/contribs/gnodev/pkg/packages/utils.go deleted file mode 100644 index 6b4a4df7217..00000000000 --- a/contribs/gnodev/pkg/packages/utils.go +++ /dev/null @@ -1,10 +0,0 @@ -package packages - -import ( - "path/filepath" - "strings" -) - -func isGnoFile(name string) bool { - return filepath.Ext(name) == ".gno" && !strings.HasPrefix(name, ".") -} diff --git a/contribs/gnodev/pkg/packages/utils_other.go b/contribs/gnodev/pkg/packages/utils_other.go deleted file mode 100644 index 0455aa57480..00000000000 --- a/contribs/gnodev/pkg/packages/utils_other.go +++ /dev/null @@ -1,9 +0,0 @@ -//go:build !windows -// +build !windows - -package packages - -// NormalizeFilepathToPath normalize path an unix like path -func NormalizeFilepathToPath(path string) string { - return path -} diff --git a/contribs/gnodev/pkg/packages/utils_windows.go b/contribs/gnodev/pkg/packages/utils_windows.go deleted file mode 100644 index 23c7fa54379..00000000000 --- a/contribs/gnodev/pkg/packages/utils_windows.go +++ /dev/null @@ -1,11 +0,0 @@ -//go:build windows -// +build windows - -package packages - -import "strings" - -// NormalizeFilepathToPath normalize path an unix like path -func NormalizeFilepathToPath(path string) string { - return strings.ReplaceAll(path, "\\", "/") -} diff --git a/contribs/gnodev/pkg/proxy/path_interceptor.go b/contribs/gnodev/pkg/proxy/path_interceptor.go index 8109753c0d9..84d2f92b22f 100644 --- a/contribs/gnodev/pkg/proxy/path_interceptor.go +++ b/contribs/gnodev/pkg/proxy/path_interceptor.go @@ -6,14 +6,11 @@ import ( "encoding/json" "errors" "fmt" - "go/parser" - "go/token" "io" "log/slog" "net" "net/http" - gopath "path" - "strconv" + "path/filepath" "strings" "sync" @@ -223,30 +220,7 @@ func (upaths uniqPaths) list() []string { return paths } -// Add a path to -func (upaths uniqPaths) addPath(path string) { - path = cleanupPath(path) - upaths[path] = struct{}{} -} - -func (upaths uniqPaths) addPackageDeps(pkg *std.MemPackage) { - fset := token.NewFileSet() - for _, file := range pkg.Files { - if !strings.HasSuffix(file.Name, ".gno") { - continue - } - - f, err := parser.ParseFile(fset, file.Name, file.Body, parser.ImportsOnly) - if err != nil { - continue - } - - for _, imp := range f.Imports { - path, _ := strconv.Unquote(imp.Path.Value) - upaths.addPath(path) - } - } -} +func (upaths uniqPaths) add(path string) { upaths[path] = struct{}{} } // handleRequest parses and processes the RPC request body. func (proxy *PathInterceptor) handleRequest(body []byte) error { @@ -319,18 +293,11 @@ func handleTx(bz []byte, upaths uniqPaths) error { for _, msg := range tx.Msgs { switch msg := msg.(type) { - case vm.MsgAddPackage: - // NOTE: Do not add the package itself to avoid conflict. - if msg.Package != nil { - upaths.addPackageDeps(msg.Package) - } - case vm.MsgRun: - // NOTE: Do not add the package itself to avoid conflict. - if msg.Package != nil { - upaths.addPackageDeps(msg.Package) - } + case vm.MsgAddPackage: // MsgAddPackage should not be handled case vm.MsgCall: - upaths.addPath(msg.PkgPath) + upaths.add(msg.PkgPath) + case vm.MsgRun: + upaths.add(msg.Package.Path) } } @@ -345,7 +312,14 @@ func handleQuery(path string, data []byte, upaths uniqPaths) error { case "vm/qrender", "vm/qfile", "vm/qfuncs", "vm/qeval": path, _, _ := strings.Cut(string(data), ":") // Cut arguments out - upaths.addPath(path) + path = filepath.Clean(path) + + // If path is a file, grab the directory instead + if ext := filepath.Ext(path); ext != "" { + path = filepath.Dir(path) + } + + upaths.add(path) return nil default: @@ -354,13 +328,3 @@ func handleQuery(path string, data []byte, upaths uniqPaths) error { // XXX: handle more cases } - -func cleanupPath(path string) string { - path = gopath.Clean(path) - // If path is a file, grab the directory instead - if ext := gopath.Ext(path); ext != "" { - path = gopath.Dir(path) - } - - return path -} diff --git a/contribs/gnodev/pkg/proxy/path_interceptor_test.go b/contribs/gnodev/pkg/proxy/path_interceptor_test.go index 7f10a62f5e3..27a7772038a 100644 --- a/contribs/gnodev/pkg/proxy/path_interceptor_test.go +++ b/contribs/gnodev/pkg/proxy/path_interceptor_test.go @@ -7,7 +7,6 @@ import ( "path" "path/filepath" "testing" - "time" "github.com/gnolang/gno/contribs/gnodev/pkg/proxy" "github.com/gnolang/gno/gno.land/pkg/gnoland/ugnot" @@ -200,12 +199,11 @@ func Render(_ string) string { return foo.Render("bar") }`, t.Logf("logs: %s", res.DeliverTx.Log) } + // MsgAddPackage should NOT trigger path handler select { case paths := <-pathChan: - require.Len(t, paths, 1) - assert.Equal(t, []string{targetPath}, paths) - case <-time.After(time.Second): - t.Fatal("paths not captured") + t.Fatalf("should not catch paths for MsgAddPackage, got: %+v", paths) + default: } }) diff --git a/contribs/gnodev/pkg/watcher/watch.go b/contribs/gnodev/pkg/watcher/watch.go index 0a7d284bd37..488401589fd 100644 --- a/contribs/gnodev/pkg/watcher/watch.go +++ b/contribs/gnodev/pkg/watcher/watch.go @@ -111,7 +111,7 @@ func (p *PackageWatcher) Stop() { p.stop() } -func (p *PackageWatcher) UpdatePackagesWatch(pkgs ...packages.Package) { +func (p *PackageWatcher) UpdatePackagesWatch(pkgs ...*packages.Package) { watchList := p.watcher.WatchList() oldPkgs := make(map[string]struct{}, len(watchList)) @@ -125,9 +125,13 @@ func (p *PackageWatcher) UpdatePackagesWatch(pkgs ...packages.Package) { continue } - dir, err := filepath.Abs(pkg.Location) + if pkg.Dir == "" { + continue + } + + dir, err := filepath.Abs(pkg.Dir) if err != nil { - p.logger.Error("Unable to get absolute path", "path", pkg.Location, "error", err) + p.logger.Error("Unable to get absolute path", "path", pkg.Dir, "error", err) continue } diff --git a/contribs/gnodev/setup_loader.go b/contribs/gnodev/setup_loader.go index f8540be59e0..e9ce1bc99d7 100644 --- a/contribs/gnodev/setup_loader.go +++ b/contribs/gnodev/setup_loader.go @@ -1,90 +1,87 @@ package main import ( - "fmt" "log/slog" - gopath "path" + "os" "path/filepath" - "regexp" "strings" "github.com/gnolang/gno/contribs/gnodev/pkg/packages" "github.com/gnolang/gno/gnovm/pkg/gnomod" - "github.com/gnolang/gno/tm2/pkg/bft/rpc/client" ) -type varResolver []packages.Resolver +func setupPackagesLoader(logger *slog.Logger, cfg *AppConfig, dirs ...string) (packages.Loader, []string) { + // Add extra workspaces (e.g., examples directory and user-provided directories) + examplesDir := filepath.Join(cfg.root, "examples") + workspaces := append([]string{examplesDir}, dirs...) -func (va varResolver) String() string { - resolvers := packages.ChainedResolver(va) - return resolvers.Name() -} - -func (va *varResolver) Set(value string) error { - name, location, found := strings.Cut(value, "=") - if !found { - return fmt.Errorf("invalid resolver format %q, should be `=`", value) + // Warn about deprecated resolver flag + if len(cfg.resolvers) > 0 { + logger.Warn("the -resolver flag is deprecated and ignored; packages are now discovered via gnomod.toml and gnowork.toml") } - var res packages.Resolver - switch name { - case "remote": - rpc, err := client.NewHTTPClient(location) - if err != nil { - return fmt.Errorf("invalid resolver remote: %q", location) - } + loader := packages.NewNativeLoader(packages.NativeLoaderConfig{ + Logger: logger, + GnoRoot: cfg.root, + ExtraWorkspaces: workspaces, + }) - res = packages.NewRemoteResolver(location, rpc) - case "root": // process everything from a root directory - res = packages.NewRootResolver(location) - case "local": // process a single directory - path, ok := guessPathGnoMod(location) - if !ok { - return fmt.Errorf("unable to read module path from gnomod.toml in %q", location) - } - - res = packages.NewLocalResolver(path, location) - default: - return fmt.Errorf("invalid resolver name: %q", name) + // Pre-populate the index for lazy loading support + // This scans workspace roots and maps import paths to filesystem directories + if err := loader.DiscoverPackages(); err != nil { + logger.Warn("failed to discover packages", "err", err) } - *va = append(*va, res) - return nil -} - -func setupPackagesResolver(logger *slog.Logger, cfg *AppConfig, dirs ...string) (packages.Resolver, []string) { - // Add root resolvers - localResolvers := make([]packages.Resolver, len(dirs)) - + // Determine paths to pre-load based on load mode var paths []string - for i, dir := range dirs { - path := guessPath(cfg, dir) - resolver := packages.NewLocalResolver(path, dir) - - if resolver.IsValid() { - logger.Info("guessing directory path", "path", path, "dir", dir) - paths = append(paths, path) // append local path - } else { - logger.Warn("no gno package found", "dir", dir) + switch cfg.loadMode { + case LoadModeAuto: + // If in examples folder, use lazy mode (no pre-load) + if isInExamplesDir(cfg.root, dirs) { + logger.Info("running from examples folder, using lazy loading") + break } - localResolvers[i] = resolver + examplesDir := filepath.Join(cfg.root, "examples") + + for _, dir := range dirs { + absDir, err := filepath.Abs(dir) + if err != nil { + continue + } + + if isWorkspaceDir(dir) { + // Workspace detected: pre-load ALL packages within this workspace + // by filtering the discovered index by directory prefix + logger.Info("workspace detected, will pre-load all packages", "dir", dir) + for _, pkg := range loader.GetIndex().List() { + // Skip examples packages + if strings.HasPrefix(pkg.Dir, examplesDir) { + continue + } + // Only include packages under this workspace + if strings.HasPrefix(pkg.Dir, absDir) { + logger.Debug("workspace package detected", "path", pkg.ImportPath) + paths = append(paths, pkg.ImportPath) + } + } + } else if pkgPath, ok := guessPathGnoMod(dir); ok { + // Single package detected + logger.Info("package detected, will be pre-loaded", "path", pkgPath, "dir", dir) + paths = append(paths, pkgPath) + } + } + case LoadModeLazy: + logger.Info("lazy mode: packages will be loaded on-demand") + case LoadModeFull: + // Pre-load all discovered packages + for _, pkg := range loader.GetIndex().List() { + paths = append(paths, pkg.ImportPath) + } + logger.Info("full mode: pre-loading all discovered packages", "count", len(paths)) } - resolver := packages.ChainResolvers( - packages.ChainResolvers(localResolvers...), // Resolve local directories - packages.ChainResolvers(cfg.resolvers...), // Use user's custom resolvers - ) - - // Enrich resolver with middleware - return packages.MiddlewareResolver(resolver, - packages.CacheMiddleware(func(pkg *packages.Package) bool { - return pkg.Kind == packages.PackageKindRemote // Only cache remote package - }), - packages.FilterStdlibs, // Filter stdlib package from resolving - packages.PackageCheckerMiddleware(logger), // Pre-check syntax to avoid bothering the node reloading on invalid files - packages.LogMiddleware(logger), // Log request - ), paths + return loader, paths } func guessPathGnoMod(dir string) (path string, ok bool) { @@ -95,13 +92,22 @@ func guessPathGnoMod(dir string) (path string, ok bool) { return modfile.Module, true } -var reInvalidChar = regexp.MustCompile(`[^\w_-]`) +func isWorkspaceDir(dir string) bool { + workFile := filepath.Join(dir, "gnowork.toml") + _, err := os.Stat(workFile) + return err == nil +} -func guessPath(cfg *AppConfig, dir string) (path string) { - if path, ok := guessPathGnoMod(dir); ok { - return path +func isInExamplesDir(gnoRoot string, dirs []string) bool { + examplesDir := filepath.Join(gnoRoot, "examples") + for _, dir := range dirs { + absDir, err := filepath.Abs(dir) + if err != nil { + continue + } + if strings.HasPrefix(absDir, examplesDir) { + return true + } } - - rname := reInvalidChar.ReplaceAllString(filepath.Base(dir), "-") - return gopath.Join(cfg.chainDomain, "/r/dev/", rname) + return false } diff --git a/gnovm/pkg/packages/load.go b/gnovm/pkg/packages/load.go index e229689c0bf..a67f87089a1 100644 --- a/gnovm/pkg/packages/load.go +++ b/gnovm/pkg/packages/load.go @@ -61,7 +61,11 @@ func Load(conf LoadConfig, patterns ...string) (PkgList, error) { panic(fmt.Errorf("context root should be absolute at this point, got %q", loaderCtx.Root)) } - expanded, err := expandPatterns(conf.GnoRoot, loaderCtx, conf.Out, patterns...) + // Discover local packages in workspace roots BEFORE expanding patterns + // This allows remote-style patterns like "gno.land/r/foo" to resolve to local dirs + localDeps := discoverPkgsForLocalDeps(conf, loaderCtx) + + expanded, err := expandPatterns(conf.GnoRoot, loaderCtx, localDeps, conf.Out, patterns...) if err != nil { return nil, err } @@ -81,8 +85,6 @@ func Load(conf LoadConfig, patterns ...string) (PkgList, error) { // load deps - localDeps := discoverPkgsForLocalDeps(conf, loaderCtx) - // mark all pattern packages for visit toVisit := []*Package(pkgs) @@ -105,9 +107,9 @@ func Load(conf LoadConfig, patterns ...string) (PkgList, error) { continue } - // load tests deps if test flag is set and the package is not a dep + // load tests deps if test flag is set importKinds := []FileKind{FileKindPackageSource} - if conf.Test && len(pkg.Match) != 0 { + if conf.Test { importKinds = append(importKinds, FileKindTest, FileKindXTest, FileKindFiletest) } @@ -149,6 +151,11 @@ func Load(conf LoadConfig, patterns ...string) (PkgList, error) { markDepForVisit(loadSinglePkg(conf.Out, conf.Fetcher, dir, conf.Fset)) } + // Skip stdlib packages - they're handled natively by the VM + // and should not be deployed as user packages + if gnolang.IsStdlib(pkg.ImportPath) { + continue + } loaded = append(loaded, pkg) } diff --git a/gnovm/pkg/packages/load_test.go b/gnovm/pkg/packages/load_test.go index 1d994210825..6bbec870cf6 100644 --- a/gnovm/pkg/packages/load_test.go +++ b/gnovm/pkg/packages/load_test.go @@ -137,32 +137,38 @@ func TestSortPkgs(t *testing.T) { }, { desc: "no_dependencies", in: []*Package{ - {ImportPath: "pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{}}, - {ImportPath: "pkg2", Dir: "/path/to/pkg2", Imports: map[FileKind][]string{}}, - {ImportPath: "pkg3", Dir: "/path/to/pkg3", Imports: map[FileKind][]string{}}, + {ImportPath: "test.land/r/pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{}}, + {ImportPath: "test.land/r/pkg2", Dir: "/path/to/pkg2", Imports: map[FileKind][]string{}}, + {ImportPath: "test.land/r/pkg3", Dir: "/path/to/pkg3", Imports: map[FileKind][]string{}}, }, - expected: []string{"pkg1", "pkg2", "pkg3"}, + expected: []string{"test.land/r/pkg1", "test.land/r/pkg2", "test.land/r/pkg3"}, }, { desc: "circular_dependencies", in: []*Package{ - {ImportPath: "pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{FileKindPackageSource: {"pkg2"}}}, - {ImportPath: "pkg2", Dir: "/path/to/pkg2", Imports: map[FileKind][]string{FileKindPackageSource: {"pkg1"}}}, + {ImportPath: "test.land/r/pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{FileKindPackageSource: {"test.land/r/pkg2"}}}, + {ImportPath: "test.land/r/pkg2", Dir: "/path/to/pkg2", Imports: map[FileKind][]string{FileKindPackageSource: {"test.land/r/pkg1"}}}, }, shouldErr: true, }, { desc: "missing_dependencies", in: []*Package{ - {ImportPath: "pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{FileKindPackageSource: {"pkg2"}}}, + {ImportPath: "test.land/r/pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{FileKindPackageSource: {"test.land/r/pkg2"}}}, }, shouldErr: true, }, { desc: "valid_dependencies", in: []*Package{ - {ImportPath: "pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{FileKindPackageSource: {"pkg2"}}}, - {ImportPath: "pkg2", Dir: "/path/to/pkg2", Imports: map[FileKind][]string{FileKindPackageSource: {"pkg3"}}}, - {ImportPath: "pkg3", Dir: "/path/to/pkg3", Imports: map[FileKind][]string{}}, + {ImportPath: "test.land/r/pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{FileKindPackageSource: {"test.land/r/pkg2"}}}, + {ImportPath: "test.land/r/pkg2", Dir: "/path/to/pkg2", Imports: map[FileKind][]string{FileKindPackageSource: {"test.land/r/pkg3"}}}, + {ImportPath: "test.land/r/pkg3", Dir: "/path/to/pkg3", Imports: map[FileKind][]string{}}, }, - expected: []string{"pkg3", "pkg2", "pkg1"}, + expected: []string{"test.land/r/pkg3", "test.land/r/pkg2", "test.land/r/pkg1"}, + }, { + desc: "stdlib_imports_skipped", + in: []*Package{ + {ImportPath: "test.land/r/pkg1", Dir: "/path/to/pkg1", Imports: map[FileKind][]string{FileKindPackageSource: {"std", "strings"}}}, + }, + expected: []string{"test.land/r/pkg1"}, }, } { t.Run(tc.desc, func(t *testing.T) { diff --git a/gnovm/pkg/packages/patterns.go b/gnovm/pkg/packages/patterns.go index 90ecfd2a354..22333df6d3a 100644 --- a/gnovm/pkg/packages/patterns.go +++ b/gnovm/pkg/packages/patterns.go @@ -20,7 +20,7 @@ type pkgMatch struct { Match []string } -func expandPatterns(gnoRoot string, loaderCtx *loaderContext, out io.Writer, patterns ...string) ([]*pkgMatch, error) { +func expandPatterns(gnoRoot string, loaderCtx *loaderContext, localDeps map[string]string, out io.Writer, patterns ...string) ([]*pkgMatch, error) { pkgMatches := []*pkgMatch(nil) addPkgDir := func(dir string, match *string) { @@ -118,7 +118,11 @@ func expandPatterns(gnoRoot string, loaderCtx *loaderContext, out io.Writer, pat var dir string if gnolang.IsStdlib(pat) { dir = StdlibDir(gnoRoot, pat) + } else if localDir, ok := localDeps[pat]; ok { + // Found in local workspace roots + dir = localDir } else { + // Fall back to mod cache dir = PackageDir(pat) } addPkgDir(dir, &match) diff --git a/gnovm/pkg/packages/pkglist.go b/gnovm/pkg/packages/pkglist.go index a3ca2e90bea..5c2af98a5e7 100644 --- a/gnovm/pkg/packages/pkglist.go +++ b/gnovm/pkg/packages/pkglist.go @@ -3,6 +3,8 @@ package packages import ( "errors" "fmt" + + "github.com/gnolang/gno/gnovm/pkg/gnolang" ) type ( @@ -58,21 +60,36 @@ func visitPackage(pkg *Package, pkgs []*Package, visited, onStack map[string]boo visited[pkg.ImportPath] = true onStack[pkg.ImportPath] = true - // Visit package's dependencies - for _, imp := range pkg.Imports[FileKindPackageSource] { - found := false - for _, p := range pkgs { - if p.ImportPath != imp { + // Visit package's dependencies (including test dependencies) + // We need to consider all import kinds to ensure proper ordering for deployment + allImportKinds := []FileKind{FileKindPackageSource, FileKindTest, FileKindXTest, FileKindFiletest} + for _, kind := range allImportKinds { + for _, imp := range pkg.Imports[kind] { + // Skip self-imports (XTest files import their own package for blackbox testing) + if imp == pkg.ImportPath { continue } - if err := visitPackage(p, pkgs, visited, onStack, sortedPkgs); err != nil { - return err + + // Skip stdlib imports - they're handled by the VM natively + // and are not included in the package list + if gnolang.IsStdlib(imp) { + continue + } + + found := false + for _, p := range pkgs { + if p.ImportPath != imp { + continue + } + if err := visitPackage(p, pkgs, visited, onStack, sortedPkgs); err != nil { + return err + } + found = true + break + } + if !found { + return fmt.Errorf("missing dependency '%s' for package '%s'", imp, pkg.ImportPath) } - found = true - break - } - if !found { - return fmt.Errorf("missing dependency '%s' for package '%s'", imp, pkg.ImportPath) } }