Skip to content

feat(gnodev): native package loader #4957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
gfanton wants to merge 10 commits into
base: master
Choose a base branch
from
Draft

feat(gnodev): native package loader #4957

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
9e6a845
wip: iter 1
gfanton Dec 8, 2025
c6b02f5
wip: iter 2
gfanton Dec 8, 2025
90739cb
wip: iter 3
gfanton Dec 9, 2025
064c37f
fix: path interceptor
gfanton Dec 9, 2025
1772cba
chore: update help
gfanton Dec 9, 2025
a5c4625
feat: add load mode
gfanton Dec 9, 2025
a8d6fea
fix: default path in auto
gfanton Dec 10, 2025
6cac9a7
fix: path loading
gfanton Dec 10, 2025
8c6112c
feat: update loader
gfanton Dec 10, 2025
2349eb8
chore: lint & fixup
gfanton Dec 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
179 changes: 75 additions & 104 deletions contribs/gnodev/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,12 +11,13 @@ Please note that this is a quick overview. For a more detailed guide, refer to t
**gnodev** [**options**] [**PKG_PATH ...**]

## Features
- **In-Memory Node**: Gnodev starts an in-memory node, automatically loading the **examples** folder and any
user-specified paths.
- **In-Memory Node**: Gnodev starts an in-memory node with automatic package discovery.
- **Package Discovery**: Automatically detects packages via `gnomod.toml` files and workspaces via `gnowork.toml`.
- **Flexible Loading Modes**: Three loading modes (`auto`, `lazy`, `full`) to balance startup time and convenience.
- **Web Interface Server**: Gnodev starts a `gnoweb` server on [`localhost:8888`](https://localhost:8888).
- **Balances and Keybase Customization**: Set account balances, load them from a file, or add new accounts via a flag.
- **Hot Reload**: Monitors the **examples** folder and specified directories for file changes, reloading the
package and automatically restarting the node as needed.
- **Hot Reload**: Monitors package directories for file changes, reloading the package and automatically
restarting the node as needed.
- **State Maintenance**: Ensures the previous node state is preserved by replaying all transactions.
- **Transaction Manipulation**: Allows for interactive cancellation and redoing of transactions.
- **State Export**: Export the current state at any time in a genesis doc format.
Expand All @@ -35,108 +36,86 @@ While `gnodev` is running, trigger specific actions by pressing the following co
- **Cmd+C**: Exit `gnodev`.

## Usage
Run `gnodev` followed by any specific options and/or package paths. The **examples** directory is loaded
automatically. Use `--minimal` to prevent this.
Run `gnodev` from a directory containing a `gnomod.toml` file, and the package will be automatically detected
and loaded. You can also pass package directories as arguments.

### Load Modes
Use the `-load` flag to control how packages are loaded:
- **auto** (default for local): Pre-loads current workspace/package only
- **lazy**: Loads all packages on-demand as they are accessed
- **full** (default for staging): Pre-loads all discovered packages

Example:
```
gnodev --add-account <bech32/name1>[:<amount1>] ./myrealm
gnodev # Auto-detect and pre-load current package
gnodev -load=lazy # Load packages on-demand only
gnodev -load=full # Pre-load all packages
```

### `gnodev -h`
[embedmd]:# (.tmp/gnodev-usage.txt)
```txt
USAGE
gnodev <cmd> [flags]
gnodev <cmd> [flags]

The gnodev command starts an in-memory node and a gno.land
web interface, primarily for realm package development.

Currently gnodev comes with two mode <local> and <staging>, those command mostly
differ by there default values, while gnodev local as default for working
locally, satging default are oriented to be use on server.

gnodev uses its own package loader and resolver system to support multiple
scenarios and use cases. It currently supports three types of resolvers, each
taking a location as an argument.
- root: This resolver takes a <dir> as its location. It attempts to resolve
packages based on your file system structure and the package path.
For example, if 'root=/user/gnome/myproject' and you try to resolve
'gno.land/r/bar/buzz' as a package, the <root> resolver will attempt to
resolve it to /user/gnome/myproject/gno.land/r/bar/buzz.
- local: This resolver also takes a <dir> as its location. It is designed to
load a single package, using the module name from 'gnomod.toml' within this
package to resolve the package.
- remote: This resolver takes a <remote> RPC address as its location. It is
meant to use a remote node as a resolver, primarily for testing a local
package against a remote node.

Resolvers can be chained, and gnodev will attempt to use them in the order they
are declared.

For example:
gnodev -resolver root=/user/gnome/myproject -resolver remote=https://rpc.gno.lands

If no resolvers can resolve a given package path, the loader will return a
"package not found" error.
gnodev comes with two modes: <local> and <staging>. These commands
differ mainly by their default values - local mode is optimized for
development, while staging mode is oriented for server usage.

If no command is provided, gnodev will automatically start in <local> mode.
Package discovery is automatic via gnomod.toml and gnowork.toml files.
Use the -load flag to control loading behavior:
- auto: Pre-load current workspace/package (default for local)
- lazy: Load packages on-demand as accessed
- full: Pre-load all discovered packages (default for staging)

For more information and flags usage description, use 'gnodev local -h'.
If no command is provided, gnodev will automatically start in <local> mode.

SUBCOMMANDS
local Start gnodev in local development mode (default)
staging Start gnodev in staging mode

```

### `gnodev local -h`
[embedmd]:# (.tmp/gnodev-local-usage.txt)
```txt
USAGE
gnodev local [flags] [package_dir...]

LOCAL: Local mode configures the node for local development usage.
This mode is optimized for realm development, providing an interactive and flexible environment.
It enables features such as interactive mode, unsafe API access for testing, and lazy loading to improve performance.
The log format is set to console for easier readability, and the web interface is accessible locally, making it ideal for iterative development and testing.

By default, the current directory and the "example" folder from "gnoroot" will be used as the root resolver.
It enables features such as interactive mode, unsafe API access for testing, and auto loading mode.
The log format is set to console for easier readability, and the web interface is accessible locally.

Package discovery is automatic via gnomod.toml and gnowork.toml files.

FLAGS
-C ... change directory context before running gnodev
-add-account ... add (or set) a premine account in the form `<bech32|name>[=<amount>]`, can be used multiple time
-balance-file ... load the provided balance file (refer to the documentation for format)
-chain-domain gno.land set node ChainDomain
-chain-id dev set node ChainID
-deploy-key g1jg8mtutu9khhfwc4nxmuhcpftf0pajdhfvsqf5 default key name or Bech32 address for deploying packages
-empty-blocks=false enable creation of empty blocks (default: ~1s interval)
-empty-blocks-interval 1 set the interval for creating empty blocks (in seconds)
-genesis ... load the given genesis file
-interactive=false enable gnodev interactive mode
-lazy-loader=true enable lazy loader
-log-format console log output format, can be `json` or `console`
-max-gas 10000000000 set the maximum gas per block
-no-replay=false do not replay previous transactions upon reload
-no-watch=false do not watch for file changes
-no-web=false disable gnoweb
-node-rpc-listener 127.0.0.1:26657 listening address for GnoLand RPC node
-paths ... additional paths to preload in the form of "gno.land/r/my/realm", separated by commas; glob is supported
-resolver ... list of additional resolvers (`root`, `local`, or `remote`) in the form of <resolver>=<location> will be executed in the given order
-txs-file ... load the provided transactions file (refer to the documentation for format)
-unsafe-api=true enable /reset and /reload endpoints which are not safe to expose publicly
-v=false enable verbose output for development
-web-help-remote ... gnoweb: web server help page's remote addr (default to <node-rpc-listener>)
-web-home ... gnoweb: set default home page, use `/` or `:none:` to use default web home redirect
-web-html=false gnoweb: enable unsafe HTML parsing in markdown rendering
-web-listener 127.0.0.1:8888 gnoweb: web server listener address
-web-with-html=false gnoweb: enable HTML parsing in markdown rendering

-C ... change directory context before running gnodev
-add-account ... add (or set) a premine account in the form `<bech32|name>[=<amount>]`
-balance-file ... load the provided balance file
-chain-domain gno.land set node ChainDomain
-chain-id dev set node ChainID
-deploy-key ... default key name or Bech32 address for deploying packages
-empty-blocks=false enable creation of empty blocks
-empty-blocks-interval 1 set the interval for creating empty blocks (in seconds)
-genesis ... load the given genesis file
-interactive=false enable gnodev interactive mode
-load auto package loading mode: auto, lazy, or full
-log-format console log output format: json or console
-max-gas 10000000000 set the maximum gas per block
-no-replay=false do not replay previous transactions upon reload
-no-watch=false do not watch for file changes
-no-web=false disable gnoweb
-node-rpc-listener ... listening address for GnoLand RPC node
-paths ... additional paths to preload (glob supported)
-txs-file ... load the provided transactions file
-unsafe-api=true enable /reset and /reload endpoints
-v=false enable verbose output
-web-home ... set default home page
-web-listener 127.0.0.1:8888 gnoweb: web server listener address
```

### `gnodev staging -h`
[embedmd]:# (.tmp/gnodev-staging-usage.txt)
```txt
USAGE
gnodev staging [flags] [package_dir...]
Expand All @@ -145,39 +124,31 @@ STAGING: Staging mode configures the node for server usage.
This mode is designed for stability and security, suitable for pre-deployment testing.
Interactive mode and unsafe API access are disabled to ensure a secure environment.
The log format is set to JSON, facilitating integration with logging systems.
Since lazy-load is disabled in this mode, the entire example folder from "gnoroot" is loaded by default.

Additionally, you can specify an additional package directory to load.

Full loading mode is used by default, pre-loading all discovered packages.

FLAGS
-add-account ... add (or set) a premine account in the form `<bech32|name>[=<amount>]`, can be used multiple time
-balance-file ... load the provided balance file (refer to the documentation for format)
-chain-domain gno.land set node ChainDomain
-chain-id dev set node ChainID
-deploy-key g1jg8mtutu9khhfwc4nxmuhcpftf0pajdhfvsqf5 default key name or Bech32 address for deploying packages
-empty-blocks=false enable creation of empty blocks (default: ~1s interval)
-empty-blocks-interval 1 set the interval for creating empty blocks (in seconds)
-genesis ... load the given genesis file
-interactive=false enable gnodev interactive mode
-lazy-loader=false enable lazy loader
-log-format json log output format, can be `json` or `console`
-max-gas 10000000000 set the maximum gas per block
-no-replay=false do not replay previous transactions upon reload
-no-watch=false do not watch for file changes
-no-web=false disable gnoweb
-node-rpc-listener 127.0.0.1:26657 listening address for GnoLand RPC node
-paths gno.land/** additional paths to preload in the form of "gno.land/r/my/realm", separated by commas; glob is supported
-resolver ... list of additional resolvers (`root`, `local`, or `remote`) in the form of <resolver>=<location> will be executed in the given order
-txs-file ... load the provided transactions file (refer to the documentation for format)
-unsafe-api=false enable /reset and /reload endpoints which are not safe to expose publicly
-v=false enable verbose output for development
-web-help-remote ... gnoweb: web server help page's remote addr (default to <node-rpc-listener>)
-web-home :none: gnoweb: set default home page, use `/` or `:none:` to use default web home redirect
-web-html=false gnoweb: enable unsafe HTML parsing in markdown rendering
-web-listener 127.0.0.1:8888 gnoweb: web server listener address
-web-with-html=false gnoweb: enable HTML parsing in markdown rendering

-add-account ... add (or set) a premine account in the form `<bech32|name>[=<amount>]`
-balance-file ... load the provided balance file
-chain-domain gno.land set node ChainDomain
-chain-id dev set node ChainID
-deploy-key ... default key name or Bech32 address for deploying packages
-empty-blocks=false enable creation of empty blocks
-empty-blocks-interval 1 set the interval for creating empty blocks (in seconds)
-genesis ... load the given genesis file
-interactive=false enable gnodev interactive mode
-load full package loading mode: auto, lazy, or full
-log-format json log output format: json or console
-max-gas 10000000000 set the maximum gas per block
-no-replay=false do not replay previous transactions upon reload
-no-watch=false do not watch for file changes
-no-web=false disable gnoweb
-node-rpc-listener ... listening address for GnoLand RPC node
-paths ... additional paths to preload (glob supported)
-txs-file ... load the provided transactions file
-unsafe-api=false enable /reset and /reload endpoints
-v=false enable verbose output
-web-home :none: set default home page
-web-listener 127.0.0.1:8888 gnoweb: web server listener address
```

### Transaction file format
Expand Down
74 changes: 60 additions & 14 deletions contribs/gnodev/app.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,13 +151,10 @@ func (ds *App) Setup(ctx context.Context, dirs ...string) (err error) {
loggerEvents := ds.logger.WithGroup(EventServerLogName)
ds.emitterServer = emitter.NewServer(loggerEvents)

// XXX: it would be nice to not have this hardcoded
examplesDir := filepath.Join(ds.cfg.root, "examples")

// Setup loader and resolver
// Setup loader
loaderLogger := ds.logger.WithGroup(LoaderLogName)
resolver, localPaths := setupPackagesResolver(loaderLogger, ds.cfg, dirs...)
ds.loader = packages.NewGlobLoader(examplesDir, resolver)
var localPaths []string
ds.loader, localPaths = setupPackagesLoader(loaderLogger, ds.cfg, dirs...)

// Get user's address book from local keybase
accountLogger := ds.logger.WithGroup(AccountsLogName)
Expand Down Expand Up @@ -195,8 +192,9 @@ func (ds *App) Setup(ctx context.Context, dirs ...string) (err error) {

address := resolveUnixOrTCPAddr(nodeCfg.TMConfig.RPC.ListenAddress)

// Setup lazy proxy
if ds.cfg.lazyLoader {
// Setup lazy proxy (enabled for auto and lazy modes, disabled for full mode)
enableProxy := ds.cfg.loadMode != LoadModeFull
if enableProxy {
proxyLogger := ds.logger.WithGroup(ProxyLogName)
ds.proxy, err = proxy.NewPathInterceptor(proxyLogger, address)
if err != nil {
Expand All @@ -222,14 +220,32 @@ func (ds *App) Setup(ctx context.Context, dirs ...string) (err error) {
}
ds.DeferClose(ds.devNode.Close)

// Setup default web home realm, fallback on first local path
// Setup default web home realm, only considering realm paths (/r/)
devNodePaths := ds.devNode.Paths()

// Filter to only realm paths
realmPaths := make([]string, 0, len(devNodePaths))
for _, p := range devNodePaths {
if strings.Contains(p, "/r/") {
realmPaths = append(realmPaths, p)
}
}

switch webHome := ds.cfg.webHome; webHome {
case "":
if len(devNodePaths) > 0 {
ds.webHomePath = strings.TrimPrefix(devNodePaths[0], ds.cfg.chainDomain)
ds.logger.WithGroup(WebLogName).Info("using default package", "path", devNodePaths[0])
// Only set web home if there are realm paths
if len(realmPaths) > 0 {
var homePath string
if ds.cfg.loadMode == LoadModeAuto && len(realmPaths) > 1 {
// Compute highest common root for auto mode with multiple realms
homePath = commonPathPrefix(realmPaths)
ds.logger.WithGroup(WebLogName).Info("using common root", "path", homePath)
} else {
// Single realm or non-auto mode: use first realm path
homePath = realmPaths[0]
ds.logger.WithGroup(WebLogName).Info("using default realm", "path", homePath)
}
ds.webHomePath = strings.TrimPrefix(homePath, ds.cfg.chainDomain)
}
case "/", ":none:": // skip
default:
Expand Down Expand Up @@ -261,7 +277,7 @@ func (ds *App) setupHandlers(ctx context.Context) (http.Handler, error) {
// Generate initial paths
initPaths := map[string]struct{}{}
for _, pkg := range ds.devNode.ListPkgs() {
initPaths[pkg.Path] = struct{}{}
initPaths[pkg.ImportPath] = struct{}{}
}

ds.proxy.HandlePath(func(paths ...string) {
Expand All @@ -274,7 +290,6 @@ func (ds *App) setupHandlers(ctx context.Context) (http.Handler, error) {

// Try to resolve the path first.
// If we are unable to resolve it, ignore and continue

if _, err := ds.loader.Resolve(path); err != nil {
proxyLogger.Debug("unable to resolve path",
"error", err,
Expand Down Expand Up @@ -545,6 +560,37 @@ func (ds *App) handleKeyPress(ctx context.Context, key rawterm.KeyPress) {
}
}

// commonPathPrefix returns the highest common directory prefix of all paths.
// For example: ["a/b/x", "a/b/y"] -> "a/b"
func commonPathPrefix(paths []string) string {
switch len(paths) {
case 0:
return ""
case 1:
return paths[0]
}

// Split first path into segments
parts := strings.Split(paths[0], "/")

// Find common prefix length across all paths
for _, p := range paths[1:] {
otherParts := strings.Split(p, "/")

// Trim parts to common length
newLen, minLen := 0, min(len(parts), len(otherParts))
for i := range minLen {
if parts[i] != otherParts[i] {
break
}
newLen = i + 1
}
parts = parts[:newLen]
}

return strings.Join(parts, "/")
}

// XXX: packages modifier does not support glob yet
func resolvePackagesModifier(cfg *AppConfig, bk *address.Book, qpaths []string) ([]gnodev.QueryPath, []string, error) {
modifiers := make([]gnodev.QueryPath, 0, len(qpaths))
Expand Down
Loading
Loading