fix: package search, login on JuliaHub.com-style installs, device-flow polling, search token scoping, --strict-phrase (#64)

* fix(packages): use /api/v1/ui/packages/info for package search and name lookup

`jh package search`, `jh search packages`, `jh package info` and every
`--package <name>` lookup called `/packages/info`, which current JuliaHub
does not route: the request falls through to the git `/packages/` handler
and returns 404. The GraphQL fallback is then refused ("query is not
allowed"), so all of these failed.

Call the web UI's package listing, `/api/v1/ui/packages/info`, instead. It
pages with `pagination[page]`/`pagination[per_page]` (at most 100 per page)
rather than limit/offset, so map the CLI's `--limit`/`--offset` window onto
the covering pages (`planRESTPages`). Order by relevance (`-score`) when a
search term is given, by stars otherwise, and always break ties by name:
packages from one repository share a score, and without a tiebreak their
order changes between requests, so consecutive pages could repeat or skip
rows.

* fix(auth): find the dex host of JuliaHub.com-style installs

Login and token refresh assumed dex lives at `<server>/dex`, special-casing
only juliahub.com (`auth.juliahub.com`). JuliaHub.com-style installs other
than production (e.g. nightly-juliahub.juliahub.dev) also serve dex from
`auth.<server>`, so `jh auth login -s <server>` failed with "failed to parse
device code response: invalid character '<'" (an HTML 502 from
`<server>/dex`), and the refresh URL written to Julia's auth.toml was wrong.

`authServerFor` now resolves the auth host once per process:
- juliahub.com stays auth.juliahub.com;
- for refresh, the stored token's issuer (`https://<host>/dex`) names it,
  trusted only when it is `<server>` or `auth.<server>`;
- otherwise (login) both candidates' OIDC discovery documents are probed
  concurrently and `auth.<server>` is preferred, `<server>` the fallback.

The stored `server=` remains the API host, so other commands keep working.

* fix(auth): poll the device-flow token endpoint as RFC 8628 requires

`jh auth login` slept 15 s, then polled every 4 s regardless of the
server's `interval` (5 s for dex) and gave up on any error other than
`authorization_pending`. So unless the user approved within ~15 s, the
second poll came too early, dex answered `slow_down`, and the login failed
with "authorization failed: slow_down".

Poll per RFC 8628 section 3.5: wait the server's interval (default 5 s)
before each poll, add 5 s to it on `slow_down`, keep polling on
`authorization_pending`, and stop with a clear message on `access_denied`,
`expired_token` or once the device code's `expires_in` has passed. The poll
loop is split out as `pollDeviceToken` with an injectable clock for tests.

Also fix the `openiod` typo in the poll request's scope.

* fix(search): send the stored token only to its own server

`jh search` (and the REST package listing behind `jh search packages`,
`jh package search` and `--package <name>`) required a stored login even
on JuliaHub.com-style installs, where search and the package listing are
public, and sent the stored login's ID token to whatever `--server` was
given, even when the token was issued by a different server.

Now the token is attached only when the stored login belongs to the target
host (`server=` matches, or is its `auth.` dex host). Otherwise the request
goes out without a token; a note on stderr explains why when a login for
another server exists, and a 401 is reported as "HTTP 401: Unauthorized"
plus "this server requires a login: run 'jh auth login -s <host>'" instead
of nginx's HTML error page. Package search no longer needs the registry
list (which requires a login) to run: without it, `--registries` names are
passed as given, and only the GraphQL fallback, which needs a login anyway,
is lost. When both REST and GraphQL fail, both errors are reported.

* fix(search): let `--strict-phrase=false` reach the server; fix `--threshold` help

`docsSearchRequest.StrictPhrase` was a plain bool with `omitempty`, so
`false` was never sent and the server's default applied. Shipped JuliaHub
config defaults to strict phrase matching, so `--strict-phrase` changed
nothing and there was no way to ask for documents containing the query's
words separately. Make it a `*bool` that is set only when the flag is given:
`--strict-phrase=false` now sends `"strictphrase": false` (e.g. 18 -> 14
results for "differential equations solver" on a default install), and
omitting the flag still leaves the server's default in effect.

The `--threshold` help called it a minimum relevance score; the server keeps
results scoring at least threshold x the top score, so describe it as a
fraction (0-1) of the best result's score.

* fix(packages): find a package's registry via the REST listing in `jh package dependency`

Without `--registry`, `jh package dependency <name>` looked the package's
registry up through the GraphQL package search only, which current JuliaHub
refuses ("query is not allowed"), so the command failed. Look it up in the
REST package listing first (`findPackageRegistry`) and keep GraphQL as the
fallback, reporting both errors if both fail.

* fix(auth): trust only a dex whose issuer matches its host; no probe for tokens we hold

Two follow-ups to the dex host discovery:

- `dexDiscoverable` accepted any discovery document with an issuer. Require
  the issuer to be the probed host's own `/dex` (as OIDC requires), so a
  host that relays another host's document (e.g. a wildcard ingress
  answering for `auth.<host>`) cannot be picked.
- `createJuliaAuthFile` (`jh run`, `jh run setup`, and after a refresh)
  resolved the refresh host through `authServerFor`, which probed the
  network even with a valid stored token. Seed it from the token's issuer
  first, as the refresh path does, so these commands make no extra calls.

* docs(readme): describe `--threshold` as a fraction of the top score and `--strict-phrase=false`

Author: tanmaykm

CLAUDE.md

@@ -14,7 +14,7 @@ The application follows a command-line interface pattern using the Cobra library
 - **auth.go**: OAuth2 device flow authentication with JWT token handling
 - **datasets.go**: Dataset operations (list, download, upload, status) with REST API integration
 - **registries.go**: Registry operations (list, config, add, update, registrator) with REST API integration
-- **packages.go**: Package operations (search, dependency) with REST API primary path (`/packages/info`), GraphQL fallback, and documentation API (`/docs/{registry}/{package}/stable/pkg.json`)
+- **packages.go**: Package operations (search, dependency) with REST API primary path (`/api/v1/ui/packages/info`), GraphQL fallback, and documentation API (`/docs/{registry}/{package}/stable/pkg.json`)
 - **projects.go**: Project management using GraphQL API. `jh project list` is paginated (`projectsPageSize` = 100 per page, one page per invocation, selected with `--page=N`, default 1) and filters by owner **server-side** (`buildProjectsFilter`). Never request the whole projects table: the Projects query costs the server several aggregates and permission checks per row, and an unpaginated request against a large install ran for hours after the 30 s client timeout fired
 - **user.go**: User information retrieval using GraphQL API and REST API for listing users
 - **tokens.go**: Token management operations (list) with REST API integration
@@ -37,7 +37,7 @@ The application follows a command-line interface pattern using the Cobra library
    - Stores tokens securely in `~/.juliahub` with 0600 permissions
 
 2. **API Integration**:
-   - **REST API**: Used for dataset operations (`/api/v1/datasets`, `/datasets/{uuid}/url/{version}`), registry operations (`/api/v1/registry/registries/descriptions`, `/api/v1/registry/config/registry/{name}`, `/api/v1/registry/config/registrator/{name}`), package search/info primary path (`/packages/info`), token management (`/app/token/activelist`), user management (`/app/config/features/manage`), admin group management (`/app/config/groups`), registry credential management (old API tried first, new API fallback), landing page management (`/api/v1/ui/layout/homepage` GET/POST — the landing page is the `greeter` widget of the homepage layout array; the CLI rewrites only that widget and POSTs the whole layout back), and vulnerability scanning (`/api/v1/ui/vulnerabilities/packages/{name}`)
+   - **REST API**: Used for dataset operations (`/api/v1/datasets`, `/datasets/{uuid}/url/{version}`), registry operations (`/api/v1/registry/registries/descriptions`, `/api/v1/registry/config/registry/{name}`, `/api/v1/registry/config/registrator/{name}`), package search/info primary path (`/api/v1/ui/packages/info`), token management (`/app/token/activelist`), user management (`/app/config/features/manage`), admin group management (`/app/config/groups`), registry credential management (old API tried first, new API fallback), landing page management (`/api/v1/ui/layout/homepage` GET/POST — the landing page is the `greeter` widget of the homepage layout array; the CLI rewrites only that widget and POSTs the whole layout back), and vulnerability scanning (`/api/v1/ui/vulnerabilities/packages/{name}`)
    - **GraphQL API**: Used for projects, user info, user list (`public_users`), group list, and package search/info fallback (`/v1/graphql`)
    - **Headers**: All GraphQL requests require `Authorization: Bearer <id_token>`, `X-Hasura-Role: jhuser`, and `X-Juliahub-Ensure-JS: true`
    - **Authentication**: Uses ID tokens (`token.IDToken`) for API calls
@@ -49,7 +49,7 @@ The application follows a command-line interface pattern using the Cobra library
    - `jh registry config`: Show registry JSON config by name; subcommands add/update accept JSON via stdin or `--file`
    - `jh registry permission`: Registry permission management (list, set, remove)
    - `jh registry registrator`: Show registrator config by name; subcommand update accepts JSON via stdin or `--file`
-   - `jh package`: Package search and dependency (REST primary via `/packages/info`, GraphQL fallback; dependency data from `/docs/{registry}/{package}/stable/pkg.json`)
+   - `jh package`: Package search and dependency (REST primary via `/api/v1/ui/packages/info`, GraphQL fallback; dependency data from `/docs/{registry}/{package}/stable/pkg.json`)
    - `jh project`: Project management (list with GraphQL, paginated, supports user filtering via `--user` and page selection via `--page`)
    - `jh user`: User information (info, list via GraphQL `public_users`)
    - `jh group`: Group information (list via GraphQL)
@@ -65,7 +65,7 @@ The application follows a command-line interface pattern using the Cobra library
    - `jh admin landing-page`: Landing page management (show/update/remove the home page's Welcome/greeter markdown card via the homepage layout API)
    - `jh vuln`: Vulnerability scanning for Julia packages (REST API; defaults to latest stable version via `GET /docs/<registry>/<package>/versions.json`; supports `--version` for a specific version, `--registry` to specify the registry for version lookup (default: General), `--advisory` to filter to a specific advisory ID, `--all` to show all advisories regardless of affected status, and `--verbose` for additional details)
    - `jh scan`: Manifest vulnerability scanning via the StaticAnalysis service (REST API under `/api/v0/static_analysis/`); uploads a `Manifest.toml` (and sibling `Project.toml` unless `--no-project`) and POSTs to `/manifest/scan` to obtain a `run_uuid`; subcommands `status` and `results` hit `/manifest/scan/status/{run_uuid}` and `/results/manifest/{run_uuid}`; **polls by default** and prints results when the scan completes — Ctrl+C detaches and the `run_uuid` (printed up front) can be passed to `jh scan status` / `jh scan results` later. Pass `--no-wait` to submit-and-exit. `--csv` requests `text/csv` (content-negotiated via `Accept` header)
-   - `jh search`: Search across indexed packages — `code <pattern>` (RE2 over source files; `--path`, `--ignore-case`), `symbols <name>` (`--type function|type|macro|module`, `--usage define|use`, validated client-side), `docs <query>` (`--threshold`, `--strict-phrase`), all with repeatable `--package` (name or UUID) / `--registry`, `--limit` and `--json`; `packages [term]` is `jh package search` under the search group (shares `runPackageSearch`)
+   - `jh search`: Search across indexed packages — `code <pattern>` (RE2 over source files; `--path`, `--ignore-case`), `symbols <name>` (`--type function|type|macro|module`, `--usage define|use`, validated client-side), `docs <query>` (`--threshold` — a fraction 0-1 of the top score, `--strict-phrase[=false]` — tri-state, both omitted from the request unless set so the server's defaults apply), all with repeatable `--package` (name or UUID) / `--registry`, `--limit` and `--json`; `packages [term]` is `jh package search` under the search group (shares `runPackageSearch`)
    - `jh clone`: Git clone with JuliaHub authentication and project name resolution
    - `jh push/fetch/pull`: Git operations with JuliaHub authentication
    - `jh git-credential`: Git credential helper for seamless authentication
@@ -318,15 +318,15 @@ go run . run -- --project=. --threads=4 -e "println(Threads.nthreads())"
 
 The CLI supports multiple JuliaHub environments:
 - Default: `juliahub.com` (uses `auth.juliahub.com` for auth)
-- Custom servers: Direct server specification
+- Custom servers: Direct server specification. The dex (auth) host is found by `authServerFor`: enterprise installs serve dex at `<server>/dex`, JuliaHub.com-style installs at `auth.<server>/dex`. Login probes both OIDC discovery documents (`/dex/.well-known/openid-configuration`, concurrently, `auth.` preferred); refresh takes the host from the stored token's `iss` claim instead (`rememberAuthServerFromToken`). The stored `server=` stays the API host
 - Server normalization: Automatically appends `.juliahub.com` to short names
 
 ## Authentication Flow
 
 The application uses OAuth2 device flow:
 1. Request device code from `/dex/device/code`
 2. Present verification URL to user
-3. Poll `/dex/token` endpoint until authorization complete
+3. Poll `/dex/token` endpoint until authorization complete (`pollDeviceToken`, RFC 8628 §3.5: the server's `interval` (default 5 s) between polls, +5 s on `slow_down`, stop on `access_denied` / `expired_token` / after `expires_in`)
 4. Store tokens in configuration file with JWT claims extraction
 
 ## API Patterns
@@ -343,7 +343,7 @@ The application uses OAuth2 device flow:
 - **User management**: `/app/config/features/manage` endpoint for listing all users
 - **Token management**: `/app/token/activelist` endpoint for listing all API tokens
 - **Registry credentials**: `GET /api/v1/sysconfig/credentials` to fetch (returns `CredentialsInfo` object directly); `POST` to add tokens/apps; `PUT` to update tokens/apps or replace all SSH credentials; `DELETE` with `{tokens:[...], githubApps:[...]}` to delete
-- **Package search/info primary**: `/packages/info` endpoint with `name`, `registries`, `tags`, `licenses`, `limit`, `offset` query params; returns `{packages: [...], meta: {total: N}}`
+- **Package search/info primary**: `/api/v1/ui/packages/info` (the web UI's package listing) with `name`, `registries` (comma-separated names), `tags`, `licenses`, `sorts[0]` (`-score` when searching by name) and page-based pagination `pagination[type]=offset&pagination[page]=N&pagination[per_page]=M` (M ≤ 100); returns `{packages: [...], meta: {total: N}}`. The CLI's `--limit`/`--offset` window is mapped onto pages by `planRESTPages`. (The older `/packages/info` path is not routed by current JuliaHub: it falls through to the git `/packages/` handler and 404s.)
 - **Package dependencies**: `/docs/{registry}/{package}/stable/pkg.json` for dependency information
 - **Search**: `POST /search/v1/code`, `/search/v1/sym`, `/search/v1/docs` with a JSON body (`pattern`, `package` [UUIDs], `registry` [names], `maxresults`, plus per-kind `pathfilter`/`ignorecase`, `types`/`usages`, `threshold`/`strictphrase`); 200 → `{"results": [...], "truncated": bool}`, any failure → non-2xx `{"code", "message", "details"?}` with stable codes (`invalid_request`, `request_too_large`, `invalid_pattern`, `invalid_pathfilter`, `invalid_types`, `invalid_usages`, `invalid_registry`, `search_failed`, `metadata_unavailable`). Older installs have only `POST /search/{code,sym,docs}`, which answer 200 with `{"success": bool, "data": <results or error string>}`; the CLI retries there when v1 answers 404/405
 - **Authentication**: Bearer token with ID token
@@ -524,12 +524,12 @@ jh run setup
 - Landing page commands (`jh admin landing-page`) use REST API: GET `/app/homepage` (show), POST `/app/config/homepage` (update), DELETE `/app/config/homepage` (remove); require appropriate permissions
 - Landing page `update` command accepts content inline as an argument, from a file via `--file`, or piped via stdin (priority: `--file` > arg > stdin)
 - Landing page response uses custom JSON unmarshaling (`homepageResponse`) to handle `message` being either an object or a string
-- Package search (`jh package search`) and info (`jh package info`) both try REST API (`/packages/info`) first, then fall back to GraphQL (`FilteredPackages` / `FilteredPackagesCount` via `/v1/graphql`) on failure; `searchPackages` prints `warning: REST package search failed (...); falling back to GraphQL` to stderr when it falls back (`getPackageInfo` falls back silently)
+- Package search (`jh package search`) and info (`jh package info`) both try REST API (`/api/v1/ui/packages/info`) first, then fall back to GraphQL (`FilteredPackages` / `FilteredPackagesCount` via `/v1/graphql`) on failure; `searchPackages` prints `warning: REST package search failed (...); falling back to GraphQL` to stderr when it falls back (`getPackageInfo` falls back silently)
 - `--registries` resolution for `jh package search` / `jh package info` / `jh search packages` goes through `resolveRegistries(server, csv)` in `registries.go` (empty = all registries; unknown name → `Registry not found: '<name>'`)
 - `jh package search` / `jh search packages --json` sets `PackageSearchParams.JSON`; both the REST and GraphQL paths then call `writePackagesJSON`, which prints `{"results": [...], "total": N}` (snake_case keys from `packageInfo`'s json tags, `tags` always an array) 2-space indented with a trailing newline — nothing else on stdout
 - **JSON output convention** (`jh search *`): `--json` prints only the document on stdout, 2-space indented, trailing newline — `{"results": <server array, untouched>, "truncated": bool}` for code/symbols/docs; warnings, the human-mode `note: results truncated (search stopped early)` line, and errors always go to stderr; exit code is 0/1 only
 - `jh search` HTTP flow: `postSearch(baseURL, kind, body)` → `postSearchWithToken(client, baseURL, kind, token, body)` → `doSearchRequest(client, baseURL, path, token, body)`; v1 first, legacy route on 404/405, `search_unavailable` when both are missing. Non-2xx bodies become `*searchError{Status, Code, Message, Details}` (`Error()` is `<code>: <message>`, or `HTTP <status>: <body>` when undecodable; legacy `success:false` → code `legacy_error`); `printSearchError` adds a `known registries: ...` line from `details.known` on `invalid_registry`
-- `--server` for `jh search code|symbols|docs` is resolved by `searchBaseURL`: an explicit `http://`/`https://` URL is used verbatim (trailing slash trimmed) so the commands can target a local search service (`--server http://localhost:4446`); anything else is `https://` + `normalizeServer`. Against an `http://` base a missing stored token is tolerated (request sent without `Authorization`, note on stderr) and `--package` must be a UUID since name lookup needs `/packages/info`; `https://` still requires `ensureValidToken()`
+- `--server` for `jh search code|symbols|docs` is resolved by `searchBaseURL`: an explicit `http://`/`https://` URL is used verbatim (trailing slash trimmed) so the commands can target a local search service (`--server http://localhost:4446`); anything else is `https://` + `normalizeServer`. Against an `http://` base a missing stored token is tolerated (request sent without `Authorization`, note on stderr) and `--package` must be a UUID since name lookup needs the REST package listing. Against an `https://` JuliaHub host the stored login's token is sent only when that login belongs to the host (`tokenForServer` / `chooseToken`; a login stored as `auth.<host>` counts) — otherwise the request goes out without a token (JuliaHub.com-style installs allow anonymous search and package listing), a note on stderr says why when a login for another server exists, and a 401 gets the hint `run 'jh auth login -s <host>'`. The REST package listing (`fetchRESTPackages`) follows the same rule, and `jh package search` / `jh search packages` fall back to the `--registries` names as given when the registry list cannot be fetched without a login
 - `--package` names are resolved with `fetchRESTPackages(server, name, 100, 0, nil)` + exact case-insensitive match; the same package appears once per registry with the same UUID, so UUIDs are deduplicated, and distinct packages sharing a name all contribute; no match → `package not found: <name>`. `--registry` names are passed through verbatim for the server to validate
 - REST API passes `--registries` as comma-separated registry names to the `registries` query param; GraphQL fallback passes registry IDs to the `registries` variable
 - `fetchRegistries` in `registries.go` is used by `listRegistries`, `packageSearchCmd`, `packageInfoCmd`, and `packageDependencyCmd` to resolve registry names to IDs (for GraphQL) and names (for REST)

README.md

@@ -306,8 +306,8 @@ Search the Julia packages indexed on JuliaHub. Every subcommand takes `--json`,
   - `--usage <kind>` - `define` or `use` (repeatable)
   - `--package`, `--registry`, `--limit`, `--json` as for `code`
 - `jh search docs <query>` - Natural-language search over rendered package documentation; results are packages ranked by score with their matching sections
-  - `--threshold <x>` - Drop results scoring below `x`
-  - `--strict-phrase` - Require the query as a phrase rather than separate terms
+  - `--threshold <x>` - Keep results scoring at least `x` (0-1) times the best result's score
+  - `--strict-phrase` / `--strict-phrase=false` - Require the query as a phrase / match its words separately (default: the server's setting, strict on a default install)
   - `--package`, `--registry`, `--limit`, `--json` as for `code`
 - `jh search packages [search-term]` - The package catalogue search (same as `jh package search`), with `--registries`, `--limit`, `--offset`, `--verbose` and `--json` (`{"results": [...], "total": N}`)
 - `--server` / `-s` accepts a JuliaHub host as elsewhere, or a full URL such as `--server http://localhost:4446` to point `code`/`symbols`/`docs` at a locally running search service (no stored login required; the request is sent unauthenticated with a note on stderr, and `--package` must then be given as a UUID). The stored token is only ever sent over `https://`, or over `http://` to a loopback host; a plain-http URL on any other host gets an unauthenticated request and a note