WIP: Multi Root Contexts #261

jeff-hykin · 2025-09-05T00:58:28Z

The design/approach/API is ready for critique, all tests pass.

Edit: now that I think about it, I should probably explain the changes. A lot of stuff had to be done to prevent circular dependencies. But I'll have to do that later.

Warning:

Linting/formatting is a complete mess
I need to add more type info and break out some minor changes (like the flake.nix) into a separate PR.

I just wanted to get this out so you could take a look in the meantime (might be several weeks/months before I get enough free time to check back in on this)

james-pre

Hey Jeff, thanks for the amazing work on this so far. I know this is still a work in progress but have some feedback.

One of my biggest concerns is that this PR appears to include some amount of PID namespacing— which is outside the scope of what I'm comfortable adding. I think we need to focus on just adding one feature in this PR. Mission creep is a very real problem, and one that I'd like to avoid. Moreover, there are some very broad changes to contexts' mechanics beyond just adding a mounts property to FSContext that I'm not sure are needed for the feature to work.

Also, I'd like this feature to be somewhat compatible with Linux's mount namespaces.

I apologize if this comes off as overly negative or pessimistic, however I want to make sure that we only make the changes needed for the feature: every line of code that is changed is another potential bug; every new line increases the amount of maintenance. I really hope that we can push this feature through without compromising quality.

Thanks again for putting so much work into this, I really appreciate it!

james-pre · 2025-09-05T01:01:21Z

src/internal/contexts.ts

 export interface FSContext {
 	/** The unique ID of the context */
-	readonly id: number;
+	readonly id: string;


Needs to stay number for compatiblity with older versions and with POSIX.

james-pre · 2025-09-05T01:02:02Z

scripts/test.js

Looks like formatting got messed up here. Make sure to keep tabs.

james-pre · 2025-09-05T01:02:40Z

deno.lock

Let's not add a bunch of extra lock files. package-lock.json is enough.

Yep, sorry. Ignore all the extra junk (flake.nix, edits to test.js, etc) This WIP is mainly to see what you think of getContext and .createChildContext I still need to clean up the misc files, type annotations, formatting, etc

james-pre · 2025-09-05T01:03:29Z

flake.nix

If this is for NixOS then cool, however I'm not sure we're ready for adding this tooling yet. Let's stick to adding the one feature for this PR.

james-pre · 2025-09-05T01:05:40Z

src/config.ts

+	/**
+	 * If set, sets case folding for the file system(s).
+	 * @default null
+	 */
+	caseFold?: CaseFold;
+


caseFold is already part of SharedConfig, this should be removed to de-dupe.

james-pre · 2025-09-05T02:26:43Z

src/index.ts

+import { context } from './context.js';
+const mounts = context.mounts;
+export { mounts };


Suggested change

import { context } from './context.js';

const mounts = context.mounts;

export { mounts };

import { context } from './context.js';

export const mounts = context.mounts;

james-pre · 2025-09-05T02:28:03Z

src/context.ts

+ * ```
+ */
+export function createIsolatedContext(options: RestrictedContextOptions = {}): BoundContext {
+	const rootContext = _createUnboundRootContext({


Is there a reason we can't just have this done via a mounts property on ContextInit?

readme.md

james-pre · 2025-09-05T02:29:45Z

readme.md

+import { configureIsolated, fs as globalThisFs } from '@zenfs/core';
+
+// create a local fs
+const {fs, path} = await configureIsolated({


Normally the context examples use fs for the global and ctx or context for the conext's FS. Let's stay consistent.

james-pre · 2025-09-05T02:30:17Z

src/backends/fetch.ts

 	async create(options: FetchOptions) {
 		const url = new URL(options.baseUrl);
-		url.pathname = normalizePath(url.pathname);
+		url.pathname = normalizePath(url.pathname, true); // TODO


Co-authored-by: James Prevett <[email protected]>

jeff-hykin · 2025-09-05T17:05:22Z

I apologize if this comes off as overly negative or pessimistic

No worries. Actually quite the opposite of worries; I think you and I are still on the same page even though you probably don't feel that way at all at the moment haha. I'm just doing a bad job of presenting this PR. I also maintain open source stuff and I know the dread of getting a massive PR. I'll be breaking this up, but its easier to do that with some guidance/feedback on a tests-passing system.

Most things you're pointing out, like a better fix to circular dependencies, I have already "gone down the rabbit hole" trying many alternatives. For example I HATE my _initContext function. It is only there because I think every other possible solution is worse (I spent 5 hours creating a visual DAG of dependencies, reworking path, stats, utils, fs, file_index just to try and avoid _initContext, but I proved it couldn't be done cleanly given some other constraints). Obviously I need to explain and make sure I'm not missing anything, but so far I think you and I are still on the same page.

I'll fix the context ID's that'll be easy (is that related to what you said about PID's?). I'll have to come back later to give a full explanation of why things like getContext seem necessary for multiple root contexts.

In the meantime, I agree performance is important. What performance tests can I run to show if something will/won't be a problem?

james-pre · 2025-09-08T16:45:23Z

I'll fix the context ID's that'll be easy (is that related to what you said about PID's?). I'll have to come back later to give a full explanation of why things like getContext seem necessary for multiple root contexts.

Yup. FSContext.id is conceptually the same as a PID on most OSes.

In the meantime, I agree performance is important. What performance tests can I run to show if something will/won't be a problem?

I wish ZenFS had comprehensive performance tests, but alas we don't. The best thing we do have is comparing the times reported by zenfs-test before and after the PR's changes.

I'm going to start working on sysfs first instead of procfs in order to avoid creating conflicts for this PR.

jeff-hykin · 2025-09-08T23:42:54Z

I'm going to start working on sysfs first instead of procfs in order to avoid creating conflicts for this PR.

Honestly, idk when I'll be able to jump back in on this so I'd say don't worry about creating conflicts.

One thing though, a bug that would be my first actual PR: I think resolve needs to use this to get the pwd. E.g. this bit of code Right now its always using the default context's pwd. Also, similar bug, normalizePath in utils needs this so it can pass it to resolve. Then, because of that, a lot of normalizePath and resolve calls need to be changed to resolve.call(this, x).

Later I'll explain why getContext makes it so much easier to catch bugs like that (which is how I found those).

james-pre · 2025-09-09T18:48:53Z

I'll probably make some of the minor changes / bug fixes (e.g. sorting out normalizePath and _resolve) on main.

jeff-hykin added 23 commits April 19, 2025 10:38

add deno lock

e97fc1f

get working on macos

bf93add

name distinction pt1

ec682e4

name distinction pt2

c4f385d

name distinction pt3

94b2719

add example to bindContext

837bd24

patch directory test to get context.test.js passing

953e334

fix --preserve in test script

05daf81

fix some not-awaited directory tests

e3e2791

add missing specifier for buffer

cb8f75b

isolated tree works!

0effbc9

add ergonomic interface for it

0e582f6

Merge remote-tracking branch 'upstream/main'

9860d4c

Merge branch 'main' into multi

80e08e8

create core approach to isolated context

93433ba

make normalizePath use context

f35774b

add flake to help with build issues

12ff9bf

allow visualizing circular dependencies

046d336

fix circular dependency

2d03216

Merge remote-tracking branch 'upstream/main' into multi

67c1d3c

merge fix; all tests passing except port

a35426a

remove debugging

35e686b

fix edgecase, all tests passing!

5bae834

jeff-hykin requested a review from james-pre as a code owner September 5, 2025 00:58

james-pre requested changes Sep 5, 2025

View reviewed changes

james-pre linked an issue Sep 5, 2025 that may be closed by this pull request

Virtual contexts: Multiple file systems in an isolated mount tree #218

Open

change feature name

773bc65

Co-authored-by: James Prevett <[email protected]>

james-pre mentioned this pull request Sep 9, 2025

Path resolution fails when changing PWD in contexts, normalizePath calls are duplicated. #263

Closed

Uh oh!

WIP: Multi Root Contexts #261

Are you sure you want to change the base?

WIP: Multi Root Contexts #261

Uh oh!

Conversation

jeff-hykin commented Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

james-pre left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin Sep 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeff-hykin commented Sep 5, 2025

Uh oh!

james-pre commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jeff-hykin commented Sep 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

james-pre commented Sep 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jeff-hykin commented Sep 5, 2025 •

edited

Loading

jeff-hykin Sep 5, 2025 •

edited

Loading

james-pre commented Sep 8, 2025 •

edited

Loading

jeff-hykin commented Sep 8, 2025 •

edited

Loading