Add ContentsManager.show_globs trait

[dlqqq]

Motivation

It would be nice to have traitlets configuration that allows extensions to selectively show files that are hidden via the ContentsManager by default. In Jupyter AI, we would like to use this setting to show .jupyter/ always by default, since this is where people can upload their custom AI personas (which will be much easier to create in Jupyter AI v3.2, BTW!).

Currently, we have to work around the lack of an equivalent ContentsManager.show_globs trait by setting ContentsManager.allow_hidden=True and setting ContentsManager.hide_globs to try to hide everything a user probably doesn't want to see. But, we have gotten complaints about our hide_globs configuration being either too loose or too wide. So I've put together my recommended fix here.

Summary

Adds a show_globs List(Unicode) trait alongside hide_globs, defaulting to []. A path matching any show_globs pattern is always shown, taking precedence over both hide_globs and hidden-file filtering. This lets a deployment layer targeted exceptions on top of the otherwise all-or-nothing allow_hidden switch, so it can keep hidden files hidden by default while always exposing a curated set of paths.

A path is hidden when any of its components starts with a dot, and a show_globs pattern exempts such a component when it matches the component's bare name or its path from the root. A path is shown only when all of its hidden components are exempt. So [".jupyter"] surfaces the .jupyter directory and its non-hidden contents (e.g. .jupyter/personas/foo.py) even with allow_hidden=False, but a nested dotfile like .jupyter/.secret stays hidden unless a pattern covers it too (add ".jupyter/.secret" or ".jupyter/.*").

Technical details

should_list gains a show_globs check for the name-glob listing filter. The hidden-file behavior lives in FileContentsManager (inherited by the async manager), where the allow_hidden/is_hidden checks now also consult show_globs: the two directory-listing sites, plus the access gates in get, save, delete, and rename, so a shown path is listable, readable, and writable rather than 404ing when opened.

Testing

Added unit tests asserting that a name matching both lists is listed (show wins), that a show_globs hidden directory and its non-hidden contents are surfaced and accessible with allow_hidden=False, that a nested dotfile stays gated unless a pattern exempts it too, and that unrelated dotfiles remain hidden.

[dlqqq]

Actually this needs to take precedence over ContentsManager.allow_hidden to be useful. Going to mark this as ready again in a bit...

[dlqqq]

OK, this is a little harder than I thought. This might stay in draft for a bit. Need to eat dinner.

[krassowski]

Previous related discussions:

• Default value for ContentManager.hidden_file #995
• Change allow_hidden config default for easier hidden files access #1335
• Don't hide .so,.dylib files by default #1457

With the proposed show_globs, if jupyter-ai wants to show .jupyter and jupyterlab-git wants to always show .githignore, how can both achieve what they want without overwriting each other's configuration?

[dlqqq]

@krassowski Good question. Unfortunately I think this is a limitation of the traitlets API, it's possible to clobber other extensions' vendored configurations. I guess consumers just have to play nice for now, e.g.:

# BAD
c.ContentsManager.show_globs = ['.jupyter']

# GOOD
c.ContentsManager.show_globs = (c.ContentsManager.show_globs or []) + ['.jupyter']

[dlqqq]

I'm a bit stuck on this PR because doing glob matching on files is actually quite a bit more complex than I thought. I want the following behavior:

1. * should not match hidden files unless explicitly specified:

• i.e. .jupyter should not match .jupyter/.secret but match .jupyter/persona.py
• but, a user could do .jupyter/.* to match .jupyter/.secret

2. The ** syntax should be supported since it's nice.

3. The match should be based solely on the path relative to the server root - it shouldn't be actually trying to walk disk because that can be very slow for NFS volumes.

4. Whatever logic introduced should work across all the current supported Python versions.

However, there don't seem to be any good stdlib approaches that satisfy all 4 of these requirements. All of the ones I found fail at one of these requirements:

Option	1. * skips dotfiles (explicit .* to match)	2. ** supported	3. Pure path match (no disk walk)	4. All supported Pythons (3.10+)
fnmatch.fnmatch	❌ bare * matches .secret	❌ **≡*	✅	✅
glob.glob / Path.glob	⚠️ skips dotfiles but no explicit opt-in; include_hidden is 3.11+	✅	❌ walks the filesystem	⚠️ include_hidden 3.11+
PurePath.match	❌ bare * matches .secret; also unanchored	❌ no real ** before 3.13	✅	✅
glob.translate / PurePath.full_match	✅	✅	✅	❌ 3.13+ only

This PR shows an AI-generated custom implementation, but frankly I do not trust it so I'm keeping this in draft.

I do think it's worth considering to add a minimal dependency to support this & simplify glob handling across Jupyter Server, but I understand that this would be a larger change that requires consensus from more maintainers.

[krassowski]

Unfortunately I think this is a limitation of the traitlets API, it's possible to clobber other extensions' vendored configurations

I think that's true for List type, but I think there are other types and mechanisms that we could consider. For example, c.ServerApp.jpserver_extensions traitlet is a dictionray which AFAIK gets merged entries from all contributing extensions (
https://jupyter-server.readthedocs.io/en/latest/developers/extensions.html#making-an-extension-discoverable). It might be that jpserver_extensions is special-cased, I am not sure. If not then c.ContentsManager.show_globs = (c.ContentsManager.show_globs or []) + ['.jupyter'] makes sense, except that we should document how to do that in extensions code (maybe in initialize_settings?).

[vidartf-jpmc]

While having a feature to selectively show paths that are hidden by default seems good, I just have a note for the motivation:

Jupyter is normally meant to be ok to start from any folder. While the .jupyter config folder is often in the user's home dir or similar. The two might not overlap, so I would recommend that whatever way there is for configuring/adding personas should not rely on the config dir being within the root folder of the Jupyter server.