feat: shallow clone git repository sources (including auth changes)

[wale]

🔗 Linked issue

Reopens and closes #3338, continues #3531

❓ Type of change

• 📖 Documentation (updates to the documentation or readme)
• 🐞 Bug fix (a non-breaking change that fixes an issue)
• 👌 Enhancement (improving an existing functionality like performance)
• ✨ New feature (a non-breaking change that adds functionality)
• ⚠️ Breaking change (fix or feature that would cause existing functionality to change)

📚 Description

Creates the option of shallow cloning (--depth=1 --single-branch) a Git repository, by enabling a boolean option cloneRepository.

export default defineContentConfig({
  collections: {
    docs: defineCollection({
      type: 'page',
      source: {
        repository: 'https://github.com/nuxt/content',
        include: 'docs/content/**',
      },
    })
  }
})

This change also adds support for cloning repositories with Git refs (tags and branches only), with another option in the source definition; alongside support for usernames with authentication tokens, as an attempt to support Forgejo-based (and other odd) providers.

export default defineContentConfig({
  collections: {
    docs: defineCollection({
      type: 'page',
      source: {
        repository: {
          url: 'https://github.com/nuxt/content',
          branch: 'v1',
          // tag: 'fake-tag',
          auth: {
            username: 'username',
            password: 'password'
            // token: 'ghp_TOKENHERE'
          }
        },
        include: 'docs/content/**',
      },
    })
  }
})

The change was required as I use a non-Github/BitBucket repository for external Markdown files, and the current solution of submodules + git-sparse-checkout was no longer tenable.

The Git source also stores .content.cache.json differently to Github/Bitbucket sources, like so:

{
  "url": "<repo-url>",
  "hash": "<git-commit-hash>",
  "createdAt": "2025-09-16T08:07:28.525Z",
  "updatedAt": "2025-09-16T08:07:28.525Z"
}

where the Git commit hash is referencing git ls-remote.

Uses isomorphic-git to avoid shelling out to the git CLI. Needs testing on bun environments.

📝 Checklist

• I have linked an issue or discussion.
• I have updated the documentation accordingly.

[vercel]

@wale is attempting to deploy a commit to the NuxtLabs Team on Vercel.

A member of the Team first needs to authorize it.

[pkg-pr-new]

npm i https://pkg.pr.new/@nuxt/content@3542

commit: b1f4678

[farnabaz]

Thanks, @wale.
I'm wondering—can’t we use the same login (with isomorphic-git) for GitHub and Bitbucket?
Ideally, removing cloneRepository and ref and add support for object syntax for repository

repository: string | { url,  branch } | { url, tag }

Also it would be awesome if we can compare performance of old logic and new one

[wale]

Thanks, @wale. I'm wondering—can’t we use the same login (with isomorphic-git) for GitHub and Bitbucket? Ideally, removing cloneRepository and ref and add support for object syntax for repository

I had initially figured it was out of scope, but that is a good idea, the change would make it much more cohesive.

Ideally, removing cloneRepository and ref and add support for object syntax for repository

repository: string | { url,  branch } | { url, tag }

That's a good idea, although wouldn't it be more sensible to keep both the previous logic (i.e. downloading a tarball from provider) and this PR? In case one doesn't work, the other might make more sense to use; preferably as Github and Bitbucket specific sources.

[farnabaz]

I'd keep only one solution if it's generic enough, So if new solution works for Github and Bitbucket lets use it. But we can provide a compatibility solution and deprecate old usage before removing them all.

As an alternative, I was thinking about using giget package to download repositories. We need to check about the coverage and performance of giget and isomorphic-git

[wale]

I'd keep only one solution if it's generic enough, So if new solution works for Github and Bitbucket lets use it. But we can provide a compatibility solution and deprecate old usage before removing them all.

That's true, I'll keep my generic solution, and hide the previous logic behind a boolean (downloadTarball?)

As an alternative, I was thinking about using giget package to download repositories.
We need to check about the coverage and performance of giget and isomorphic-git

Personally I think giget is out of the question, because it has a similar issue of a limited amount of providers (+ not really using Git), kicking my specific issue upstream. I'm using isomorphic-git because it covers my requirements of performing a git clone operation, and users downstream aren't required to implement their own provider.

[farnabaz]

That's true, I'll keep my generic solution, and hide the previous logic behind a boolean (downloadTarball?)
By compatibility I don't mean the tar ball download. I mean the config syntax.

What I like is to make it work with current config. And deprecate it in favor of new syntax if needed. So users don't need to change their config on upgrade.

If any change is necessary we force it in v4

[vercel]

When a repository is provided as a string URL with a branch or tag (e.g., https://github.com/owner/repo/tree/mybranch), the extracted branch/tag is not passed to downloadGitRepository, causing the wrong branch to be cloned.

View Details

📝 Patch Details

diff --git a/src/utils/source.ts b/src/utils/source.ts
index a838cbf2..98586c67 100644
--- a/src/utils/source.ts
+++ b/src/utils/source.ts
@@ -55,7 +55,12 @@ export function defineGitSource(source: CollectionSource): ResolvedCollectionSou
         const { source: gitSource, owner, name } = repository
         resolvedSource.cwd = join(rootDir, '.data', 'content', `${gitSource}-${owner}-${name}-${repository.ref || 'main'}`)
 
-        await downloadGitRepository(source.repository!, resolvedSource.cwd!)
+        let ref: { branch?: string; tag?: string } | undefined
+        if (repository.ref) {
+          ref = { branch: repository.ref }
+        }
+
+        await downloadGitRepository(repository.toString(), resolvedSource.cwd!, undefined, ref)
       }
     }
 
@@ -65,7 +70,7 @@ export function defineGitSource(source: CollectionSource): ResolvedCollectionSou
         const { source: gitSource, owner, name } = repository
         resolvedSource.cwd = join(rootDir, '.data', 'content', `${gitSource}-${owner}-${name}-${repository.ref || 'main'}`)
 
-        let ref: object | undefined
+        let ref: { branch?: string; tag?: string } | undefined
 
         if (source.repository.branch && source.repository.tag) {
           throw new Error('Cannot specify both branch and tag for git repository. Please specify one of `branch` or `tag`.')

Analysis

Branch reference not passed to git.clone() in defineGitSource()

What fails: defineGitSource() in src/utils/source.ts extracts branch from URL (e.g., https://github.com/owner/repo/tree/mybranch) but passes malformed URL to downloadGitRepository(), causing git.clone() to fail with 404 error

How to reproduce:

const source = {
  repository: 'https://github.com/owner/repo/tree/mybranch',
  include: '**/*.md'
};
const resolved = defineGitSource(source);
await resolved.prepare({ rootDir: '/tmp' }); // git.clone() fails with 404

Result: Git clone fails with "HTTP Error: 404 Not Found" because git.clone() receives https://github.com/owner/repo/tree/mybranch (GitHub web UI format) instead of clean git URL

Expected: Should extract clean URL via repository.toString() and pass branch as separate ref parameter: downloadGitRepository('https://github.com/owner/repo', cwd, undefined, {branch: 'mybranch'})