Skip to content

Action cache init fails on overlayfs: renameTo of corrupted cache throws EXDEV #31042

Description

@scaiper

Description of the bug:

We run Bazel in a container where the output base is backed by overlayfs. When the persistent action cache is corrupted, CompactPersistentActionCache.logAndThrowOrRecurse tries to preserve it via:

cacheRoot.renameTo(corruptedCacheRoot); // action_cache -> action_cache.bad

On overlayfs this rename fails with EXDEV ("Invalid cross-device link") when the action_cache directory exists only in the lower layer — the kernel refuses to rename pure-lower directories (unlike files, directories can't be transparently copied up on rename).

This is the same EXDEV error you get when renaming across different filesystems. Userspace tools like mv handle it gracefully by falling back to copy-and-delete; in Bazel it aborts the whole build:

Couldn't create action cache: ... Invalid cross-device link. If error persists, use 'bazel clean'.

I'd suggest, if the corrupted cache can't be moved aside, fall back to just deleting it.

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

No response

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 9.2.0

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?


If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

P1I'll work on this now. (Assignee required)type: bug

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions