add support for nested gitignores

[luuumine]

hello,

i tried to use statix in my repo and had issues with it checking ignored files / folders.

issue: statix only cares about the .gitignore at the root of the repo and not the nested ones (that I largely use)

this PR support for nested .gitignore files when walking the repo, and adds a couple tests to make sure the patterns are applied relative to their directory.
the fix makes the walker pick up nested .gitignore files and apply their rules relative to the directory they’re in (if unrestricted is not true).

disclaimer: this PR was largely made with support from gemma4-12b

all the tests are passing and i have confirmed expected behavior by adding 2 more tests and testing against my repo.

please tell me if anything needs fixing or if that's not an issue you're willing to deal with!

have a great day

[mightyiam]

These tests, did you see them fail before the implementation commit?

[mightyiam]

Looking at this... I'm not sure why there would be distinct ignore and extra_ignores as opposed to a single ignores. Thoughts?

[luuumine]

i guess we could probably refactor that another way to avoid having a "repeatition" but my idea was the following:

• we need a Gitignore object (ignore element) at every step to know if the walker needs to traverse folders and files and everything
• but we need to keep the full list of ignored stuff (extra_ignores element) to rebuild a new ignore object when encountering a new .gitignore file in a nested subfolder

so i think we can't do without both? i may be wrong there but i don't see how to avoid this issue

[mightyiam]

This discussion brings me back to thoughts I had before. Why do we have implementation of such common file traversal with gitignore logic in this project?

[luuumine]

what do you mean? surely we want to not check gitignored files, to avoid raising errors on files that are not relevant to the user

[mightyiam]

What I mean is, isn't there a crate that "just does this" exact behavior?

[luuumine]

i didn't think about that earlier but i just saw that the regular ignore crate did..
I think we can just loop over their Walk iterator directly
https://docs.rs/ignore/latest/ignore/struct.Walk.html

This example shows the most basic usage of this crate. This code will recursively traverse the current directory while automatically filtering out files and directories according to ignore globs found in files like .ignore and .gitignore:

use ignore::Walk;

for result in Walk::new("./") {
    // Each item yielded by the iterator is either a directory entry or an
    // error, so either print the path or the error.
    match result {
        Ok(entry) => println!("{}", entry.path().display()),
        Err(err) => println!("ERROR: {}", err),
    }
}

[mightyiam]

Would you mind doing that, instead? Our test coverage should provide some confidence, right?

[luuumine]

sure, I'll try to look into it this weekend.
should I make a new PR?

[mightyiam]

Great! Whatever you find most convenient.

[luuumine]

i answered your points in the reply threads directly, feel free to ping me if needed

These tests, did you see them fail before the implementation commit?

yes! nested gitignores were ignored (thus leading to statix fully analyzing every .direnv in my repos!), panic log if needed:

lumine@luminix: statix on HEAD at af1a1c2 [$?1] via ❄️ default-shell-env via 🦀 v1.97.1 
>  cargo test -p statix --test walk
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.07s
     Running tests/walk.rs (target/debug/deps/walk-c85cf6aada5076bb)

running 12 tests
test directory::default_is_current_directory ... ok
test ignore_flag::excludes_matching_file ... ok
test ignore_flag::applies_even_when_unrestricted ... ok
test unrestricted::bypasses_gitignore ... ok
test gitignored_files::are_excluded ... ok
test directory::all_nix_files_are_linted ... ok
test single_file::only_it_is_linted ... ok
test gitignored_files::in_subdirs_are_excluded ... ok
test directory::nix_files_in_subdirs_are_linted ... ok
test directory::non_nix_files_are_excluded ... ok
test gitignored_files::nested_gitignore_patterns_are_relative_to_subdir ... FAILED
test gitignored_files::nested_gitignore_is_respected ... FAILED

failures:

---- gitignored_files::nested_gitignore_patterns_are_relative_to_subdir stdout ----

thread 'gitignored_files::nested_gitignore_patterns_are_relative_to_subdir' (1101649) panicked at bin/tests/walk.rs:183:9:
assertion `left == right` failed
  left: Paths({"./a/build/ignored.nix", "./linted.nix", "./b/build/not_ignored.nix"})
 right: ["./linted.nix", "./b/build/not_ignored.nix"]
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<walk::Paths, [&str; 2]>
             at /rustc/8bab26f4f68e0e26f0bb7960be334d5b520ea452/library/core/src/panicking.rs:394:5
   4: walk::gitignored_files::nested_gitignore_patterns_are_relative_to_subdir
             at ./tests/walk.rs:183:9
   5: walk::gitignored_files::nested_gitignore_patterns_are_relative_to_subdir::{closure#0}
             at ./tests/walk.rs:173:58
   6: <walk::gitignored_files::nested_gitignore_patterns_are_relative_to_subdir::{closure#0} as core::ops::function::FnOnce<()>>::call_once
             at /rustc/8bab26f4f68e0e26f0bb7960be334d5b520ea452/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

---- gitignored_files::nested_gitignore_is_respected stdout ----

thread 'gitignored_files::nested_gitignore_is_respected' (1101648) panicked at bin/tests/walk.rs:169:9:
assertion `left == right` failed
  left: Paths({"./generated/linted.nix", "./linted.nix"})
 right: ["./linted.nix"]
stack backtrace:
   0: __rustc::rust_begin_unwind
   1: core::panicking::panic_fmt
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed::<walk::Paths, [&str; 1]>
             at /rustc/8bab26f4f68e0e26f0bb7960be334d5b520ea452/library/core/src/panicking.rs:394:5
   4: walk::gitignored_files::nested_gitignore_is_respected
             at ./tests/walk.rs:169:9
   5: walk::gitignored_files::nested_gitignore_is_respected::{closure#0}
             at ./tests/walk.rs:160:39
   6: <walk::gitignored_files::nested_gitignore_is_respected::{closure#0} as core::ops::function::FnOnce<()>>::call_once
             at /rustc/8bab26f4f68e0e26f0bb7960be334d5b520ea452/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    gitignored_files::nested_gitignore_is_respected
    gitignored_files::nested_gitignore_patterns_are_relative_to_subdir

test result: FAILED. 10 passed; 2 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s

error: test failed, to rerun pass `-p statix --test walk````

[luuumine]

here's the new PR: #2756 with changes we discussed
i believe you can close this one now