Skip to content

du: match --exclude patterns after each '/', as GNU does - #16701

Open
nishant9144 wants to merge 3 commits into
uutils:mainfrom
nishant9144:du-exclude-suffix
Open

nishant9144 wants to merge 3 commits into
uutils:mainfrom
nishant9144:du-exclude-suffix

Conversation

@nishant9144

Copy link
Copy Markdown
Contributor

Fixes #16691

du matched an --exclude pattern only against the whole path and the file name. With . as the operand the paths start with ./, so --exclude=target/debug never matched ./target/debug, and target and ./target gave different totals.

GNU (exclude.c, used without EXCLUDE_ANCHORED) also tries the pattern on every part of the path that follows a /. This does the same, in a small is_excluded helper used for the operands and for the entries found while walking them, in both the openat and the regular traversal.

Compared with GNU du 9.4 on the issue's tree plus a nested a/target/debug (du -bs, bytes):

before after GNU
--exclude=target/debug . 1305 5 5
--exclude=target/debug ./target 1000 0 0
--exclude=target/debug a/target 300 0 0
--exclude=t*t/debug . 1305 5 5
--exclude=target ./target listed excluded excluded
--exclude=/target . 1305 1305 1305

A pattern starting with / still only matches a whole path, as in GNU.

I left the separate file-name check as it is, since #12771 is changing that line.

Tests

  • New test_du_exclude_matches_after_slash: fails on main, passes with the change.
  • cargo fmt, cargo clippy -p uu_du --all-targets -D warnings, and the du tests pass on Linux.

An --exclude pattern was matched only against the whole path and the
file name, so 'du --exclude=target/debug .' still counted
./target/debug, and './target' and 'target' gave different totals.

Like GNU's exclude.c, also try the pattern on every part of the path
that follows a '/', for the operands as well as the entries found while
walking them.

Fixes uutils#16691
Copilot AI balanced review requested due to automatic review settings November 18, 2026 19:51

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

Comment thread src/uu/du/src/du.rs Outdated
for pattern in &options.excludes {
if pattern.matches(&this_stat.path.to_string_lossy())
if is_excluded(pattern, &this_stat.path.to_string_lossy())
|| pattern.matches(&entry_name.to_string_lossy())

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is_excluded already tries the part after the last '/', which is entry_name, so this check is redundant now, no?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, removed in both traversals. is_excluded now splits on std::path::is_separator, so on Windows the last part is still the file name in du_regular.

Comment thread src/uu/du/src/du.rs Outdated
/// Whether `path` matches an `--exclude` pattern. As in GNU, the pattern is
/// tried on the whole path and on every part of it that follows a `/`, so
/// `target/debug` also excludes `./target/debug` and `a/target/debug`.
fn is_excluded(pattern: &Pattern, path: &str) -> bool {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could take &[Pattern] and do the loop itself? we have the same for pattern in excludes 3 times

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, it takes all the patterns now and the three loops are gone.

Comment thread tests/by-util/test_du.rs
}
ts.ucmd()
.arg("--exclude=target/debug")
.arg(".")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could you please also cover a glob like t*t/debug and a nested a/target/debug? you have them in the PR description

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added: both patterns, with ., ./target, .//target, a and ./a/target as operands, plus a check that a/target itself is still listed.

@github-actions

Copy link
Copy Markdown

Binary size comparison:

Individual binary size comparison VS main (threshold: >=5% AND >=4 KB).

Total size of compared binaries: 155.48 MB (+100 KB, +0.06%)

Significant per-binary changes:
  env     1.20 MB ->    1.28 MB  (+80 KB, +6.51%)

is_excluded now takes every pattern, so the three call sites no longer
loop over them, and drops the separate file-name check: the part after
the last separator is the file name. Splitting on any path separator
keeps that true on Windows.

Test a glob and a nested directory as well.
Copilot AI balanced review requested due to automatic review settings November 19, 2026 18:13

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

@github-actions

Copy link
Copy Markdown

GNU testsuite comparison:

Congrats! The gnu test tests/tail/follow-name is no longer failing!
Congrats! The gnu test tests/tail/pid-pipe is no longer failing!

@codspeed

codspeed Bot commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will regress 2 benchmarks

3 warnings, 1 note

⚠️ 120 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

⚠️ 8 benchmarks with a significant change ran on different hardware

Their base and head results were measured on machines with a different CPU, core count or memory, so the changes might not come from the code. Run base and head on the same runner type.

Learn more about Macro Runners

See affected benchmarks

⚠️ 5 benchmarks with a significant change ran in a different software environment

Their base and head results were measured with a different OS or toolchain, such as compiler, runtime or library versions, so the changes might come from this difference rather than from the code. Open an affected benchmark to see what changed.

See affected benchmarks

ℹ️ 1 benchmark has no flamegraph

The flamegraph wasn't generated: most of the time was spent in runtime internals, which would leave nothing useful to analyze.

See affected benchmarks

⚡ 6 improved benchmarks
❌ 2 regressed benchmarks
✅ 405 untouched benchmarks
⏩ 70 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
❌ CPU Simulation true_consecutive_calls 294.7 ns 350.2 ns -15.86%
❌ CPU Simulation dd_copy_with_skip 4.8 ms 4.9 ms -3.16%
⚡ Memory socket_addrs_direct[100000] 68 KB 8.7 KB ×7.9
⚡ Memory hostname_ip_lookup[100000] 66.4 KB 9.1 KB ×7.3
⚡ Memory cp_recursive_deep_tree[(120, 4)] 699.2 KB 581.2 KB +20.31%
⚡ CPU Simulation du_wide_tree[(5000, 500)] 15.6 ms 15 ms +4.2%
⚡ CPU Simulation cksum_crc 34.2 ms 33.1 ms +3.36%
⚡ CPU Simulation cksum_multiple_files 51.1 ms 49.5 ms +3.27%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing nishant9144:du-exclude-suffix (2e04735) with main (9311390)2

Open in CodSpeed

Footnotes

  1. 70 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

  2. No successful run was found on main (276ffe9) during the generation of this report, so 9311390 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

Copilot AI balanced review requested due to automatic review settings November 20, 2026 14:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing. Review the Copilot code review billing settings.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

du: adding ./ to the operand silently defeats --exclude=target/debug

3 participants