pcre2grep(1): Please add `-T` flag

[alejandro-colomar]

$ MANWIDTH=72 man grep | sed -n '/-T/,/^$/p'
       -T, --initial-tab
              Make sure that the first character of actual line content
              lies  on  a tab stop, so that the alignment of tabs looks
              normal.  This is useful with options  that  prefix  their
              output to the actual content: -H,-n, and -b.  In order to
              improve  the  probability  that  lines from a single file
              will all start at the same column, this also  causes  the
              line number and byte offset (if present) to be printed in
              a minimum size field width.

This is a nice GNU extension for having aligned output. I've replaced some uses of grep -P by pcre2grep because grep(1) isn't powerful enough (it doesn't accept more than one pattern), and am missing the nice alignment.

[PhilipHazel]

I always thought that tabs were a disaster and should be considered harmful. :-)

[alejandro-colomar]

Actually, with spaces it gets worse. The alignment is unconditionally messed by grep(1) (unless you use -T), while with tabs it's only messed sometimes.

alx@debian:~/tmp$ cat fooo | nl
     1	foo {
     2		foo1;
     3		foo2;
     4		foo3;
     5		foo4;
     6		foo5;
     7		foo6;
     8		foo7;
     9		bar8;
    10		bar9;
    11	}
alx@debian:~/tmp$ grep -n bar fooo
9:	bar8;
10:	bar9;
alx@debian:~/tmp$ grep -n bar fooo /dev/null
fooo:9:	bar8;
fooo:10:	bar9;
alx@debian:~/tmp$ grep -Tn bar fooo /dev/null
fooo: 9:		bar8;
fooo:10:		bar9;
alx@debian:~/tmp$ sed 's/\t/        /' <fooo >spcs
alx@debian:~/tmp$ grep -n bar spcs 
9:        bar8;
10:        bar9;
alx@debian:~/tmp$ grep -n bar spcs /dev/null
spcs:9:        bar8;
spcs:10:        bar9;
alx@debian:~/tmp$ grep -Tn bar spcs /dev/null
spcs:  9:	        bar8;
spcs: 10:	        bar9;

[carenas]

While there is a precedent of formatted output with -O, I am not sure if -T provides a powerful enough alternative to post processing the output yourself to align it whichever way it is preferred.

To align the line numbers, we will likely need to delay printing the matches until the whole file is processed, and I presume the alignment might still be wrong if the next file name is too long or too short.

[alejandro-colomar]

While there is a precedent of formatted output with -O, I am not sure if -T provides a powerful enough alternative to post processing the output yourself to align it whichever way it is preferred.

Hmmm, yeah, I could try that.

To align the line numbers, we will likely need to delay printing the matches until the whole file is processed, and I presume the alignment might still be wrong if the next file name is too long or too short.

I only care about alignment in the same file; I assume different files might be differently aligned. But it would be nice to align for line numbers, which is my main issue. For that, you could even hardcode some heuristics, such as assuming a file won't be larger than say, INT_MAX lines, and thus make room for 10 digits.

[carenas]

While there is a precedent of formatted output with -O, I am not sure if -T provides a powerful enough alternative to post processing the output yourself to align it whichever way it is preferred.

Hmmm, yeah, I could try that.

installed GNU grep and it shows (where f is just a copy of your original test file):

% ggrep -HnT bar fooo f                         
fooo: 9:		bar8;
fooo:10:		bar9;
f: 9:		bar8;
f:10:		bar9;
% pcre2grep -Hn bar fooo f | awk -F: '{ printf("%4s:%2d:%s\n", $1, $2, $3); }'
fooo: 9:	bar8;
fooo:10:	bar9;
   f: 9:	bar8;
   f:10:	bar9;

[alejandro-colomar]

While there is a precedent of formatted output with -O, I am not sure if -T provides a powerful enough alternative to post processing the output yourself to align it whichever way it is preferred.

Hmmm, yeah, I could try that.

installed GNU grep and it shows (where f is just a copy of your original test file):

% ggrep -HnT bar fooo f                         
fooo: 9:		bar8;
fooo:10:		bar9;
f: 9:		bar8;
f:10:		bar9;
% pcre2grep -Hn bar fooo f | awk -F: '{ printf("%4s:%2d:%s\n", $1, $2, $3); }'
fooo: 9:	bar8;
fooo:10:	bar9;
   f: 9:	bar8;
   f:10:	bar9;

Yeah, some script could work. This one isn't very robust, as it would break with files that have ':' in them:

alx@debian:~/tmp$ cat foo2 
foo {
	foo1;
	foo2;
	foo3;
	foo4;
	foo5;
	foo6;
	foo7;
	bar8;
	bar9:;
}
alx@debian:~/tmp$ pcre2grep -Hn bar foo2 | awk -F: '{ printf("%4s:%d:%s\n", $1, $2, $3); }'
foo2:9:	bar8;
foo2:10:	bar9

But something can be done in a similar way.

[alejandro-colomar]

Here's something more robust, but not pretty:

alx@debian:~/tmp$ cat foo2
foo {
	foo1;
	foo2;
	foo3;
	foo4;
	foo5;
	foo6;
	foo7;
	bar8;
	bar9:;
}
alx@debian:~/tmp$ pcre2grep -Hn bar foo2 | sed -E 's/([^:]+:[^:]+:)(.*)/\1\n\t\2/' | awk -F: '/^[^\t]/{ printf("%4s:%2d:\n", $1, $2); }; /^\t/ {print $0}' | sed 'N;s/\n\t//'
foo2: 9:	bar8;
foo2:10:	bar9:;

I wonder if it can be done in a more readable way.

[alejandro-colomar]

Slightly simpler:

alx@debian:~/tmp$ cat foo2 
foo {
	foo1;
	foo2;
	foo3;
	foo4;
	foo5;
	foo6;
	foo7;
	bar8;
	bar9:;
}
alx@debian:~/tmp$ pcre2grep -Hn bar foo2 | sed -E 's/([^:]+:[^:]+:)(.*)/\1\n\t\2/' | awk -F: '/^[^\t]/{ printf("%4s:%2d:", $1, $2); }; /^\t/ {print $0}' | sed 's/:\t/:/'
foo2: 9:	bar8;
foo2:10:	bar9:;

[NWilson]

I would accept a patch implementing -T for pcre2grep, if it isn't overly complicated.

The code in grep isn't all that complicated: https://cgit.git.savannah.gnu.org/cgit/grep.git/tree/src/grep.c
(search for align_tabs and offset_width).

The only downside is that -T isn't particularly clever, to be honest, and doesn't produce very nice output. It aligns filenames by \t which doesn't actually align the contents unless the filenames happen to be similar (equal mod 8, standard tab width). It aligns the line numbers by left-padding, but it uses a pretty pessimistic algorithm that takes the file's size (in bytes) as the maximum number of lines. So for a 1000-byte file with 20 lines, it assumes line numbers can range from 1-1000, and formats them as 4-characters wide. For input from stdin, it left-pads to 19 spaces (!!) because that's the maximum digits for a 64-bit value.

You can get prettier output with awk postprocessing.

So, do you want it? Not sure.

---

Anyway - GNU grep supports PCRE2 as a backend! Is there anything wrong with using grep -PTn ... to get PCRE2 matching?

[alejandro-colomar]

I would accept a patch implementing -T for pcre2grep, if it isn't overly complicated.

The code in grep isn't all that complicated: https://cgit.git.savannah.gnu.org/cgit/grep.git/tree/src/grep.c (search for align_tabs and offset_width).

The only downside is that -T isn't particularly clever, to be honest, and doesn't produce very nice output. It aligns filenames by \t which doesn't actually align the contents unless the filenames happen to be similar (equal mod 8, standard tab width). It aligns the line numbers by left-padding, but it uses a pretty pessimistic algorithm that takes the file's size (in bytes) as the maximum number of lines. So for a 1000-byte file with 20 lines, it assumes line numbers can range from 1-1000, and formats them as 4-characters wide. For input from stdin, it left-pads to 19 spaces (!!) because that's the maximum digits for a 64-bit value.

You can get prettier output with awk postprocessing.

I think I agree with this part. I think the awk(1) postprocessing ends up being nicer than grep(1)'s -T.

FWIW, I have applied this patch to where I needed this, because it works now:

commit 459e96d7ad44b4db98428959ac7740fbac48f092
Author: Alejandro Colomar <alx@kernel.org>
Date:   Sun Aug 24 17:51:45 2025 +0200

    share/mk/: lint-man-poems: Align output
    
    pcre2grep(1) doesn't have a -T flag, so we need to script a bit to do
    something similar.
    
    Signed-off-by: Alejandro Colomar <alx@kernel.org>

diff --git a/share/mk/lint/man/poems.mk b/share/mk/lint/man/poems.mk
index c1a55d58a..6a60cde5b 100644
--- a/share/mk/lint/man/poems.mk
+++ b/share/mk/lint/man/poems.mk
@@ -8,10 +8,12 @@ MAKEFILE_LINT_MAN_POEMS_INCLUDED := 1
 
 include $(MAKEFILEDIR)/build/man/man.mk
 include $(MAKEFILEDIR)/build/man/mdoc.mk
+include $(MAKEFILEDIR)/configure/build-depends/awk/awk.mk
 include $(MAKEFILEDIR)/configure/build-depends/coreutils/cat.mk
 include $(MAKEFILEDIR)/configure/build-depends/coreutils/echo.mk
 include $(MAKEFILEDIR)/configure/build-depends/coreutils/touch.mk
 include $(MAKEFILEDIR)/configure/build-depends/pcre2-utils/pcre2grep.mk
+include $(MAKEFILEDIR)/configure/build-depends/sed/sed.mk
 
 
 _XFAIL_LINT_man_poems := \
@@ -32,7 +34,12 @@ $(_LINT_man_poems): %.lint-man.poems.touch: % $(poems_pcre2grep) $(MK) | $$(@D)/
        $(CAT) <$< \
        | if $(PCRE2GREP) -f '$(poems_pcre2grep)' >/dev/null; then \
                >&2 $(ECHO) "lint-man-poems: $<: Use semantic newlines (see man-pages(7)):"; \
-               >&2 $(PCRE2GREP) -n -f '$(poems_pcre2grep)' <$<; \
+               $(PCRE2GREP) -n -f '$(poems_pcre2grep)' <$< \
+               | $(SED) -E 's/([^:]+:)(.*)/\1\n\t\2/' \
+               | $(AWK) -F: \
+                       '/^[^\t]/{ printf("%7d:", $$1); }; \
+                        /^\t/{ print($$0); };' \
+               >&2; \
                exit 1; \
        fi;
        $(TOUCH) $@

So, do you want it? Not sure.

Hmmmm, I'm not sure what to answer. Within a build system, even if awk(1) postprocessing of output might be nicer, there's still a point in using -T: keeping the build system simple. Given how the xz-backdoor was planned, I think having a simple build system that can be read by arbitrary people is a good thing, even if it doesn't produce the most pleasant alignment when used.

So, I think if you provided a -T flag, I'd start using it. So, yes, I want this feature.

Anyway - GNU grep supports PCRE2 as a backend! Is there anything wrong with using grep -PTn ... to get PCRE2 matching?

As I said earlier, for some reason, GNU grep(1) doesn't support more than one pattern matching. I need to match two patterns, which makes it impossible to use. I could put both regexes into something like (A|B) in a single line, but I prefer readable source code, so I really want to avoid that.

[NWilson]

Well then, patches welcome!

If we did do it, I'd suggest just keeping the line number alignment simple and using a fixed width (eg 9 characters).

[alejandro-colomar]

Well then, patches welcome!

If we did do it, I'd suggest just keeping the line number alignment simple and using a fixed width (eg 9 characters).

I agree with that. I think grep(1)'s heuristics are horrible. Anyway, they'll have misalignment when they change files, so missing the alignment in some large files isn't terrible.

However, I'd use a fixed width of 7, because that means the 8th character is the colon, and then the tab starts in a tab stop, which is a good thing. And after all, I don't expect to be using pcre2grep(1) on files larger than 9999999 lines (10M - 1).

[carenas]

Instead of having an implementation of -T, I would rather like to see a --table where we could assign each column to a field and indicate it to be left or right padded and its size, which might be more useful IMHO, we might even convince GNU grep to copy that design after.

[alejandro-colomar]

Instead of having an implementation of -T, I would rather see a --table where we could assign each column to a field and indicate it to be left of right padded and its size, which might be more useful IMHO, we might even convince GNU grep to copy that design after.

I think if anyone wants to be that fancy, they really want sed(1)/awk(1) post-processing.

-T is for "give me some cheap alignment, even if it's less-than-perfect".