Skip to content

Commit 6ab3389

Browse files
IvanNardiIvanNardi
authored
Add support for out-of-tree builds (#2993)
Initial work to support out-of-tree builds ``` ./autogen.sh mkdir build cd build ../configure make make check ``` IMPORTANT: `autogen.sh` doesn't call `configure` automatically anymore!! You have to do: `./autogen.sh && ./configure --$OPTIONS`. A little bit annoying but the pattern `autogen && configure && make` is very common on Linux. Known issues: * `make doc` doesn't work in out-of-tree builds, yet * Windows/MinGW/DPDK (out-of-tree) builds have not been tested, so it is unlikely they work See: #2992
1 parent e9751ce commit 6ab3389

26 files changed

+308
-180
lines changed

‎.github/workflows/build-rpm.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ jobs:
2323
run: |
2424
# symlink source directory for ndpi.spec
2525
ln -sr . $HOME/nDPI
26-
./autogen.sh
26+
./autogen.sh && ./configure
2727
cd packages/rpm
2828
./configure
2929

‎.github/workflows/build.yml‎

Lines changed: 56 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ jobs:
2929
sudo apt-get install autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev
3030
- name: Build nDPI library
3131
run: |
32-
./autogen.sh
32+
./autogen.sh && ./configure
3333
make -j
3434
sudo make install
3535
- name: Generate Python bindings
@@ -182,7 +182,7 @@ jobs:
182182
brew install libmaxminddb
183183
- name: Configure nDPI
184184
run: |
185-
./autogen.sh --enable-option-checking=fatal --enable-debug-messages ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs ${{ matrix.global_context}}
185+
./autogen.sh && ./configure --enable-option-checking=fatal --enable-debug-messages ${{ matrix.msan }} ${{ matrix.pcre }} ${{ matrix.maxminddb }} --enable-tls-sigs ${{ matrix.global_context}}
186186
- name: Build nDPI
187187
run: |
188188
make -j all
@@ -230,7 +230,7 @@ jobs:
230230
if: startsWith(matrix.os, 'ubuntu') && !endsWith(matrix.os, 'arm') && !startsWith(matrix.msan, '--with-') && !startsWith(matrix.nBPF, 'nBPF') && !startsWith(matrix.global_context, '--without') # Only on a few "standard" builds
231231
run: |
232232
make distclean
233-
./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --host=x86_64-w64-mingw32
233+
./autogen.sh && ./configure --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --host=x86_64-w64-mingw32
234234
make -j $(nproc) all
235235
env:
236236
CC:
@@ -263,7 +263,7 @@ jobs:
263263
sudo apt-get install autoconf automake debhelper libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev rrdtool librrd-dev parallel
264264
- name: Configure nDPI
265265
run: |
266-
AR=${{ matrix.ar }} RANLIB=${{ matrix.ranlib }} ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --with-sanitizer --with-lto-and-gold-linker
266+
./autogen.sh && AR=${{ matrix.ar }} RANLIB=${{ matrix.ranlib }} ./configure --enable-option-checking=fatal --enable-debug-messages --with-sanitizer --with-lto-and-gold-linker
267267
- name: Build nDPI
268268
run: |
269269
make -j $(nproc) all
@@ -294,7 +294,7 @@ jobs:
294294
install: git mingw-w64-x86_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86_64-json-c mingw-w64-x86_64-crt-git mingw-w64-x86_64-pcre mingw-w64-x86_64-libpcap parallel
295295
- name: Configure nDPI on Windows msys2
296296
run: |
297-
msys2 -c './autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap'
297+
msys2 -c './autogen.sh && ./configure --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap'
298298
- name: Build nDPI on Windows msys2
299299
run: |
300300
msys2 -c 'make -j all'
@@ -306,3 +306,54 @@ jobs:
306306
msys2 -c './tests/do.sh'
307307
msys2 -c './tests/do-unit.sh'
308308
msys2 -c './tests/do-dga.sh'
309+
310+
test-out-of-tree-builds:
311+
name: Out-of-tree builds
312+
runs-on: ubuntu-latest
313+
env:
314+
CFLAGS: -Wextra -Werror -DNDPI_EXTENDED_SANITY_CHECKS
315+
steps:
316+
- uses: actions/checkout@v4
317+
with:
318+
fetch-depth: 0 # Shallow clones should be disabled for a better relevancy of analysis
319+
- name: Install Ubuntu Prerequisites
320+
run: |
321+
sudo apt-get update
322+
sudo apt-get install autoconf automake debhelper libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev rrdtool librrd-dev parallel
323+
- name: Autogen nDPI
324+
run: |
325+
./autogen.sh
326+
- name: Configure and build nDPI (standard)
327+
run: |
328+
mkdir -p build-standard
329+
cd build-standard
330+
../configure
331+
make -j $(nproc) all
332+
make -j $(nproc) -C example ndpiSimpleIntegration
333+
make -j $(nproc) -C rrdtool
334+
cd -
335+
- name: Configure and build nDPI (fuzzing)
336+
run: |
337+
mkdir -p build-fuzzing
338+
cd build-fuzzing
339+
CC=clang CXX=clang++ ../configure --with-sanitizer --enable-fuzztargets
340+
make -j $(nproc) all
341+
make -j $(nproc) -C example ndpiSimpleIntegration
342+
make -j $(nproc) -C rrdtool
343+
cd -
344+
- name: Print nDPI long help
345+
run: |
346+
cd ./build-standard/example && ./ndpiReader -H && cd -
347+
cd ./build-fuzzing/example && ./ndpiReader -H && cd -
348+
- name: Tests #We are only intersting into tests scripts being valid --> less tests than usual
349+
run: |
350+
cd build-standard
351+
NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 NDPI_TEST_ONLY_RECENTLY_UPDATED_PCAPS=1 ./tests/do.sh
352+
./tests/do-unit.sh
353+
./tests/do-dga.sh
354+
cd -
355+
cd build-fuzzing
356+
MAX_TOTAL_TIME=10 NDPI_FORCE_PARALLEL_UTESTS=1 NDPI_SKIP_PARALLEL_BAR=1 NDPI_TEST_ONLY_RECENTLY_UPDATED_PCAPS=1 ./tests/do.sh
357+
./tests/do-unit.sh
358+
./tests/do-dga.sh
359+
cd -

‎.github/workflows/build_docker.yml‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ jobs:
5050
"apt-get -y update &&
5151
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev rrdtool librrd-dev &&
5252
git config --global --add safe.directory $(realpath .) &&
53-
env CC=gcc ./autogen.sh --enable-option-checking=fatal --enable-debug-messages &&
53+
env CC=gcc ./autogen.sh && ./configure --enable-option-checking=fatal --enable-debug-messages &&
5454
make -j $(nproc) all &&
5555
make -C example ndpiSimpleIntegration &&
5656
make -C rrdtool &&
@@ -74,7 +74,7 @@ jobs:
7474
"apt-get -y update &&
7575
apt-get -y install git wdiff colordiff autoconf automake libtool pkg-config gettext libjson-c-dev flex bison libpcap-dev rrdtool librrd-dev &&
7676
git config --global --add safe.directory $(realpath .) &&
77-
env CC=gcc ./autogen.sh --enable-option-checking=fatal --enable-debug-messages &&
77+
env CC=gcc ./autogen.sh && ./configure --enable-option-checking=fatal --enable-debug-messages &&
7878
make -j $(nproc) all &&
7979
make -C example ndpiSimpleIntegration &&
8080
make -C rrdtool &&

‎.github/workflows/build_masan.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ jobs:
3333
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libjson-c-dev libpcap-dev rrdtool librrd-dev parallel libpcre3-dev libmaxminddb-dev
3434
- name: Configure nDPI
3535
run: |
36-
./autogen.sh --enable-option-checking=fatal --with-pcre2 --with-maxminddb --with-memory-sanitizer --disable-memory-track-origins
36+
./autogen.sh && ./configure --enable-option-checking=fatal --with-pcre2 --with-maxminddb --with-memory-sanitizer --disable-memory-track-origins
3737
- name: Build nDPI
3838
run: |
3939
make -j $(nproc) all

‎.github/workflows/build_scheduled.yml‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ jobs:
2323
sudo apt-get install libpcre3-dev libmaxminddb-dev lcov
2424
sudo apt-get install wdiff colordiff
2525
- name: Configure
26-
run: ./autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre2 --with-maxminddb --enable-tls-sigs
26+
run: ./autogen.sh && ./configure --enable-option-checking=fatal --enable-debug-messages --enable-code-coverage --with-pcre2 --with-maxminddb --enable-tls-sigs
2727
- name: Build
2828
run: make all
2929
- name: Test
@@ -55,7 +55,7 @@ jobs:
5555
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison doxygen
5656
- name: Configure nDPI library
5757
run: |
58-
./autogen.sh --with-only-libndpi --enable-option-checking=fatal --enable-debug-build
58+
./autogen.sh && ./configure --with-only-libndpi --enable-option-checking=fatal --enable-debug-build
5959
- name: Generate Documentation
6060
run: |
6161
pip install --upgrade pip
@@ -89,7 +89,7 @@ jobs:
8989
pprof -h
9090
- name: Configure nDPI library
9191
run: |
92-
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs
92+
./autogen.sh && ./configure --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs
9393
- name: Build nDPI library
9494
run: |
9595
make -j
@@ -123,7 +123,7 @@ jobs:
123123
sudo apt-get install autoconf automake libtool pkg-config gettext flex bison libjson-c-dev libpcap-dev rrdtool librrd-dev parallel
124124
- name: Configure nDPI
125125
run: |
126-
./autogen.sh --enable-option-checking=fatal --with-thread-sanitizer
126+
./autogen.sh && ./configure --enable-option-checking=fatal --with-thread-sanitizer
127127
- name: Build nDPI
128128
run: |
129129
make -j $(nproc) all
@@ -159,7 +159,7 @@ jobs:
159159
brew install coreutils wdiff colordiff autoconf automake libtool gettext json-c rrdtool parallel libgcrypt
160160
- name: Configure nDPI
161161
run: |
162-
./autogen.sh --enable-option-checking=fatal --with-local-libgcrypt
162+
./autogen.sh && ./configure --enable-option-checking=fatal --with-local-libgcrypt
163163
- name: Build nDPI
164164
run: |
165165
make -j $(nproc) all
@@ -192,7 +192,7 @@ jobs:
192192
install: git mingw-w64-x86_64-toolchain automake1.16 automake-wrapper autoconf libtool make mingw-w64-x86_64-json-c mingw-w64-x86_64-crt-git mingw-w64-x86_64-pcre mingw-w64-x86_64-libpcap mingw-w64-x86_64-libgcrypt parallel
193193
- name: Configure nDPI on Windows msys2
194194
run: |
195-
msys2 -c './autogen.sh --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap --with-local-libgcrypt'
195+
msys2 -c './autogen.sh && ./configure --enable-option-checking=fatal --enable-debug-messages --enable-tls-sigs --disable-npcap --with-local-libgcrypt'
196196
- name: Build nDPI on Windows msys2
197197
run: |
198198
msys2 -c 'make -j all'

‎.github/workflows/codeql.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ jobs:
4040
pprof -h
4141
- name: Configure nDPI library
4242
run: |
43-
./autogen.sh --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs --enable-debug-messages
43+
./autogen.sh && ./configure --enable-gprof --enable-option-checking=fatal --with-pcre2 --with-maxminddb --enable-tls-sigs --enable-debug-messages
4444
4545
- name: Initialize CodeQL
4646
uses: github/codeql-action/init@v3

‎Makefile.am‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,10 +53,10 @@ clean-coverage:
5353
find . -type f -name "*.gcno" -exec rm "{}" +
5454

5555
pull:
56-
git pull --rebase
57-
if test -d ../nDPI-custom; then echo "Pulling ../nDPI-custom"; cd ../nDPI-custom; git pull --rebase; cd ../nDPI; fi
56+
cd $(top_srcdir) && git pull --rebase
57+
if test -d $(top_srcdir)/../nDPI-custom; then echo "Pulling ../nDPI-custom"; cd $(top_srcdir)/../nDPI-custom && git pull --rebase; fi
5858

5959
changelog:
60-
git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
60+
cd $(top_srcdir) && git log --since={`curl -s https://github.com/ntop/ndpi/releases | grep datetime | head -n1 | egrep -o "[0-9]+\-[0-9]+\-[0-9]+"`} --name-only --pretty=format:" - %s" | grep "^ " > CHANGELOG.latest
6161

6262
.PHONY: doc doc-view coverage clean-coverage changelog

‎README.fuzzer.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@
77
- export CXX=/usr/bin/clang++-18
88
- export LD=/usr/bin/ld.lld-18
99
- Run autogen.sh
10-
- ./autogen.sh --with-sanitizer --enable-fuzztargets
10+
- ./autogen.sh && ./configure --with-sanitizer --enable-fuzztargets
1111

1212

1313
## Testing nDPI with ClusterFuzz Artifacts

‎README.md‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -13,19 +13,25 @@ A generic FAQ about nDPI is available [here](https://github.com/ntop/nDPI/blob/d
1313

1414
In order to compile this project do
1515

16-
- ./autogen.sh
16+
- ./autogen.sh && ./configure
1717
- make
1818

1919
If you get some errors while compiling `croaring.c`, try:
2020

21-
- ./autogen.sh --enable-old-croaring
21+
- ./autogen.sh && ./configure --enable-old-croaring
2222
- make
2323

2424
To compile the library w/o any tools or tests:
2525

26-
- ./autogen.sh --with-only-libndpi
26+
- ./autogen.sh && ./configure --with-only-libndpi
2727
- make
2828

29+
Out-of-tree builds are supported:
30+
- ./autogen.sh
31+
- mkdir build
32+
- cd build
33+
- ../configure && make
34+
2935
To run tests do additionally:
3036

3137
- ./tests/do.sh # Generate and check for diff's in PCAP files
@@ -85,7 +91,7 @@ The entire procedure of adding new protocols in detail:
8591
5. Choose (do not change anything) a selection bitmask from: `src/include/ndpi_define.h`
8692
6. Set protocol default ports in `ndpi_init_protocol_defaults` in: `src/lib/ndpi_main.c`
8793
7. Be sure to have nBPF support, cloning `PF_RING` in the same directory where you cloned `nDPI`: `git clone https://github.com/ntop/PF_RING/ && cd PF_RING/userland/nbpf && ./configure && make`. You can ignore the `/bin/sh: 1: ../lib/pfring_config: not found` error
88-
8. From the `nDPI` root directory, `./autogen.sh --with-pcre2` (nBPF and PCRE2 are usually optional, but they are needed to run/update *all* the unit tests)
94+
8. From the `nDPI` root directory, `./autogen.sh && ./configure --with-pcre2` (nBPF and PCRE2 are usually optional, but they are needed to run/update *all* the unit tests)
8995
9. `make`
9096
10. `make check`
9197
11. Update the documentation, adding this new protocol to `doc/protocols.rst`

‎autogen.sh‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,6 @@ fi
4242

4343
autoreconf -ivf
4444

45-
echo "./configure $*"
46-
chmod +x configure
47-
./configure "$@"
45+
#####
46+
# Don't call `configure` here!!!! It breaks out-of-tree builds
47+
#####

0 commit comments

Comments
 (0)