bug: Always 0 vulns, Nuclei not work #1480

Flv-cmd · 2025-01-30T09:08:42Z

Is there an existing issue for this?

I have searched the existing issues

Current Behavior

All scans I run always show 0 vulnerabilities. The scans, with different configurations, sometimes finish quickly in a few seconds, while other times they take a couple of minutes but still find 0 vulnerabilities. The Nuclei module is not working properly. It just doesn't see the targets, or I don't know...

Expected Behavior

Nuclei should detect vulnerabilities based on the selected templates and target configurations. The scan should return valid results instead of always showing 0 vulnerabilities.

Steps To Reproduce

1.Run Nuclei with different target configurations and templates.
2.Observe that the scan sometimes finishes in a few seconds, while other times it takes a couple of minutes.
3.Regardless of the configuration, the scan always returns 0 vulnerabilities.
4.The issue persists across multiple scans and target variations.

Environment

- reNgine: 2.2.0
- Nuclei: 3.3.9
- OS: Kali Linux
- Python: last
- Docker Engine: last
- Docker Compose: last

Anything else?

No response

github-actions · 2025-01-30T09:08:54Z

Hey @Flv-cmd! 👋 Thanks for flagging this bug! 🐛🔍

You're our superhero bug hunter! 🦸‍♂️🦸‍♀️ Before we suit up to squash this bug, could you please:

📚 Double-check our documentation: https://rengine.wiki
🕵️ Make sure it's not a known issue
📝 Provide all the juicy details about this sneaky bug

Once again - thanks for your vigilance! 🛠️🚀

dmchaledev · 2025-02-19T16:08:18Z

Could you please share the output of docker-compose logs --tail=300 after finishing a scan so we can see if there are any clues to DB issues? I've seen this before when Nuclei output changes and the data needs to be normalized again for saving in DB, or when parameters to initiate Nuclei change and the scans aren't completing successfully.

Flv-cmd · 2025-02-20T14:36:58Z

Could you please share the output of docker-compose logs --tail=300 after finishing a scan so we can see if there are any clues to DB issues? I've seen this before when Nuclei output changes and the data needs to be normalized again for saving in DB, or when parameters to initiate Nuclei change and the scans aren't completing successfully.

Okay, there are different errors in different scans.

In the first case, I got "Error: KeyError('template-url')" and the scan finished with 0 vulnerabilities found.
#1476

In the second case, everything completed successfully, but just like in this issue:
#1487
the dashboard visually showed 0 vulnerabilities, even though up to 58 were actually found (14 high, 42 medium, 2 low).

I don't even know which case logs to show. Previously, over the past month, I’ve deployed and deleted Rengine about 5 times, each time trying to set it up without errors. By strictly following the installation instructions, you will inevitably get a 100% error during scanning. Something will not work. This project needs serious improvement.

rengine-db-1 | The files belonging to this database system will be owned by user "postgres".
rengine-db-1 | This user must also own the server process.
rengine-db-1 |
rengine-db-1 | The database cluster will be initialized with locale "en_US.utf8".
rengine-db-1 | The default database encoding has accordingly been set to "UTF8".
rengine-db-1 | The default text search configuration will be set to "english".
rengine-db-1 |
rengine-db-1 | Data page checksums are disabled.
rengine-db-1 |
rengine-db-1 | fixing permissions on existing directory /var/lib/postgresql/data ... ok
rengine-db-1 | creating subdirectories ... ok
rengine-db-1 | selecting dynamic shared memory implementation ... posix
rengine-db-1 | selecting default max_connections ... 100
rengine-db-1 | selecting default shared_buffers ... 128MB
rengine-db-1 | selecting default time zone ... UTC
rengine-db-1 | creating configuration files ... ok
rengine-db-1 | running bootstrap script ... ok
rengine-db-1 | sh: locale: not found
rengine-db-1 | 2025-02-20 12:52:51.498 UTC [30] WARNING: no usable system locales were found
rengine-db-1 | performing post-bootstrap initialization ... ok
rengine-db-1 | initdb: warning: enabling "trust" authentication for local connections
rengine-db-1 | You can change this by editing pg_hba.conf or using the option -A, or
rengine-db-1 | --auth-local and --auth-host, the next time you run initdb.
rengine-db-1 | syncing data to disk ... ok
rengine-db-1 |
rengine-db-1 |
rengine-db-1 | Success. You can now start the database server using:
rengine-db-1 |
rengine-db-1 | pg_ctl -D /var/lib/postgresql/data -l logfile start
rengine-db-1 |
rengine-db-1 | waiting for server to start....2025-02-20 12:52:52.254 UTC [35] LOG: starting PostgreSQL 12.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
rengine-db-1 | 2025-02-20 12:52:52.262 UTC [35] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
rengine-db-1 | 2025-02-20 12:52:52.299 UTC [36] LOG: database system was shut down at 2025-02-20 12:52:51 UTC
rengine-db-1 | 2025-02-20 12:52:52.309 UTC [35] LOG: database system is ready to accept connections
rengine-db-1 | done
rengine-db-1 | server started
rengine-db-1 | CREATE DATABASE
rengine-db-1 |
rengine-db-1 |
rengine-db-1 | /usr/local/bin/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
rengine-db-1 |
rengine-db-1 | waiting for server to shut down....2025-02-20 12:52:52.456 UTC [35] LOG: received fast shutdown request
rengine-db-1 | 2025-02-20 12:52:52.458 UTC [35] LOG: aborting any active transactions
rengine-db-1 | 2025-02-20 12:52:52.460 UTC [35] LOG: background worker "logical replication launcher" (PID 42) exited with exit code 1
rengine-db-1 | 2025-02-20 12:52:52.461 UTC [37] LOG: shutting down
rengine-db-1 | 2025-02-20 12:52:52.490 UTC [35] LOG: database system is shut down
rengine-db-1 | done
rengine-db-1 | server stopped
rengine-db-1 |
rengine-db-1 | PostgreSQL init process complete; ready for start up.
rengine-db-1 |
rengine-db-1 | 2025-02-20 12:52:52.570 UTC [1] LOG: starting PostgreSQL 12.3 on x86_64-pc-linux-musl, compiled by gcc (Alpine 9.3.0) 9.3.0, 64-bit
rengine-db-1 | 2025-02-20 12:52:52.570 UTC [1] LOG: listening on IPv4 address "0.0.0.0", port 5432
rengine-db-1 | 2025-02-20 12:52:52.570 UTC [1] LOG: listening on IPv6 address "::", port 5432
rengine-db-1 | 2025-02-20 12:52:52.575 UTC [1] LOG: listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
rengine-db-1 | 2025-02-20 12:52:52.604 UTC [46] LOG: database system was shut down at 2025-02-20 12:52:52 UTC
rengine-db-1 | 2025-02-20 12:52:52.611 UTC [1] LOG: database system is ready to accept connections
rengine-db-1 | 2025-02-20 12:52:55.240 UTC [57] ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
rengine-db-1 | 2025-02-20 12:52:55.240 UTC [57] DETAIL: Key (typname, typnamespace)=(django_admin_log_id_seq, 2200) already exists.
rengine-db-1 | 2025-02-20 12:52:55.240 UTC [57] STATEMENT: CREATE TABLE "django_admin_log" ("id" serial NOT NULL PRIMARY KEY, "action_time" timestamp with time zone NOT NULL, "object_id" text NULL, "object_repr" varchar(200) NOT NULL, "action_flag" smallint NOT NULL CHECK ("action_flag" >= 0), "change_message" text NOT NULL, "content_type_id" integer NULL, "user_id" integer NOT NULL)
rengine-db-1 | 2025-02-20 12:52:55.560 UTC [58] ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
rengine-db-1 | 2025-02-20 12:52:55.560 UTC [58] DETAIL: Key (typname, typnamespace)=(dashboard_netlasapikey_id_seq, 2200) already exists.
rengine-db-1 | 2025-02-20 12:52:55.560 UTC [58] STATEMENT: CREATE TABLE "dashboard_netlasapikey" ("id" serial NOT NULL PRIMARY KEY, "key" varchar(500) NOT NULL)
rengine-celery-1 | run_command | INFO | rm /usr/src/scan_results/testphp.vulnweb.com_3/httpx_input.txt

rengine-celery-beat-1 | pikepdf._core | pikepdf C++ to Python logger bridge initialized
rengine-celery-beat-1 | System check identified some issues:
rengine-celery-beat-1 |
rengine-celery-beat-1 | WARNINGS:
rengine-celery-beat-1 | startScan.EndPoint.techs: (fields.W340) null has no effect on ManyToManyField.
rengine-celery-beat-1 | Operations to perform:
rengine-celery-beat-1 | Apply all migrations: admin, auth, contenttypes, dashboard, django_celery_beat, recon_note, scanEngine, sessions, startScan, targetApp
rengine-celery-beat-1 | Running migrations:
rengine-celery-beat-1 | Traceback (most recent call last):
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 82, in _execute
rengine-celery-beat-1 | return self.cursor.execute(sql)
rengine-celery-beat-1 | psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
rengine-celery-beat-1 | DETAIL: Key (typname, typnamespace)=(django_admin_log_id_seq, 2200) already exists.
rengine-celery-beat-1 |
rengine-celery-beat-1 |
rengine-celery-beat-1 | The above exception was the direct cause of the following exception:
rengine-celery-beat-1 |
rengine-celery-beat-1 | Traceback (most recent call last):
rengine-celery-beat-1 | File "/usr/src/app/manage.py", line 26, in
rengine-celery-beat-1 | main()
rengine-celery-beat-1 | File "/usr/src/app/manage.py", line 22, in main
rengine-celery-beat-1 | execute_from_command_line(sys.argv)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/init.py", line 419, in execute_from_command_line
rengine-celery-beat-1 | utility.execute()
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/init.py", line 413, in execute
rengine-celery-beat-1 | self.fetch_command(subcommand).run_from_argv(self.argv)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 354, in run_from_argv
rengine-celery-beat-1 | self.execute(*args, **cmd_options)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 398, in execute
rengine-celery-beat-1 | output = self.handle(*args, **options)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 89, in wrapped
rengine-celery-beat-1 | res = handle_func(*args, **kwargs)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/migrate.py", line 244, in handle
rengine-celery-beat-1 | post_migrate_state = executor.migrate(
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 117, in migrate
rengine-celery-beat-1 | state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
rengine-celery-beat-1 | state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 227, in apply_migration
rengine-celery-beat-1 | state = migration.apply(state, schema_editor)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/migration.py", line 126, in apply
rengine-celery-beat-1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/operations/models.py", line 92, in database_forwards
rengine-celery-beat-1 | schema_editor.create_model(model)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/schema.py", line 345, in create_model
rengine-celery-beat-1 | self.execute(sql, params or None)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/schema.py", line 145, in execute
rengine-celery-beat-1 | cursor.execute(sql, params)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
rengine-celery-beat-1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
rengine-celery-beat-1 | return executor(sql, params, many, context)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
rengine-celery-beat-1 | with self.db.wrap_database_errors:
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in exit
rengine-celery-beat-1 | raise dj_exc_value.with_traceback(traceback) from exc_value
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 82, in _execute
rengine-celery-beat-1 | return self.cursor.execute(sql)
rengine-celery-beat-1 | django.db.utils.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
rengine-celery-beat-1 | DETAIL: Key (typname, typnamespace)=(django_admin_log_id_seq, 2200) already exists.
rengine-celery-beat-1 |
rengine-celery-beat-1 | Applying admin.0001_initial...celery beat v5.4.0 (opalescent) is starting.
rengine-celery-beat-1 | pikepdf._core | pikepdf C++ to Python logger bridge initialized
rengine-celery-beat-1 | __ - ... __ - _
rengine-celery-beat-1 | LocalTime -> 2025-02-20 12:52:59
rengine-celery-beat-1 | Configuration ->
rengine-celery-beat-1 | . broker -> redis://redis:6379/0
rengine-celery-beat-1 | . loader -> celery.loaders.app.AppLoader
rengine-celery-beat-1 | . scheduler -> django_celery_beat.schedulers.DatabaseScheduler
rengine-celery-beat-1 |
rengine-celery-beat-1 | . logfile -> [stderr]@%INFO
rengine-celery-beat-1 | . maxinterval -> 5.00 seconds (5s)
rengine-celery-1 | nuclei_individual_severity_module | WARNING | http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-1/3/wp-content/plugins/flexible-custom-post-type/edit-post.php?id=%3C%2Fscript%3E%3Cscript%3Ealert%28document.domain%29%3C%2Fscript%3E | MEDIUM | WordPress Plugin Flexible Custom Post Type < 0.1.7 - Cross-Site Scripting | rengine-celery-1 | nuclei_individual_severity_module | WARNING | http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-1/3L/wp-content/plugins/flexible-custom-post-type/edit-post.php?id=%3C%2Fscript%3E%3Cscript%3Ealert%28document.domain%29%3C%2Fscript%3E | MEDIUM | WordPress Plugin Flexible Custom Post Type < 0.1.7 - Cross-Site Scripting |
rengine-celery-1 | nuclei_individual_severity_module | WARNING | http://testphp.vulnweb.com/Mod_Rewrite_Shop/BuyProduct-2/3L/wp-content/plugins/flexible-custom-post-type/edit-post.php?id=%3C%2Fscript%3E%3Cscript%3Ealert%28document.domain%29%3C%2Fscript%3E | MEDIUM | WordPress Plugin Flexible Custom Post Type < 0.1.7 - Cross-Site Scripting | rengine-celery-1 | nuclei_individual_severity_module | WARNING | http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/web-camera-a4tech/2/wp-content/plugins/flexible-custom-post-type/edit-post.php?id=%3C%2Fscript%3E%3Cscript%3Ealert%28document.domain%29%3C%2Fscript%3E | MEDIUM | WordPress Plugin Flexible Custom Post Type < 0.1.7 - Cross-Site Scripting |
rengine-celery-1 | nuclei_individual_severity_module | WARNING | http://testphp.vulnweb.com/Mod_Rewrite_Shop/Details/network-attached-storage-dlink/1/Connection:/wp-content/plugins/flexible-custom-post-type/edit-post.php?id=%3C%2Fscript%3E%3Cscript%3Ealert%28document.domain%29%3C%2Fscript%3E | MEDIUM | WordPress Plugin Flexible Custom Post Type < 0.1.7 - Cross-Site Scripting | `
rengine-celery-1 | nuclei_scan | INFO | Vulnerability scan with all severities completed...
rengine-celery-1 | vulnerability_scan | INFO | Vulnerability scan completed...
rengine-celery-1 | send_scan_notif | WARNING | Sending notification "Scan #3 summary" [success]

Interesting

ERROR: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL: Key (typname, typnamespace)=(django_admin_log_id_seq, 2200) already exists.

In a cleanly built project, there should be no errors or issues.

But this project didn't work with Nuclei v3.3.7, v3.3.8, or v3.3.9.

Flv-cmd · 2025-02-20T15:01:27Z

And other scan log with "Error: KeyError('template-url')"

rengine-celery-1 | http://testhtml5.vulnweb.com/ajax/latest?offset=15 [200] [content_length=2]
rengine-celery-1 | run_command | INFO | rm /usr/src/scan_results/testhtml5.vulnweb.com_4/httpx_input.txt
rengine-celery-1 | run_command | WARNING | None
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "debug_logic"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf debug_logic | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_debug_logic.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "idor"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf idor | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_idor.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "interestingEXT"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf interestingEXT | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_interestingEXT.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "interestingparams"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf interestingparams | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_interestingparams.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "interestingsubs"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf interestingsubs | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_interestingsubs.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "lfi"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf lfi | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_lfi.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "rce"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf rce | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_rce.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "redirect"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf redirect | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_redirect.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "sqli"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf sqli | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_sqli.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "ssrf"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf ssrf | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_ssrf.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "ssti"
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf ssti | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_ssti.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | fetch_url | WARNING | Running gf on pattern "xss"
rengine-celery-beat-1 | _ _ _
rengine-celery-beat-1 | | \ | | ()
rengine-celery-beat-1 | _ __ | | | __ _ _ _ __ ___
rengine-celery-beat-1 | | '/ _ \ . |/ _ | | ' \ / _
rengine-celery-beat-1 | | | | __/ |\ | (| | | | | | /
rengine-celery-beat-1 | |_| _|| _|_, ||| |_|__|
rengine-celery-beat-1 | / |
rengine-celery-beat-1 | |/
rengine-celery-beat-1 |
rengine-celery-beat-1 | pikepdf._core | pikepdf C++ to Python logger bridge initialized
rengine-celery-beat-1 | System check identified some issues:
rengine-celery-beat-1 |
rengine-celery-beat-1 | WARNINGS:
rengine-celery-beat-1 | startScan.EndPoint.techs: (fields.W340) null has no effect on ManyToManyField.
rengine-celery-beat-1 | Operations to perform:
rengine-celery-beat-1 | Apply all migrations: admin, auth, contenttypes, dashboard, django_celery_beat, recon_note, scanEngine, sessions, startScan, targetApp
rengine-celery-beat-1 | Running migrations:
rengine-celery-beat-1 | Traceback (most recent call last):
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 82, in _execute
rengine-celery-beat-1 | return self.cursor.execute(sql)
rengine-celery-beat-1 | psycopg2.errors.UniqueViolation: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
rengine-celery-beat-1 | DETAIL: Key (typname, typnamespace)=(django_admin_log_id_seq, 2200) already exists.
rengine-celery-beat-1 |
rengine-celery-beat-1 |
rengine-celery-beat-1 | The above exception was the direct cause of the following exception:
rengine-celery-beat-1 |
rengine-celery-beat-1 | Traceback (most recent call last):
rengine-celery-beat-1 | File "/usr/src/app/manage.py", line 26, in
rengine-celery-beat-1 | main()
rengine-celery-beat-1 | File "/usr/src/app/manage.py", line 22, in main
rengine-celery-beat-1 | execute_from_command_line(sys.argv)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/init.py", line 419, in execute_from_command_line
rengine-celery-beat-1 | utility.execute()
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/init.py", line 413, in execute
rengine-celery-beat-1 | self.fetch_command(subcommand).run_from_argv(self.argv)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 354, in run_from_argv
rengine-celery-beat-1 | self.execute(*args, **cmd_options)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 398, in execute
rengine-celery-beat-1 | output = self.handle(*args, **options)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/base.py", line 89, in wrapped
rengine-celery-beat-1 | res = handle_func(*args, **kwargs)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/core/management/commands/migrate.py", line 244, in handle
rengine-celery-beat-1 | post_migrate_state = executor.migrate(
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 117, in migrate
rengine-celery-beat-1 | state = self._migrate_all_forwards(state, plan, full_plan, fake=fake, fake_initial=fake_initial)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 147, in _migrate_all_forwards
rengine-celery-beat-1 | state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/executor.py", line 227, in apply_migration
rengine-celery-beat-1 | state = migration.apply(state, schema_editor)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/migration.py", line 126, in apply
rengine-celery-beat-1 | operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/migrations/operations/models.py", line 92, in database_forwards
rengine-celery-beat-1 | schema_editor.create_model(model)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/schema.py", line 345, in create_model
rengine-celery-beat-1 | self.execute(sql, params or None)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/base/schema.py", line 145, in execute
rengine-celery-beat-1 | cursor.execute(sql, params)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 66, in execute
rengine-celery-beat-1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
rengine-celery-beat-1 | return executor(sql, params, many, context)
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 79, in _execute
rengine-celery-beat-1 | with self.db.wrap_database_errors:
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/utils.py", line 90, in exit
rengine-celery-beat-1 | raise dj_exc_value.with_traceback(traceback) from exc_value
rengine-celery-beat-1 | File "/usr/local/lib/python3.10/dist-packages/django/db/backends/utils.py", line 82, in _execute
rengine-celery-beat-1 | return self.cursor.execute(sql)
rengine-celery-beat-1 | django.db.utils.IntegrityError: duplicate key value violates unique constraint "pg_type_typname_nsp_index"
rengine-celery-beat-1 | DETAIL: Key (typname, typnamespace)=(django_admin_log_id_seq, 2200) already exists.
rengine-celery-beat-1 |
rengine-celery-beat-1 | Applying admin.0001_initial...celery beat v5.4.0 (opalescent) is starting.
rengine-celery-beat-1 | pikepdf._core | pikepdf C++ to Python logger bridge initialized
rengine-celery-beat-1 | __ - ... __ - _
rengine-celery-beat-1 | LocalTime -> 2025-02-20 12:52:59
rengine-celery-beat-1 | Configuration ->
rengine-celery-beat-1 | . broker -> redis://redis:6379/0
rengine-celery-beat-1 | . loader -> celery.loaders.app.AppLoader
rengine-celery-beat-1 | . scheduler -> django_celery_beat.schedulers.DatabaseScheduler
rengine-celery-beat-1 |
rengine-celery-beat-1 | . logfile -> [stderr]@%INFO
rengine-celery-beat-1 | . maxinterval -> 5.00 seconds (5s)
rengine-redis-1 | 1:C 20 Feb 2025 12:52:50.937 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see jemalloc/jemalloc#1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect.
rengine-redis-1 | 1:C 20 Feb 2025 12:52:50.937 * oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
rengine-redis-1 | 1:C 20 Feb 2025 12:52:50.937 * Redis version=7.4.2, bits=64, commit=00000000, modified=0, pid=1, just started
rengine-redis-1 | 1:C 20 Feb 2025 12:52:50.937 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
rengine-redis-1 | 1:M 20 Feb 2025 12:52:50.938 * monotonic clock: POSIX clock_gettime
rengine-redis-1 | 1:M 20 Feb 2025 12:52:50.939 * Running mode=standalone, port=6379.
rengine-redis-1 | 1:M 20 Feb 2025 12:52:50.939 * Server initialized
rengine-redis-1 | 1:M 20 Feb 2025 12:52:50.939 * Ready to accept connections tcp
rengine-redis-1 | 1:M 20 Feb 2025 12:57:51.015 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 12:57:51.016 * Background saving started by pid 21
rengine-redis-1 | 21:C 20 Feb 2025 12:57:51.025 * DB saved on disk
rengine-redis-1 | 21:C 20 Feb 2025 12:57:51.026 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 12:57:51.117 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:08:33.814 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:08:33.815 * Background saving started by pid 22
rengine-redis-1 | 22:C 20 Feb 2025 13:08:33.829 * DB saved on disk
rengine-redis-1 | 22:C 20 Feb 2025 13:08:33.830 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:08:33.915 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:13:34.059 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:13:34.060 * Background saving started by pid 23
rengine-redis-1 | 23:C 20 Feb 2025 13:13:34.076 * DB saved on disk
rengine-redis-1 | 23:C 20 Feb 2025 13:13:34.077 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:13:34.161 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:30:54.110 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:30:54.111 * Background saving started by pid 24
rengine-redis-1 | 24:C 20 Feb 2025 13:30:54.124 * DB saved on disk
rengine-redis-1 | 24:C 20 Feb 2025 13:30:54.124 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:30:54.213 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:42:11.883 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:42:11.884 * Background saving started by pid 25
rengine-redis-1 | 25:C 20 Feb 2025 13:42:11.902 * DB saved on disk
rengine-redis-1 | 25:C 20 Feb 2025 13:42:11.903 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:42:11.985 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:47:12.008 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:47:12.009 * Background saving started by pid 26
rengine-redis-1 | 26:C 20 Feb 2025 13:47:12.026 * DB saved on disk
rengine-redis-1 | 26:C 20 Feb 2025 13:47:12.027 * Fork CoW for RDB: current 0 MB, peak 0 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:47:12.111 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:51:17.087 * 10000 changes in 60 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:51:17.088 * Background saving started by pid 27
rengine-redis-1 | 27:C 20 Feb 2025 13:51:17.144 * DB saved on disk
rengine-redis-1 | 27:C 20 Feb 2025 13:51:17.146 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:51:17.190 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:52:18.028 * 10000 changes in 60 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:52:18.029 * Background saving started by pid 28
rengine-redis-1 | 28:C 20 Feb 2025 13:52:18.073 * DB saved on disk
rengine-redis-1 | 28:C 20 Feb 2025 13:52:18.074 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:52:18.130 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:53:19.069 * 10000 changes in 60 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:53:19.070 * Background saving started by pid 29
rengine-redis-1 | 29:C 20 Feb 2025 13:53:19.117 * DB saved on disk
rengine-redis-1 | 29:C 20 Feb 2025 13:53:19.118 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:53:19.171 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 13:58:20.029 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 13:58:20.030 * Background saving started by pid 30
rengine-redis-1 | 30:C 20 Feb 2025 13:58:20.088 * DB saved on disk
rengine-redis-1 | 30:C 20 Feb 2025 13:58:20.089 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 13:58:20.131 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 14:32:24.996 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 14:32:24.997 * Background saving started by pid 31
rengine-redis-1 | 31:C 20 Feb 2025 14:32:25.035 * DB saved on disk
rengine-redis-1 | 31:C 20 Feb 2025 14:32:25.036 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
rengine-redis-1 | 1:M 20 Feb 2025 14:32:25.099 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 14:42:15.800 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 14:42:15.801 * Background saving started by pid 32
rengine-redis-1 | 32:C 20 Feb 2025 14:42:15.863 * DB saved on disk
rengine-redis-1 | 32:C 20 Feb 2025 14:42:15.865 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 1 MB
rengine-redis-1 | 1:M 20 Feb 2025 14:42:15.902 * Background saving terminated with success
rengine-redis-1 | 1:M 20 Feb 2025 14:47:16.073 * 100 changes in 300 seconds. Saving...
rengine-redis-1 | 1:M 20 Feb 2025 14:47:16.074 * Background saving started by pid 33
rengine-redis-1 | 33:C 20 Feb 2025 14:47:16.144 * DB saved on disk
rengine-redis-1 | 33:C 20 Feb 2025 14:47:16.145 * Fork CoW for RDB: current 1 MB, peak 1 MB, average 0 MB
rengine-redis-1 | 1:M 20 Feb 2025 14:47:16.176 * Background saving terminated with success
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_fetch_url.txt | gf xss | grep -Eo 'https?://([a-z0-9]+[.])testhtml5.vulnweb.com.' >> /usr/src/scan_results/testhtml5.vulnweb.com_4/gf_patterns_xss.txt
rengine-celery-1 | run_command | WARNING | 25
rengine-celery-1 | vulnerability_scan | INFO | Running Vulnerability Scan Queue
rengine-celery-1 | s3scanner | INFO | s3scanner -bucket-file /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_subdomain_discovery.txt -enumerate -provider gcp -threads 30 -json
rengine-celery-1 | run_command | INFO | cat /usr/src/scan_results/testhtml5.vulnweb.com_4/input_endpoints_vulnerability_scan.txt | unfurl -u format %s://%d%p |uro > /usr/src/scan_results/testhtml5.vulnweb.com_4/urls_unfurled.txt
rengine-celery-1 | run_command | WARNING | 28
rengine-celery-1 | s3scanner | INFO | s3scanner -bucket-file /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_subdomain_discovery.txt -enumerate -provider aws -threads 30 -json
rengine-celery-1 | s3scanner | INFO | s3scanner -bucket-file /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_subdomain_discovery.txt -enumerate -provider digitalocean -threads 30 -json
rengine-celery-1 | s3scanner | INFO | s3scanner -bucket-file /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_subdomain_discovery.txt -enumerate -provider dreamhost -threads 30 -json
rengine-celery-1 | s3scanner | INFO | s3scanner -bucket-file /usr/src/scan_results/testhtml5.vulnweb.com_4/#4_subdomain_discovery.txt -enumerate -provider linode -threads 30 -json
rengine-celery-1 | run_command | INFO | sort -u /usr/src/scan_results/testhtml5.vulnweb.com_4/urls_unfurled.txt -o /usr/src/scan_results/testhtml5.vulnweb.com_4/urls_unfurled.txt
rengine-celery-1 | run_command | WARNING | 28
rengine-celery-1 | run_command | INFO | nuclei -update-templates
rengine-celery-1 | run_command | WARNING | 28
rengine-celery-1 | nuclei_individual_severity_module | INFO | Running vulnerability scan with severity: low
rengine-celery-1 | nuclei_individual_severity_module | INFO | Running vulnerability scan with severity: medium
rengine-celery-1 | nuclei_individual_severity_module | INFO | nuclei -j -irr -l /usr/src/scan_results/testhtml5.vulnweb.com_4/urls_unfurled.txt -c 50 -retries 1 -rl 150 -timeout 5 -silent -t /root/nuclei-templates -severity low
rengine-celery-1 | nuclei_individual_severity_module | INFO | Running vulnerability scan with severity: high
rengine-celery-1 | nuclei_individual_severity_module | INFO | Running vulnerability scan with severity: critical
rengine-celery-1 | nuclei_individual_severity_module | INFO | nuclei -j -irr -l /usr/src/scan_results/testhtml5.vulnweb.com_4/urls_unfurled.txt -c 50 -retries 1 -rl 150 -timeout 5 -silent -t /root/nuclei-templates -severity high
rengine-celery-1 | nuclei_individual_severity_module | INFO | nuclei -j -irr -l /usr/src/scan_results/testhtml5.vulnweb.com_4/urls_unfurled.txt -c 50 -retries 1 -rl 150 -timeout 5 -silent -t /root/nuclei-templates -severity critical
rengine-celery-1 | nuclei_individual_severity_module | INFO | nuclei -j -irr -l /usr/src/scan_results/testhtml5.vulnweb.com_4/urls_unfurled.txt -c 50 -retries 1 -rl 150 -timeout 5 -silent -t /root/nuclei-templates -severity medium
rengine-celery-1 | nuclei_individual_severity_module | ERROR | 'template-url'
rengine-celery-1 | Traceback (most recent call last):
rengine-celery-1 | File "/usr/src/app/reNgine/celery_custom_task.py", line 130, in call
rengine-celery-1 | self.result = self.run(*args, **kwargs)
rengine-celery-1 | File "/usr/src/app/reNgine/tasks.py", line 2144, in nuclei_individual_severity_module
rengine-celery-1 | vuln_data = parse_nuclei_result(line)
rengine-celery-1 | File "/usr/src/app/reNgine/tasks.py", line 3674, in parse_nuclei_result
rengine-celery-1 | 'template_url': line['template-url'],
rengine-celery-1 | KeyError: 'template-url'
rengine-celery-1 | nuclei_scan | INFO | Vulnerability scan with all severities completed...
rengine-celery-1 | vulnerability_scan | INFO | Vulnerability scan completed...
rengine-celery-1 | send_scan_notif | WARNING | Sending notification "Scan #4 summary" [error]

dmchaledev · 2025-02-20T18:27:47Z

For #1476 , could you please try modifying the Nuclei configuration file? I think there was a breaking change in 3.3.6-3.3.9 that made template-url a required param, even if it's empty. This allowed Nuclei to get much further in test scans after I was able to recreate the template-url failures.

For the vulnerability rendering issue #1480, I identified a missing comma on line 968 of the /vuln page that would impact the vulnerability table loading properly. Can you hit the API in your instance to verify that the vulnerability data is in fact in the database for your instance and reaching the page in the expected format?
https://<your_rengine_ip>/api/listVulnerability/?project=default&format=datatables
ex. https://21.215.148.119/api/listVulnerability/?project=default&format=datatables

And yeah, there's definitely issues. Hopefully we can lay these both to rest and have an easy path to reviewing vulnerability results. Separately I'm working on improving the initial experience from setup to first scan to generating first report to help ease that frustration.

dmchaledev · 2025-02-21T05:27:29Z

Hi @Flv-cmd

This is part of a combination of three errors I have identified in testing today impacting vulnerabilities tab and this is resolved in my test branch:

Nuclei is seeing an error when template-url is absent from the default configuration as a more recent release of Nuclei made this a required param, so I have added an empty array for the definition, template-url: [], to the default configuration.
There is a missing comma between arguments in the javascript rendering the vulnerability_table on the vulnerabilities tab.
Last, there is a typo in the variable holding the fetched vulnerability data in the render function which was causing the table to never complete loading, it was encodedData and should have been encodedURLData.

I will put these changes together into a commit on a fresh install and make a PR to address these issues, targeting end of this weekend as I'm sick at the moment and moving slow.

Flv-cmd · 2025-02-21T12:43:05Z

For #1476 , could you please try modifying the Nuclei configuration file? I think there was a breaking change in 3.3.6-3.3.9 that made template-url a required param, even if it's empty. This allowed Nuclei to get much further in test scans after I was able to recreate the template-url failures.

For the vulnerability rendering issue #1480, I identified a missing comma on line 968 of the /vuln page that would impact the vulnerability table loading properly. Can you hit the API in your instance to verify that the vulnerability data is in fact in the database for your instance and reaching the page in the expected format? https://<your_rengine_ip>/api/listVulnerability/?project=default&format=datatables ex. https://21.215.148.119/api/listVulnerability/?project=default&format=datatables

And yeah, there's definitely issues. Hopefully we can lay these both to rest and have an easy path to reviewing vulnerability results. Separately I'm working on improving the initial experience from setup to first scan to generating first report to help ease that frustration.

@dmchaledev
Thank you for your response. I did everything you suggested, fixed all the errors, and initially, the API request indeed returned an empty database. There were issues with commas and other minor details.

After making changes to:

/usr/src/app/startScan/templates/startScan/vulnerabilities.html
/usr/src/app/startScan/templates/startScan/detail_scan.html
/usr/src/app/targetApp/templates/target/summary.html
I edited encodedData to encodedURLData and added commas after the brackets, like this:

"targets": 11,
},

Also, in tasks.py, I changed:

'template_url': line['template-url']
to:
'template_url': line.get('template-url', [])

After these changes, everything started working. I deliberately didn’t go into the technical details of my fixes because I simply followed all your suggestions. Now it works; at least, I tested two scans, and both showed vulnerabilities.

Another question is why it finds so few vulns, but I think that’s more about the scan configuration itself. Hmm.

Once again, I appreciate your attention to this issue, and I’m looking forward to the official fixes in the repository.

dmchaledev · 2025-02-23T03:04:34Z

Ooo, I love this approach for tasks.py in leiu of making the changes directly to the default yaml configuration, I'll include that as well as setting default configuration initially. Thank you for sharing your fix!

…k for template-url variable, fixing missing comma, fixing encodedURLData typo

dmchaledev · 2025-02-23T05:19:26Z

I've submitted a pull request addressing this issue here, using the tasks.py approach you took instead: #1488

#1476-#1480-#1487 Fix, setting fallback for Nuclei variable and fixing JS errors in Vulnerabilities page

wifiuk · 2025-02-28T13:56:26Z

I'm never getting nuclei to work.

always just see stuff like this:

`nuclei -j -irr -l /usr/src/scan_results/DOMAIN.com_672/urls_unfurled.txt -c 5 -retries 1 -rl 100 -timeout 5 -silent -t /root/nuclei-templates -severity info

null`

on any domain I scan

zerodaykb · 2025-03-06T13:09:53Z

Similarly for me - for some reason when I run rengine with nuclei on my VPS it shows 0 endpoints and I think due to that there are 0 vulns found. But when I run it locally - everything is ok. I'm using docker for both.
Config looks like this:

subdomain_discovery: {
  'uses_tools': ['subfinder', 'chaos', 'ctfr', 'sublist3r', 'tlsx', 'oneforall', 'netlas'],
  'enable_http_crawl': true,
  'threads': 30,
  'timeout': 5,
}
http_crawl: {}
osint: {
  'discover': [
      'emails',
      'metainfo'
    ],
  'dorks': [
    'login_pages',
    'admin_panels',
    'dashboard_pages',
    'config_files',
    'exposed_documents',
  ],
  'intensity': 'normal',
  'documents_limit': 50
}
vulnerability_scan: {
  'run_nuclei': true,
  'run_dalfox': true,
  'run_crlfuzz': true,
  'enable_http_crawl': false,
  'concurrency': 50,
  'intensity': 'normal',
  'rate_limit': 150,
  'retries': 1,
  'timeout': 5,
  'fetch_gpt_report': false,
  'nuclei': {
    'use_nuclei_config': false,
    'severities': ['low', 'medium', 'high', 'critical']
  }
}

I think there is some problem with httpx execution on my VPS. When I check docker logs httpx doesn't execute.
Do you know how to reset it? :)

VPS:

rengine-celery-1       | subdomain_discovery                | INFO | Initiating HTTP Crawl
rengine-celery-1       | subdomain_discovery                | INFO | Running From Subdomain Scan...
rengine-celery-1       | py.warnings | /usr/local/lib/python3.10/dist-packages/celery/canvas.py:2303: CPendingDeprecationWarning:
rengine-celery-1       |
rengine-celery-1       | task_allow_error_cb_on_chord_header=False is pending deprecation in a future release of Celery.
rengine-celery-1       | Please test the new behavior by setting task_allow_error_cb_on_chord_header to True and report any concerns you might have in our issue tracker before we make a final decision regarding how errbacks should behave when used with chords.
rengine-celery-1       |
rengine-celery-1       |
rengine-celery-1       | vulnerability_scan                 | INFO | Running Vulnerability Scan Queue

LOCAL:

rengine-celery-1       | subdomain_discovery                | INFO | Initiating HTTP Crawl
rengine-celery-1       | subdomain_discovery                | INFO | Running From Subdomain Scan...
rengine-celery-1       | subdomain_discovery                | INFO | /go/bin/httpx -cl -ct -rt -location -td -websocket -cname -asn -cdn -probe -random-agent -t 2 -json -l /usr/src/scan_results/REDACTED.COM_37/httpx_input.txt -silent -fr
rengine-celery-1       | subdomain_discovery                | WARNING | https://REDACTED.COM/some_endpoint [200] `2734B` `cloudflare` `593.683242ms`
rengine-celery-1       | subdomain_discovery                | WARNING | https://REDACTED.COM/some_endpoint [200] `2734B` `cloudflare` `813.211897ms`
rengine-celery-1       | remove_duplicate_endpoints         | INFO | Removing duplicate endpoints based on ['content_length', 'page_title']
rengine-celery-1       | run_command                        | INFO | rm /usr/src/scan_results/REDACTED.COM_37/httpx_input.txt
rengine-celery-1       | run_command                        | WARNING | 279
rengine-celery-1       | py.warnings | /usr/local/lib/python3.10/dist-packages/celery/canvas.py:2303: CPendingDeprecationWarning:
rengine-celery-1       |
rengine-celery-1       | task_allow_error_cb_on_chord_header=False is pending deprecation in a future release of Celery.
rengine-celery-1       | Please test the new behavior by setting task_allow_error_cb_on_chord_header to True and report any concerns you might have in our issue tracker before we make a final decision regarding how errbacks should behave when used with chords.
rengine-celery-1       |
rengine-celery-1       |
rengine-celery-1       | vulnerability_scan                 | INFO | Running Vulnerability Scan Queue

Also, when you run rengine for first time it downloads some packages in container. It has this python httpx in rengine-celery-1 container:

# which httpx
/usr/local/bin/httpx

Maybe httpx alias in .bashrc is not working? I don't know...

zerodaykb · 2025-03-07T09:22:13Z

It started working! I did this:

docker exec -it rengine-web-1 sh
rm /go/bin/httpx

docker exec -it rengine-celery-1 sh
rm /go/bin/httpx

and then go to "Tool arsenal" panel -> check updates for httpx
After installing it started working.

fyi @wifiuk maybe it's the same case for you.

Flv-cmd added the bug Something isn't working label Jan 30, 2025

Flv-cmd changed the title ~~bug: <title>Always 0 vulns, Nuclei not work~~ bug: Always 0 vulns, Nuclei not work Jan 30, 2025

dmchaledev added a commit to dmchaledev/rengine that referenced this issue Feb 23, 2025

yogeshojha#1476-yogeshojha#1480-yogeshojha#1487 fixes, adding fallbac…

2ae822b

…k for template-url variable, fixing missing comma, fixing encodedURLData typo

dmchaledev mentioned this issue Feb 23, 2025

#1476-#1480-#1487 Fix, setting fallback for Nuclei variable and fixing JS errors in Vulnerabilities page #1488

Merged

yogeshojha added a commit that referenced this issue Feb 24, 2025

Merge pull request #1488 from dmchaledev/vuln_render_fix

f2f4058

#1476-#1480-#1487 Fix, setting fallback for Nuclei variable and fixing JS errors in Vulnerabilities page

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

bug: Always 0 vulns, Nuclei not work #1480

bug: Always 0 vulns, Nuclei not work #1480

Flv-cmd commented Jan 30, 2025 •

edited

Loading

github-actions bot commented Jan 30, 2025

dmchaledev commented Feb 19, 2025

Flv-cmd commented Feb 20, 2025 •

edited

Loading

Flv-cmd commented Feb 20, 2025

dmchaledev commented Feb 20, 2025

dmchaledev commented Feb 21, 2025

Flv-cmd commented Feb 21, 2025 •

edited

Loading

dmchaledev commented Feb 23, 2025

dmchaledev commented Feb 23, 2025

wifiuk commented Feb 28, 2025

zerodaykb commented Mar 6, 2025 •

edited

Loading

zerodaykb commented Mar 7, 2025

bug: Always 0 vulns, Nuclei not work #1480

bug: Always 0 vulns, Nuclei not work #1480

Comments

Flv-cmd commented Jan 30, 2025 • edited Loading

Is there an existing issue for this?

Current Behavior

Expected Behavior

Steps To Reproduce

Environment

Anything else?

github-actions bot commented Jan 30, 2025

dmchaledev commented Feb 19, 2025

Flv-cmd commented Feb 20, 2025 • edited Loading

Flv-cmd commented Feb 20, 2025

dmchaledev commented Feb 20, 2025

dmchaledev commented Feb 21, 2025

Flv-cmd commented Feb 21, 2025 • edited Loading

dmchaledev commented Feb 23, 2025

dmchaledev commented Feb 23, 2025

wifiuk commented Feb 28, 2025

zerodaykb commented Mar 6, 2025 • edited Loading

zerodaykb commented Mar 7, 2025

Flv-cmd commented Jan 30, 2025 •

edited

Loading

Flv-cmd commented Feb 20, 2025 •

edited

Loading

Flv-cmd commented Feb 21, 2025 •

edited

Loading

zerodaykb commented Mar 6, 2025 •

edited

Loading