add `-dast` flag and multiple bug fixes for dast templates #4941

tarunKoyalwar · 2024-03-25T15:35:49Z

Proposed Changes

add -dast flag and deprecate -fuzz flag
fuzz: add default get method in fuzzing filters
fuzz: remove residual payload logic from old implementation
fuzz: fix no values to generate requests error
improve stdout template loading stats
stdout: force display warnings if no templates are loaded
closes filter support to dast templates #4935
closes interactsh-url dependent variables are not lazy evaluated #4946
closes Non-printable characters in extractor output are inconsistently escaped #4929

Before

when running dast template with multiple mode, payloads were stacked after every run instead of being reset

$  nuclei -l integration_tests/fuzz/testData/ginandjuice.proxify.yaml -im yaml -t b.yaml -fuzz -v         130 ↵

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.2

		projectdiscovery.io

[VER] Started metrics server at localhost:9092
[INF] Current nuclei version: v3.2.2 (latest)
[INF] Current nuclei-templates version: v9.8.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 85
[INF] Templates loaded for current scan: 1
[WRN] Loaded 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 9
[VER] [fuzz0test] Sent HTTP request to http://127.0.0.1:8082/blog/post?postId=3%27&source=proxify%27
[VER] [fuzz0test] Sent HTTP request to http://127.0.0.1:8082/blog/post?postId=3%27%22&source=proxify%27%22
[VER] [fuzz0test] Sent HTTP request to http://127.0.0.1:8082/blog/post?postId=3%27%22%3B&source=proxify%27%22%3B
[INF] No results found. Better luck next time!

After

$  ./nuclei -l integration_tests/fuzz/testData/ginandjuice.proxify.yaml -im yaml -t b.yaml -dast -v

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.2

		projectdiscovery.io

[VER] Started metrics server at localhost:9092
[INF] Current nuclei version: v3.2.2 (latest)
[INF] Current nuclei-templates version: v9.8.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 85
[INF] Templates loaded for current scan: 1
[WRN] Loading 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 9
[VER] [fuzz0test] Sent HTTP request to http://127.0.0.1:8082/blog/post?postId=3'&source=proxify'
[VER] [fuzz0test] Sent HTTP request to http://127.0.0.1:8082/blog/post?postId=3"&source=proxify"
[VER] [fuzz0test] Sent HTTP request to http://127.0.0.1:8082/blog/post?postId=3;&source=proxify;
[INF] No results found. Better luck next time!

tarunKoyalwar · 2024-03-25T16:00:17Z

closes Non-printable characters in extractor output are inconsistently escaped #4929 without breaking json extracted output

$ ./nuclei -t a.yaml -u scanme.sh

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.2

		projectdiscovery.io

[INF] Current nuclei version: v3.2.2 (latest)
[INF] Current nuclei-templates version: v9.8.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 85
[INF] Templates loaded for current scan: 1
[WRN] Loading 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 1
[ssh-server-enumeration] [javascript] [info] scanme.sh:22 ["{"AlgorithmSelection":{"client_to_server_alg_group":{"cipher":"aes128-ctr","compression":"none","mac":"hmac-sha2-256"},"dh_kex_algorithm":"curve25519-sha256@libssh.org","host_key_algorithm":"ecdsa-sha2-nistp256","server_to_client_alg_group":{"cipher":"aes128-ctr","compression":"none","mac":"hmac-sha2-256"}},"Banner":"","ClientID":null,"ClientKex":null,"Crypto":null,"DHKeyExchange":{"curve25519_sha256_params":{"server_public":"JksLByu2//6pnjcij46E6ovX9XzUkE7Xu6Ctnax8HkI="},"server_host_key":{"algorithm":"ecdsa-sha2-nistp256","ecdsa_public_key":{"b":"WsY12Ko6k+ez671VdpiGvGUdBrDMU7D2O848PifSYEs=","curve":"P-256","gx":"axfR8uEsQkf4vOblY6RA8ncDfYEt6zOg9KE5RdiYwpY=","gy":"T+NC4v4af5uO5+tKfA+eFivOM1drMV7Oy7ZAaDe/UfU=","length":256,"n":"/////wAAAAD//////////7zm+q2nF56E87nKwvxjJVE=","p":"/////wAAAAEAAAAAAAAAAAAAAAD///////////////8=","x":"z26Q0tpPfVKxgLHzuj0SxaECCYLqlIm5tNy3Fz5KsUw=","y":"tjGcTXlRlQy67VjJLj5iqO3X+VvGEFw2bkRSSsHHrCg="},"fingerprint_sha256":"28cdf69e089470409de139506f5f33fdcc5b747641d974da3236863aa8a98ca5","raw":"AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBM9ukNLaT31SsYCx87o9EsWhAgmC6pSJubTctxc+SrFMtjGcTXlRlQy67VjJLj5iqO3X+VvGEFw2bkRSSsHHrCg="},"server_signature":{"h":"mk0WH/upBHoC+NJqYA9Fj+Yu/VouaibNJfP/B4Q1ADI=","parsed":{"algorithm":"ecdsa-sha2-nistp256","value":"AAAAIQDVxji4Wi9YYmzQQfOc/c/+msf7zzGKMGeBYOPpklQXVgAAACBzzmpcB0y1u9tePWvC1Y/BJ9qF65l38tbwNvvJhc2BUw=="},"raw":"AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAABJAAAAIQDVxji4Wi9YYmzQQfOc/c/+msf7zzGKMGeBYOPpklQXVgAAACBzzmpcB0y1u9tePWvC1Y/BJ9qF65l38tbwNvvJhc2BUw=="}},"ServerID":{"Comment":"Ubuntu-4ubuntu0.11","ProtoVersion":"2.0","Raw":"SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.11","SoftwareVersion":"OpenSSH_8.2p1"},"ServerKex":{"client_to_server_ciphers":["chacha20-poly1305@openssh.com","aes128-ctr","aes192-ctr","aes256-ctr","aes128-gcm@openssh.com","aes256-gcm@openssh.com"],"client_to_server_compression":["none","zlib@openssh.com"],"client_to_server_macs":["umac-64-etm@openssh.com","umac-128-etm@openssh.com","hmac-sha2-256-etm@openssh.com","hmac-sha2-512-etm@openssh.com","hmac-sha1-etm@openssh.com","umac-64@openssh.com","umac-128@openssh.com","hmac-sha2-256","hmac-sha2-512","hmac-sha1"],"cookie":"1MCxZqbogvRN/wo7mhuUow==","first_kex_follows":false,"host_key_algorithms":["rsa-sha2-512","rsa-sha2-256","ssh-rsa","ecdsa-sha2-nistp256","ssh-ed25519"],"kex_algorithms":["curve25519-sha256","curve25519-sha256@libssh.org","ecdh-sha2-nistp256","ecdh-sha2-nistp384","ecdh-sha2-nistp521","diffie-hellman-group-exchange-sha256","diffie-hellman-group16-sha512","diffie-hellman-group18-sha512","diffie-hellman-group14-sha256","kex-strict-s-v00@openssh.com"],"reserved":0,"server_to_client_ciphers":["chacha20-poly1305@openssh.com","aes128-ctr","aes192-ctr","aes256-ctr","aes128-gcm@openssh.com","aes256-gcm@openssh.com"],"server_to_client_compression":["none","zlib@openssh.com"],"server_to_client_macs":["umac-64-etm@openssh.com","umac-128-etm@openssh.com","hmac-sha2-256-etm@openssh.com","hmac-sha2-512-etm@openssh.com","hmac-sha1-etm@openssh.com","umac-64@openssh.com","umac-128@openssh.com","hmac-sha2-256","hmac-sha2-512","hmac-sha1"]},"UserAuth":["publickey","password"]}"]

tarunKoyalwar · 2024-03-25T16:20:28Z

aws request signature templates can only be run in signed & verified templates

$ ./nuclei -t ~/Codebase/nuclei-templates/http/cloud/aws                                             

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.2

		projectdiscovery.io

[WRN] Found 9 templates loaded with deprecated protocol syntax, update before v3 for continued support.
[WRN] Skipping 25 templates, HTTP Request signatures can only be used in Signed & Verified templates.
[INF] Current nuclei version: v3.2.2 (latest)
[INF] Current nuclei-templates version: v9.8.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] No results found. Better luck next time!
[FTL] Could not run nuclei: no templates provided for scan

pkg/output/format_screen.go

Mzack9999

minor suggestions

pkg/catalog/loader/loader.go

pkg/fuzz/component/path.go

pkg/fuzz/dataformat/kv.go

pkg/fuzz/parts.go

tarunKoyalwar · 2024-03-26T14:36:10Z

Fix lazy eval of `interactsh-url` (see: #4946)

$ ./nuclei -u https://scanme.sh -t a.yaml

                     __     _
   ____  __  _______/ /__  (_)
  / __ \/ / / / ___/ / _ \/ /
 / / / / /_/ / /__/ /  __/ /
/_/ /_/\__,_/\___/_/\___/_/   v3.2.2

		projectdiscovery.io

[INF] Current nuclei version: v3.2.2 (latest)
[INF] Current nuclei-templates version: v9.8.0 (latest)
[WRN] Scan results upload to cloud is disabled.
[INF] New templates added in latest release: 85
[INF] Templates loaded for current scan: 1
[WRN] Loading 1 unsigned templates for scan. Use with caution.
[INF] Targets loaded for current scan: 1
[INF] Using Interactsh Server: oast.online
[some-exploit] [http] [critical] https://scanme.sh/bnNsb29rdXAgY28xZGtmc280N21oOGNwOGg1MTA2ODRyb2NtNHE5dWd5Lm9hc3Qub25saW5l ["nslookup co1dkfso47mh8cp8h510684rocm4q9ugy.oast.online"]

Mzack9999

lgtm - Might be worth considering the implementation in a follow up task of a HTTP proxy mode for RFC-compliant requests, so that DAST mode can be performed on in-flight requests transparently and nuclei could virtually support all third party apps supporting proxy, without the need of various intermediate formats like openapi

tarunKoyalwar · 2024-03-28T13:32:13Z

follow-up #4953

ansanyuan · 2024-08-16T02:53:28Z

Can this dast engine test post content in http packages? and headers, similar to awvs. Or can we only test the fields in GET at present, and there is little information on the Internet, so let's ask here.THX^^

tarunKoyalwar added 7 commits March 25, 2024 19:58

add default get method

6d6527d

remove residual payload logic from old implementation

2554cba

fuzz: clone current state of component

1510ae9

fuzz: bug fix stacking of payloads in multiple mode

b30d198

improve stdout template loading stats

05cc1a3

stdout: force display warnings if no templates are loaded

5d5408e

update flags in README.md

51d66de

tarunKoyalwar self-assigned this Mar 25, 2024

tarunKoyalwar marked this pull request as ready for review March 25, 2024 15:42

tarunKoyalwar requested a review from ehsandeep March 25, 2024 15:42

tarunKoyalwar changed the title ~~issue 4935 dast filters~~ add -dast flag and multiple bug fixes for dast templates Mar 25, 2024

quote non-ascii chars in extractor output

82a7d60

tarunKoyalwar mentioned this pull request Mar 25, 2024

Non-printable characters in extractor output are inconsistently escaped #4929

Closed

tarunKoyalwar linked an issue Mar 25, 2024 that may be closed by this pull request

Non-printable characters in extractor output are inconsistently escaped #4929

Closed

aws request signature can only be used in signed & verified tmpls

4d2dfd6

tarunKoyalwar added 3 commits March 25, 2024 23:45

deprecate request signature

78d2144

remove logic related to deprecated fuzzing input

60fe34d

update test to use ordered params

0d6cff2

ehsandeep requested a review from Mzack9999 March 26, 2024 12:22

tovask reviewed Mar 26, 2024

View reviewed changes

pkg/output/format_screen.go Outdated Show resolved Hide resolved

Mzack9999 requested changes Mar 26, 2024

View reviewed changes

pkg/catalog/loader/loader.go Outdated Show resolved Hide resolved

pkg/fuzz/component/path.go Outdated Show resolved Hide resolved

pkg/fuzz/dataformat/kv.go Outdated Show resolved Hide resolved

pkg/fuzz/parts.go Outdated Show resolved Hide resolved

fix interactsh-url lazy eval: #4946

26e26ff

tarunKoyalwar added 2 commits March 26, 2024 20:19

output: skip unnecessary updates when unescaping

a4d5d8b

updates as per requested changes

c99847b

tarunKoyalwar requested a review from Mzack9999 March 26, 2024 14:53

Mzack9999 approved these changes Mar 26, 2024

View reviewed changes

DhiyaneshGeek mentioned this pull request Mar 27, 2024

Create CVE-2024-28255.yaml projectdiscovery/nuclei-templates#9421

Merged

2 tasks

This was referenced Mar 27, 2024

Update JSONSchema Libraries #4943

Merged

Support / add proxy-server input mode to fuzz / run checks on live traffic #4953

Open

ehsandeep approved these changes Mar 29, 2024

View reviewed changes

ehsandeep merged commit e88889b into dev Mar 29, 2024
12 checks passed

ehsandeep deleted the issue-4935-dast-filters branch March 29, 2024 08:01

ehsandeep mentioned this pull request Mar 29, 2024

url deduping for fuzzing #4962

Open

BrewTestBot mentioned this pull request Apr 3, 2024

nuclei 3.2.3 Homebrew/homebrew-core#167929

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add `-dast` flag and multiple bug fixes for dast templates #4941

add `-dast` flag and multiple bug fixes for dast templates #4941

tarunKoyalwar commented Mar 25, 2024 •

edited

Loading

tarunKoyalwar commented Mar 25, 2024

tarunKoyalwar commented Mar 25, 2024

Mzack9999 left a comment

tarunKoyalwar commented Mar 26, 2024

Mzack9999 left a comment

tarunKoyalwar commented Mar 28, 2024

ansanyuan commented Aug 16, 2024

add -dast flag and multiple bug fixes for dast templates #4941

add -dast flag and multiple bug fixes for dast templates #4941

Conversation

tarunKoyalwar commented Mar 25, 2024 • edited Loading

Proposed Changes

Before

After

tarunKoyalwar commented Mar 25, 2024

tarunKoyalwar commented Mar 25, 2024

Mzack9999 left a comment

Choose a reason for hiding this comment

tarunKoyalwar commented Mar 26, 2024

Fix lazy eval of interactsh-url (see: #4946)

Mzack9999 left a comment

Choose a reason for hiding this comment

tarunKoyalwar commented Mar 28, 2024

ansanyuan commented Aug 16, 2024

add `-dast` flag and multiple bug fixes for dast templates #4941

add `-dast` flag and multiple bug fixes for dast templates #4941

tarunKoyalwar commented Mar 25, 2024 •

edited

Loading

Fix lazy eval of `interactsh-url` (see: #4946)