feat(http): add skip-secret-file field

[dwisiswant0]

Proposed changes

Closes #5521 #5455

How has this been tested?

# secret-file-test.yaml
static:
  - type: basicauth
    domains:
      - scanme.sh
    username: foo
    password: bar

# secret-file-test-template.yaml
id: secret-file-test-template

info:
  name: Secret File Test Template
  author: dwisiswant0
  severity: info
  tags: test

http:
  - method: HEAD
    path:
      - "{{BaseURL}}/basic/use-secret-file"
    headers:
      Authorization: Bearer YmFyOmZvbwo= # bar:foo -- should be overriden by secret file
      X-Skip-Secret-File: false

  - method: HEAD
    path:
      - "{{BaseURL}}/basic/skip-secret-file"
    headers:
      Authorization: Bearer YmFyOmZvbwo= # bar:foo
      X-Skip-Secret-File: true
    skip-secret-file: true

  - raw:
      - |
        HEAD /raw/use-secret-file HTTP/1.1
        Host: {{Hostname}}
        Authorization: Bearer YmFyOmZvbwo=
        X-Skip-Secret-File: false
    skip-secret-file: false

  - raw:
      - |
        HEAD /raw/skip-secret-file HTTP/1.1
        Host: {{Hostname}}
        Authorization: Bearer YmFyOmZvbwo=
        X-Skip-Secret-File: true
    skip-secret-file: true

Proof

$ go run cmd/nuclei/main.go -duc -sf /tmp/secret-file-test.yaml -t /tmp/secret-file-test-template.yaml -u https://scanme.sh -debug-req
[INF] [secret-file-test-template] Dumped HTTP request for https://scanme.sh/basic/use-secret-file

HEAD /basic/use-secret-file HTTP/1.1
Host: scanme.sh
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.45 Safari/537.36
Connection: close
Accept: */*
Accept-Language: en
Authorization: Basic Zm9vOmJhcg==
X-Bypass-Secret-File: false

[INF] [secret-file-test-template] Dumped HTTP request for https://scanme.sh/basic/bypass-secret-file

HEAD /basic/bypass-secret-file HTTP/1.1
Host: scanme.sh
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0
Connection: close
Accept: */*
Accept-Language: en
Authorization: Bearer YmFyOmZvbwo=
X-Bypass-Secret-File: true

[INF] [secret-file-test-template] Dumped HTTP request for https://scanme.sh/raw/use-secret-file

HEAD /raw/use-secret-file HTTP/1.1
Host: scanme.sh
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 14_2_1) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.3.1 Safari/605.1.15
Connection: close
Authorization: Basic Zm9vOmJhcg==
X-Bypass-Secret-File: false

[INF] [secret-file-test-template] Dumped HTTP request for https://scanme.sh/raw/bypass-secret-file

HEAD /raw/bypass-secret-file HTTP/1.1
Host: scanme.sh
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4.1 Safari/605.1.46
Connection: close
Authorization: Bearer YmFyOmZvbwo=
X-Bypass-Secret-File: true

[INF] No results found. Better luck next time!

Checklist

• Pull request is created against the dev branch
• All checks passed (lint, unit/integration/regression tests etc.) with my changes
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

[ehsandeep]

@dwisiswant0 can we also update the docs at https://docs.projectdiscovery.io/tools/nuclei/authenticated-scans

[dwisiswant0]

@dwisiswant0 can we also update the docs at https://docs.projectdiscovery.io/tools/nuclei/authenticated-scans

Yeah, I'm currently drafting this.