Clustering of Multi-Protocol Templates can panic or not execute correctly

[jdewald]

Nuclei version: v3.2.2

Current Behavior:

If you have a template that has multiple protocols (the example from https://docs.projectdiscovery.io/templates/protocols/multi-protocol works here), where only one of its protocols clusters other templates (e.g. ones without the second protocol), under certain circumstances a panic will occur when it tries to execute the "mismatched" protocol. In the case where there is successful clustering, it won't execute the "mismatched" portion of the cluster.

Expected Behavior:

Clustering should not lead to panic (:)), additionally if templates are clustered, all aspects of the template should execute.

Steps To Reproduce:

dns-http-template.yaml - simply the contents from the multi-protocol example

a-template.yaml - same contents, with the dns protocol section removed and id changed to a-http-template
b-template.yaml - same contents, with the dns protocol section removed and id changed to b-http-template
e-template.yaml - same for above, but using e-http-template

For example for a-template.yaml

id: a-http-template

info:
  name: a http template
  author: pdteam
  severity: info

http:
  - method: GET # http request
    path:
      - "{{BaseURL}}"

    matchers:
      - type: dsl
        dsl:
          - contains(http_body,'Domain not found') # check for string from http response
          - contains(dns_cname, 'github.io') # check for cname from dns response
        condition: and

Now run:
./nuclei -vv -t dns-http-template.yaml -t a-template.yaml -target http://localhost

Template load output

[a-http-template] a http template (@pdteam) [info]
[dns-http-template] dns + http takeover template (@pdteam) [info]

You'll receive this panic:

goroutine 1 [running]:
github.com/projectdiscovery/nuclei/v3/pkg/templates.NewClusterExecuter({0x14000fe2ce0, 0x2, 0x14000ddd7b8?}, 0x14000fe8240)
	github.com/projectdiscovery/nuclei/v3/pkg/templates/cluster.go:214 +0x78c
github.com/projectdiscovery/nuclei/v3/pkg/templates.ClusterTemplates({_, _, _}, {{0x0, 0x0}, {0x0, 0x0}, {{0x0, 0x0}, {{0x0, ...}}, ...}, ...})
	github.com/projectdiscovery/nuclei/v3/pkg/templates/cluster.go:158 +0x2b4
github.com/projectdiscovery/nuclei/v3/pkg/core.(*Engine).ExecuteScanWithOpts(0x1400086e6c0, {0x14000fe2cc0?, 0x2, 0x2}, {0x105a12150, 0x14000d82f00}, 0x0)
	github.com/projectdiscovery/nuclei/v3/pkg/core/execute_options.go:45 +0x130
github.com/projectdiscovery/nuclei/v3/internal/runner.(*Runner).executeTemplatesInput(0x14000a795e0, 0x140001e2900, 0x140001e3c20?)
	github.com/projectdiscovery/nuclei/v3/internal/runner/runner.go:633 +0x1e8
github.com/projectdiscovery/nuclei/v3/internal/runner.(*Runner).runStandardEnumeration(_, {{0x0, 0x0}, {0x0, 0x0}, {{0x0, 0x0}, {{0x0, 0x0}}, {{0x0, ...}}, ...}, ...}, ...)
	github.com/projectdiscovery/nuclei/v3/internal/runner/runner.go:336 +0x64
github.com/projectdiscovery/nuclei/v3/internal/runner.(*Runner).RunEnumeration(0x14000a795e0)
	github.com/projectdiscovery/nuclei/v3/internal/runner/runner.go:548 +0x110c
main.main()
	./main.go:170 +0x530

However, if you do this instead, you will not get a crash:
./nuclei -vv -t dns-http-template.yaml -t e-template.yaml -target http://localhost

Template load output:

[dns-http-template] dns + http takeover template (@pdteam) [info]
[e-http-template] e http template (@pdteam) [info]
[INF] No results found. Better luck next time!

There's no indication of clustering in this case and everything is executed:

./nuclei -debug -vv -t dns-http-template.yaml -t e-template.yaml -target http://localhost 2>&1 | grep 'GET\|QUERY'
;; opcode: QUERY, status: NOERROR, id: 37497
;; flags: rd; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 1
GET / HTTP/1.1
;; opcode: QUERY, status: NXDOMAIN, id: 37497
;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 1, ADDITIONAL: 1
GET / HTTP/1.1

Finally if you execute:
./nuclei -vv -t dns-http-template.yaml -t a-template.yaml -t b-template.yaml -target http://localhost

...
[a-http-template] a http template (@pdteam) [info]
[b-http-template] b http template (@pdteam) [info]
[dns-http-template] dns + http takeover template (@pdteam) [info]
[INF] Templates clustered: 3 (Reduced 3 Requests)

Clustering occurs, but the DNS QUERY is not executed, only the clustered HTTP GET:

./nuclei -debug -vv -t dns-http-template.yaml -t a-template.yaml -t b-template.yaml -target http://localhost >/dev/null 2>&1 | grep 'QUERY\|GET'
GET / HTTP/1.1

So it seems to be the case that if the DNS template is loaded first the "cluster" will get treated as having DNS protocol requests, even though the actual clustering occurred on the http protocol. However if it clusters/executes against the "http" ones first, the dns protocol isn't seen.

Anything else:

I suspect that multi-protocol templates simply just didn't get factored into clustering (perhaps didn't exist at the time, I'm fairly new to nuclei usage).

I think the simplest fix is likely to treat "multi-protocol" the same as "multiple requests" and not cluster or if there is a way to only cluster when all protocols match (and have 1 request each?).

[jdewald]

That was quick, thanks!