panic: runtime error: invalid memory address or nil pointer dereference

[briefcasehi]

OS

Linux scan 5.3.0-1022-azure #23~18.04.1-Ubuntu SMP Mon May 11 11:55:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

cmd

wc -l subdomain.txt
1969 subdomain.txt

cat subdomain.txt | httpx -silent -title -json -o result.json

Get Error

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x6faccc]

goroutine 517 [running]:
github.com/projectdiscovery/httpx/common/httpx.(*HTTPX).NewRequest(0xc000022b80, 0x7a8ff1, 0x3, 0xc000eb4570, 0x2c, 0xc000eb4570, 0x2c, 0x0)
        /root/go/pkg/mod/github.com/projectdiscovery/httpx@v0.0.1/common/httpx/httpx.go:151 +0x7c
main.analyze(0xc000022b80, 0x7a9e90, 0x5, 0xc000eb4510, 0x24, 0x0, 0xc00007d0e0, 0xc0000721e0)
        /root/go/pkg/mod/github.com/projectdiscovery/httpx@v0.0.1/cmd/httpx/httpx.go:165 +0x173
main.main.func2(0xc00000e0e0, 0xc000022b80, 0x7a9e90, 0x5, 0xc00007d0e0, 0xc0000721e0, 0xc000eb4510, 0x24)
        /root/go/pkg/mod/github.com/projectdiscovery/httpx@v0.0.1/cmd/httpx/httpx.go:121 +0xab
created by main.main
        /root/go/pkg/mod/github.com/projectdiscovery/httpx@v0.0.1/cmd/httpx/httpx.go:119 +0x88a

[ehsandeep]

Hey @briefcasehi,

Not managed to reproduce this, but definitely something needs to be fixed here, thanks for creating this issue, also when you use json as output, it will include all the details as default including title, length, status code in the results, so you can ignore using -title as an extra flag when json is used.

root@b0x:~# wc 11.txt 
7614 11.txt
root@b0x:~# cat 11.txt | httpx -silent -title -json -o result.json | wc 
210

[briefcasehi]

I cannot ignore using -title as an extra flag when json is used.

echo 'www.hackerone.com' | httpx -silent -json

{"url":"https://www.hackerone.com","content-length":54166,"status-code":200,"title":"","error":null,"vhost":false}

 echo 'www.hackerone.com' | httpx -silent -title -json

{"url":"https://www.hackerone.com","content-length":54166,"status-code":200,"title":"Bug Bounty - Hacker Powered Security Testing | HackerOne","error":null,"vhost":false}

[ehsandeep]

looks like you found another issue, this is not expected behavior, thanks, I will keep you updated on this.

[ehsandeep]

Hey @briefcasehi,

Both issues should be fixed, you can see the latest changes with

GO111MODULE=on go get -u -v github.com/projectdiscovery/httpx/cmd/httpx@master

[briefcasehi]

@bauthard

Thanks a lot!