Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add flag to preview first N characters #1316

Merged
merged 14 commits into from
Sep 7, 2023
Merged

Conversation

RamanaReddy0M
Copy link
Contributor

@RamanaReddy0M RamanaReddy0M commented Aug 9, 2023

closes #935

test cmd's

✗ go run . -u https://example.com -bp -strip
✗ go run . -u https://example.com -bp=150 -strip=html
✗ go run . -u https://example.com -bp
✗ go run . -u https://example.com -bp=200

NOTE: Used dyanmic var to achieve this: projectdiscovery/goflags#130

@RamanaReddy0M RamanaReddy0M self-assigned this Aug 9, 2023
@RamanaReddy0M RamanaReddy0M linked an issue Aug 9, 2023 that may be closed by this pull request
Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can remove the html tags from response body and only print actual content in the same line, for example:

go run . -u https://example.com -silent -bp=30

https://example.com [Example Dom]

@RamanaReddy0M
Copy link
Contributor Author

RamanaReddy0M commented Aug 10, 2023

✗ go run . -u https://example.com -bp -strip

    __    __  __       _  __
   / /_  / /_/ /_____ | |/ /
  / __ \/ __/ __/ __ \|   /
 / / / / /_/ /_/ /_/ /   |
/_/ /_/\__/\__/ .___/_/|_|
             /_/

                projectdiscovery.io

[INF] Current httpx version v1.3.4 (latest)
https://example.com [ Example Domain body { background-color: #f0f0f2; margin: 0; padding: 0; font-family: -apple-system,]

Copy link
Member

@Mzack9999 Mzack9999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The body should be printed within ... [ body preview ] ... (maybe we should consider -strip newline,html as default) just like -title
  • Consider using https://github.com/microcosm-cc/bluemonday for specific html sanitisation (there should be already an instance at
    htmlPolicy *bluemonday.Policy
    )
  • Update readme/docs
  • Consider adding the new flags to functional tests

Copy link
Member

@Mzack9999 Mzack9999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm!

$ echo https://example.com | go run . -bp 
...
https://example.com [<!doctype html>\n<html>\n<head>\n <title>Example Domain</title>\n\n <meta charset="utf-8" />\n <meta]
$ echo https://example.com | go run . -bp -strip
...
https://example.com [ Example Domain This domain is for use in illustrative examples in documents. You may use this domai]

@Mzack9999 Mzack9999 added the Type: Enhancement Most issues will probably ask for additions or changes. label Aug 29, 2023
Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CLI

$ echo https://www.hackerone.com | ./httpx -silent -bp
https://www.hackerone.com [Skip to main content 6th Edition of the Hacker Powered Security Report is available for downloadGe]

JSON

$ echo https://www.hackerone.com | ./httpx -silent -bp -json | jq -r .body_preview
Skip to main content 6th Edition of the Hacker Powered Security Report is available for downloadGe

@ehsandeep ehsandeep merged commit 76adadf into dev Sep 7, 2023
11 checks passed
@ehsandeep ehsandeep deleted the issue-935-body-preview branch September 7, 2023 09:56
@ehsandeep ehsandeep added the Status: Completed Nothing further to be done with this issue. Awaiting to be closed. label Sep 7, 2023
@hktalent
Copy link

@RamanaReddy0M 😊
Please support this
-bp=200,500
It means only get
Content with length 500 bytes starting from 200 bytes

This is useful for batch detection of specific vulnerabilities

Of course, regarding the capture of body

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Status: Completed Nothing further to be done with this issue. Awaiting to be closed. Type: Enhancement Most issues will probably ask for additions or changes.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flag to display first N characters of the response body
4 participants