Skip to content

Local Search

ktx edited this page Jan 4, 2020 · 10 revisions

This section covers searching for breach data locally using h8mail.

Local search for clear-text files can be performed using -lb or --local-breach.
Local search for tar.gz compressed files can be performed using -gz or --gzip.
You can pass multiple files or folders for each options.

Targeting and searching

Similar to the -t option, the -lb and -gz options can be given files, folders and globs (*) as local search parameters.

Using globing for a clear-text search:

$ h8mail -t john.smith@fcorp.com -lb ./Leaks/2019-* -sk

Using multiple glob inputs, looking through .tar.gz compressed local data:

$ h8mail -t john.smith@fcorp.com -gz ../Collection1/ ../Collection2/EU_* -sk

Since h8mail will target all implicitly indicated files, you can also glob with the extension:

$ h8mail -t targets.txt -gz dump/*.tar.gz -sk

Looking for all emails from target domain:

$ h8mail -t fcorp.com -gz ../Collection1/ -sk --loose

Run multiple searches with regrouped results:

$ h8mail -t targets.txt -lb ../Collection99/ -c h8mail_config.ini -o search_output.csv

Local search mechanics

h8mail can search local data for your target. When using the -lb or -gz options, h8mail performs a multi-processed search, using one separate process for each file, one worker for each core your CPU has (python default).
The local search can be performed in combination with the online scans using the provided APIs, or stay offline using the -sk option.

Multi processing

By default, h8mail's local search uses multi processing. Performing a multi process search is fast, and to keep it fast information is not shared between processes unless the search is complete.
This also means that h8mail will not show a precise search status while a file is being searched. This can sometimes be stressful for bigger files.
If you wish to cancel multi-processed search, and search files one by one instead, simply use -sf or --single-file. This will also show a more precise progress of the current local search.

Loose and offline search

You can use h8mail to look for "loose" patterns in local files using the --loose options, and keep the search offline by skipping default providers with -sk or --skip.
Keep in mind that using custom query with -q performs an implicit --loose an thus will not look for email patterns when targeting.
This will also performs an implicit -sk option and will not query default providers.


Breach Compilation

Use the native Breach Compilation script using -bc or --breachcomp.
This feature was already part of h8mail’s first version, and has been reintegrated. h8mail uses the query.sh script that is included in the torrent, meaning this will only work on Linux & OSX platforms.
Windows users can still use the generic -lb on the data, or have a working bash on Windows.

h8mail was built using the Breach Compilation version referenced here.

To use this option, simply point -bc to the downloaded BreachCompilation folder.

$ h8mail -t targets.txt -bc ./BreachedCompilation/ -sk

A blog post demonstrating h8mail and the Breach Compilation with Docker can be found here.


TLDR

h8mail can search clear-text files using -lb and tar.gz files using -gz.
Skip default online checks using -sk.
Skip multi processing using -sf.
Search for any string (hashs, IPs) using --loose.
Search the Breach Compilation with its native search script using -bc.

Clone this wiki locally