Development Guide

Thank you for considering helping out with the code. We welcome anyone on the Internet, and thank you for even the smallest improvements!

Make a module

If you want to write a new module, the best source is written modules and previous pull requests. Create a template module and insert your module into the corresponding directory in the module directory. i.e, if your module is a scanning module, you can create a new directory in the "module" directory called "scan" and place your module in the scan directory. Modules that have added to the "scan" directory are loaded into the framework at runtime and listed under a category matching the directory's name. You can easily run your module by name in the framework.

[maryam][default] > portscan

Module Structure

# modules are defined and configured by the 'meta' class variable
# 'meta' is a dictionary that contains information about the module, ranging from basic information, 
# below is an example 'meta' declaration that contains all of the possible definitions
meta = {
	# Complete Module Name
	'name': 'Google Dork Search',
	'author': '<Your-Name>',
	'version': '0.1',
	'description': 'Search your dork in the google and get result',
	'comments': (
		'Notes',
		'Notes'
	),
	# Sources that you use in your module
	'sources': ('google',),
	'options': (
		# ('<OPTION-NAME', '<VALUE>', '<REQUIRED>', '<DESCRIPTION>', '<SHORT-OPTS>', '<ACTION>', '<TYPE>')
		('dork', None, True, 'Google dork string', '-d', 'store', str),
		('limit', 2, False, 'Google search limit', '-l', 'store', int),
		('count', 50, False, 'Link count in page(min=10, max=100)', '-c', 'store', int),
	),
    'examples': ('godork -d <DORK> -l 15 --output',)
}
# the core of your module will be on module_api. It should just return the output of your module
# .. In the JSON format.
def module_api(self):
	# Use the option name to access option values.
	dork = self.options['dork']
	limit = self.options['limit']
	count = self.options['count']
	# This is how you can access ready classes: self.<class_name>(args).<methods>
	run = self.google(dork, limit, count)
	run.run_crawl()
	# Note that output keys shouldn't be upper-case or camel-case: pwned_emails, not PwnedEmails, pwnedEmails
	output = {'links': run.links} 

	# self.save_gather({<DATA>}, '<Module-Name>', '<QUERY|DORK|HOSTNAME>', <OUTPUT-FLAG(True,False)>)
	self.save_gather(output, 'osint/godork', dork, output=self.options['output'])
	return output

# module_run will show the results. If you have another format for the output you can change this piece of code.
# alert_results simply shows the results.
def module_run(self):
	self.alert_results(module_api(self))

Make a util class

If you want to write a new util class, the best source is written classes and previous pull requests. Utils classes use in modules. Classes will add to the core/util directory, which is widely used, and will be imported by running the framework. you can also reload the classes with the reload * commaand.

An example class

# main doesn't change
class main:
	# you should describe all of parameters
	def __init__(self, q, limit=2):
		""" ask.com search engine

			q         : Query for search
			limit	  : Number of pages
		"""
		# It can be used to access ready methods.
		self.framework = main.framework
		self.q = self.framework.urlib(q).quote
		self.limit = limit
		self._pages = ''
		self.ask = 'www.ask.com'
	# The main method that sets important attributes.
	def run_crawl(self):
		urls = [f"https://{self.ask}/web?q={self.q}&page={i}" for i in range(1, self.limit + 1)]
		max_attempt = len(urls)
		for url in range(len(urls)):
			self.framework.verbose(f"[ASK] Searching in {url} page...")
			try:
				req = self.framework.request(url=urls[url])
			except:
				self.framework.error('[ASK] ConnectionError')
				max_attempt -= 1
				if max_attempt == 0:
					self.framework.error('Ask is missed!')
					break
			else:
				page = req.text
				if '>Next</li>' not in page:
					self._pages += page
					break
				self._pages += page

	@property
	def pages(self):
		return self._pages

	@property
	def dns(self):
		return self.framework.page_parse(self._pages).get_dns(self.q)

	@property
	def emails(self):
		return self.framework.page_parse(self._pages).get_emails(self.q)

	@property
	def docs(self):
		return self.framework.page_parse(self._pages).get_docs(self.q)

In the next you can use ask class in modules:

def module_api(self):
	run = self.ask('microsoft.com', 5)
	run.run_crawl
	emails = run.emails
	for i in emails:
		self.output(i)

Ready methods

• To send a request and get a response

	# All of params
	req = self.request(url=url, timeout=timeout, headers={}, 
		cookie=cookie, auth={}, redirect=False, method='POST', content='', payload={}, proxy=proxy)
	# A simple request
	req = self.request(url)
	# Response. Response is a requests object
	req.headers # headers
	req.status_code # Status request code
	req.text # Read
	req.json() # Json data

• Print errors and exceptions

	self.error('this is an error')
	# output: [!] This is an error.

• Print normal output

	self.output('this is a normal output', color='N') # color is optional.default='N'
	# colors = {'N': 'native', 'R': 'red', 'B': 'blue', 'G': 'green', 'O': 'orange', 'P': 'purple', 'C': 'cyan'}
	# output: [*] this is a normal output

• Formats and presents important output.

	self.alert('this is an alert')
	# output: [*] this is a normal output
	# color: green

• Formats and presents output if in debug mode (very verbose)

	self.debug('msg')
	# output: [*] msg
	# color: blue

• Formats and presents output if in verbose mode.

	self.verbose('This is verbose output')
	# output: [*] This is verbose output
	# color: blue

• Formats and presents styled header text

	self.heading('PHONE', 0)
	# output:

	# -----
	# PHONE
	# -----
	self.heading('PHONE')
	# output:

	# PHONE
	# -----

• Format and display a heading.
  • level is the style of heading. Currently, there are only two style options: 0 and 1.
• Build, display, and store an ASCII table of given data.
  • tdata is the table data as a two-dimensional list (list of lists), with each list representing a row of data. Each list must be of the same length.
  • header (optional) is a list containing the header values. By default, tables are displayed without a header row.
  • title (optional) is the title of the table. By default, tables are displayed without a title.

	self.table(table_data, header=['host', 'ip_address'], title='Hosts')
	# output: 
	"""
	  +-------------------------------------+
	  |                Hosts                |
	  +-------------------------------------+
	  |        host        |   ip_address   |
	  +-------------------------------------+
	  | www.apache.org     | 192.87.106.229 |
	  | www.google.com     | 74.125.225.176 |
	  | www.twitter.com    | 199.59.148.10  |
	  | www.whitehouse.gov | 157.238.74.67  |
	  +-------------------------------------+
	"""

• JSON output

	self.exporter(filename, value, format)
	# Format can be json,txt,xml,csv

Code-Style

• Use simple quotes for simple strings: 'string'
• Use double-quotes for string formation:

f"module_{name}"
f'module "{names}" '

exception: ' # If you want to use double quotes in the expression, there is no need to use double quotes

• Use r prefix for Regular Expressions along with the double quotes:

r"[^\w]+"

exception: r'a href="[^"]+" ' _# If you want to use double quotes in the expression, there is no need to use double quotes _

• Try to avoid using the try/except as far as possible. If you want to use it format it like the following structure:

try:
    ...
except Exception as e:
    ...
else:
    ...

• Variable names shouldn't be camelLower or CamelUpper

fetchIps = .. # wrong
FetchIps = .. # wrong
fetch_ips = .. # true

• Use \t instead of 4 spaces.

• Space errors

def x(a=b, c=d) # correct
def x(a=b,c=d) # wrong
def x(a = b, c = d) # wrong
x(a,b) # wrong
x(a, b) # correct
{'a':'b','c':'d'} # wrong
{'a': 'b', 'c': 'd'} # correct
{'a' : 'b', 'c' : 'd'} # wrong
var_name=1 # wrong
var_name =1 # wrong
var_name = 1 # correct
if(var_name==1): # wrong
if (var_name==1): # wrong
if (var_name == 1): # wrong
if var_name == 1: # correct

Note: If you have any question please contact me