Web-Crawler

Project of WOC

• This project is about a web crawler,a tool which extracts information about webpages.
• I will use Python language for the project.
• This Crawler crawls over the internet and stores links,images and screenshots of linkavailable onthe website.
• for further help: use command "python WCSC.py man" or "python WCSC.py -help" or use Help button provided in GUI

Modules/libraries used:

• tldextract
• selenium
• os
• bs4
• requests
• sys
• termcolor
• itertools
• keyboard
• time
• re
• tkinter

Below are some Snippets of working of the tool:

• GUI interface

• man_page

• Headers

#Steps to follow: -clone the repo to the desired directory by following commands

git clone https://github.com/Alphx-rgb/Web-Crawler.git

-Now run the gui.py command -seek help for usage

Written by Ayush Budhiraja, for any suggestions do contact me at : ayushbudhiraja795@gmail.com