Wikipedia Graph Visualization using NetworkX

Table of Contents

• About the Project
  • Tasks
  • To Do
  • Built With
• Fork the Repo and Contribute
• Contact

About the Project

In this project, we explore NetworkX, a Python library for graph algorithms and visualizations. We scrape Wikipedia pages for any arbitrary search word and get the first web link referred by the wiki page. Then, we visit that page and again get the first link. We recursively keep visiting the web links and store them in a graph. If the link is already visited, we stop. The links/search words are stored as nodes in a directed graph. The graph is stored using Pickle library in Python. The final visualization of the graph in a HTML page is done using PyVis. PyVis takes in the NetworkX graph and renders a HTML page using vis.js.

Tasks

1. Webscaping using requests and Beautiful Soup.
2. Cleaning links to get words to be used as graph nodes.
3. Build graph with nodes obtained using NetworkX.
4. Visualization of graph using PyVis.

To Do

1. Recursively visit and store all links from first non-empty <p> tag in each wiki page. Try -
   • Breadth First Search (BFS)
   • Depth First Search (DFS)

Built With

• NetworkX
• PyVis
• Beautiful Soup
• Requests
• Pickle
• Jupyter Notebook

Fork the Repo and Contribute

Contributions are what make the open source community such an amazing place to be learn, inspire, and create. Any contributions you make are greatly appreciated.

1. Fork the Project (click on Fork in the top-left corner)
2. Create your Feature Branch (git checkout -b feature)
3. Commit your Changes (git commit -m 'Add some AmazingFeature')
4. Push to the Branch (git push origin feature)
5. Open a Pull Request

Contact

Sinjoy Saha

• LinkedIn
• Twitter