Skip to content

Latest commit

 

History

History
11 lines (10 loc) · 583 Bytes

README.md

File metadata and controls

11 lines (10 loc) · 583 Bytes

Erowid Scraper

1º time trying to create an effective scrapper

For now, its only the basic, I still have to plan the data structure and which info will be scrapped or not, still have a lot to explore.

To-do

  • Get all data from each row of the table
  • Parse all the data from the tables from each page
  • Make requests from each experience page (async and concurrent)
  • Format the data received to Markdown from each page for insertion into a database
  • Data insertions using psycopg2 to PostgreSQL
  • Create a tag for each unique substance found