pylestra_scrapy_2020-02-07

Instructions

• Create or activate a new virtualenv

• Install dependencies

$ pip install -r requirements.txt

• Run the spider

$ scrapy crawl imob

• Run the shell for debug

$ scrapy shell {url}

• Run the spider and export items as json

$ scrapy crawl imob -o file.json

• Run the spider and export items as csv

$ scrapy crawl imob -o file.csv

• Run the spider and export items as xml

$ scrapy crawl imob -o file.xml

Reference links

Architecture overview

Learn more about css and xpath Selectors

All the settings

Enjoy a tutorial

Presentation slides