This a gem which creates an API for booking and pricing data for the following sites:
- Hostelworld
- Hostelbookers (back soon, use < 0.6)
- Gomio (removed too slow to speak of)
Web scrapping utility for publicly accessible data such as dates and prices.
This gem strictly follows the Well-behaved bots Policy, the standard policy of Google, Yahoo & many other search engines. If a site owner wishes to give instructions to web robots, including this one, he/she must place a text file called robots.txt in the root of the web site hierarchy (e.g. www.example.com/robots.txt). This text file should contain the instructions in a specific format (see examples below). Robots that choose to follow the instructions try to fetch this file and read the instructions before fetching any other file from the web site. If this file doesn’t exist web robots assume that the web owner wishes to provide no specific instructions.
User-agent: * Disallow: /