Skip to content

🦛 scrapes websites and generates rss feeds

License

Notifications You must be signed in to change notification settings

hueyy/HungryHippo

Repository files navigation

HungryHippo

Docker Image Version (latest semver) GitHub

To a hungry hippo, everything is a feed

HungryHippo generates RSS/ATOM/JSON feeds from regularly-updating public sites.

Examples

Site Sample Link HungryHippo Link
Telegraph telegraph.co.uk/opinion /individual-site/telegraph.co.uk/?url=https://www.telegraph.co.uk/opinion/
UCL Laws News ucl.ac.uk/laws/news /individual-site/ucl.ac.uk/news/?url=https://cms-feed.ucl.ac.uk/s/search.json?collection=drupal-laws-new%26&meta_UclOrgUnit=%22UCL Faculty of Laws%22&title=UCL%20Laws%20News
The Edge (SG) theedgesingapore.com /individual-site/theedgesingapore.com
PDPC (SG) - Undertakings pdpc.gov.sg /individual-site/pdpc.gov.sg/undertaking
GitHub Code Search github.com/search /github?q="xkcd.com"
SAL Annual Review of Cases journalsonline.academypublishing.org.sg/e-First/Singapore-Academy-of-Law-Annual-Review-of-Singapore-Cases /individual-site/academypublishing.org.sg/annual-review-of-cases
SAL Journal journalsonline.academypublishing.org.sg/Journals/Singapore-Academy-of-Law-Journal/e-First /individual-site/academypublishing.org.sg/sal-journal
SAL Practitioner (Fintech) journalsonline.academypublishing.org.sg/Journals/SAL-Practitioner/Fintech /individual-site/academypublishing.org.sg/sal-practitioner/?area=Fintech

Pop the HungryHippo link into a RSS reader like Feedly, BazQux, or RSSOwl. Or you can use IE.

You can get a list of supported sites by visiting /individual-site.

If something isn't working or if you would like a new site to be added, please create a GitHub issue.

Running HungryHippo

With docker:

  docker build . -t hungryhippo
  docker run -p 3000:3000 hungryhippo

Without docker:

  pnpm i
  pnpm run start:dev

GitHub

Optionally, provide the GITHUB_OAUTH_TOKEN environment variable to enable the GitHub search RSS feeds.

Development

I may have taken the Hungry Hippo theme a bit too far. In short:

  • Muncher: scrapes websites
  • Digestor: turns scraped items into feeds

Credits

Photo taken by David Goehring