Cross Platform C# web crawler framework built for speed and flexibility. Please star this project! +1.

A simple web crawler, using Abot, that indexes page contents into Azure Search.

A local discord bot that joins a voice channel and plays youtube links as audio (Music player)

Implementation of two interfaces (ICrawledUrlRepository, IPagesToCrawlRepository). Allows to resume crawl after interruption by serialization collections of urls. NOTE in general this will not work because some collections of state are not serialized, for example a collection in CrawlContext. This solves my particular problem