Reason Rust Scraper

This is a code repository for implementing how to scraping and crawling websites using Rust, and ReasonML. This project was created by me with the aim of being a tools in my company i work. However, because of the sudden change of ideas and concepts finally i make it like this hehe, as well as my first project when learning & use ReasonML.

If you have never heard of scraping, try to read this web scraping for more information.

Requirements

Server

1. Rust
2. Rustup
3. Cargo
4. Apache & MySQL

Client

1. Node
2. Bucklescript

Getting started

1. Clone this repo
2. Start Apache & MySQL server
3. Create new database and .env file and set config, like this .env
4. npm install to install all req'd dependencies

Running

To running this project locally:

1. For MacOS, and you have a iTerm. You can running easily, type this command

 $ sudo chmod +x ./run.sh && ./run.sh

2. Or, you can running manually, for client side:

  $ npm start

3. Open a new terminal tab/window, and type this command:

 $ npm server

4. Open a new terminal tab/window again, for running server side:

 $ cargo run

5. Open in you browser http://localhost:8000/

Testing

$ sudo chmod +x ./test.sh && ./test.sh

Build

Client:

$ npm run build

Server:

$ cargo build

Contributing

Pull requests are welcome. For major changes, please:

1. Create issues and PRs - bugs, missing documentation, typos, unreadable code...
2. Make sure to update tests as appropriate