The entire recommender system is developed using layers of lambda architecture and is implemented in three different directories:
To get the OVA file, follow the link given below.
https://drive.google.com/open?id=1paASmSftbf2hcrWD0FBYdt0iloqkJihO
Follow these steps to start the VM.
- Import the appliance as an Ubuntu 64-bit OS in Oracle VMWare.
- Minimum requirements: Hard disk - 20GB, RAM - 4GB (6-8GB preferred)
- Credentials: Username - bdm Password - bdm
- Enter the command: sudo su -
- Enter the password again: bdm
- This is a preinstalled environement and consists
- Hadoop
- Spark
- Cassandra
- Jupyter Notebooks
- To open jupyter notebooks with pyspark and cassandra, enter this command:
PYSPARK_DRIVER_PYTHON="jupyter" \
PYSPARK_DRIVER_PYTHON_OPTS="notebook --ip=<IP_ADDRESS>" \
pyspark --packages datastax:spark-cassandra-connector:2.0.0-s_2.11 \
--conf spark.cassandra.connection.host=127.0.0.1 - To get the IP_ADDRESS of VM, type: ifconfig in the terminal.
To run cassandra database follow these steps:
- Login as root user.
- Run: /usr/local/lib/apache-cassandra-3.10/bin/cassandra -R
- To access cqlsh: /usr/local/lib/apache-cassandra-3.10/bin/cqlsh
- Rest of the instructions are available in the python notebooks.
Follow further instructions given in the directories to run this project.