2019_group_4_s3555631_s3795748_s3878708

Authors

• Chriss Santi
• Jarvin
• Hillary

Description

This project is about building a platform for scalable computing

Techs

Computing

• Apache Spark

Files System

• Hadoop HDFS

Database

• Cassandra

Browser

• Hue

Web

• Flask
• Html

Deployment

• Local: Docker Compose (StandAlone for Hadoop and Spark)
• Cloud: Docker Swarm or Another technology: yet to be define

Build the project

Local:

Still in development !!!!!

We provide a makefile located in the /docker/compose

• First run the docker-compose file located in /docker/compose
• your app files must be located in the directory app
• build your submit image by running ./buildSubmit.sh located in /docker/scripts
• run the submit image by runnnig the script ./submitJob.sh located in /docker/scripts

Cloud

TODO:

Well manage the environments variables:
(docker/scripts)