data_analysis_thanwayaAma

Omdena challange

web scrabbing steps:

• range from 100,000 to 999,999

• Group A(الدور الثانى فقط)only 140,000 student

  • https://g12.emis.gov.eg/
  • https://natega.elwatannews.com/
  • http://natega.elbalad.news/
  • http://natega.dostor.org/

• Group B(الدور الاول) 600,000 student

  • https://shbabbek.com/natega

• make http request : https://curlconverter.com/

• making web scrab using (Beauitful Soub library)

• three wayes to make web scrabbing::

1. sync approach:slow way
2. async approach : faster than sync
3. async cocurrent way : most effictive way

• last step save the result into csv file

EDA STEP:

• merge two datset for both first and second trial
• add gender column
• delete interntional student
• do EDA for the data

Visualization:

• power BI or tableau or
• streamlit library in python https://app.powerbi.com/groups/me/reports/a635e7f0-7c31-41d7-9031-8871ef6dea1e?ctid=f349c2fd-fc94-4893-abe4-cfbe7ed52842&pbi_source=linkShare