data_distribution.txt

                    DATA DISTRIBUTION

DOMAIN WISE
    Medical Publication          13,410
    News                         18,389
    Religious texts              48,927
    Miscellaneous                63,896
    Software                     79,132
    Educational content         213,284
    TED Talk                    303,131
    Fictional Books             473,306
    Movies Subtitles            721,174
    Wikipedia                 1,094,248
    Legal Document            1,134,813
                      -----------------
    TOTAL                     4,163,710

RESOURCE WISE 
    Data sources come from the following three categories:
        1. Available parallel corpuses 
            OpenSubtitle,WikiMatrix,Bible-Uedin, Qed, KDE4, Gnome,Ubuntu,Tatoeba,TED2020v1,ELRC: https://opus.nlpl.eu/

        2. Scrape from the internet:
            Websites for Medical data:
                https://yhoctphcm.ump.edu.vn
                http://jmp.huemed-univ.edu.vn
                http://tonghoiyhoc.vn
                http://hoinhikhoavn.com
                http://hoiyhoctphcm.org.vn
                https://jns.vn
                https://jprp.vn
                http://hocvienquany.edu.vn
                https://sinhlyhoc.com.vn
                https://tapchinghiencuuyhoc.vn
                http://tapchi.vienbongquocgia.vn
                http://vienduoclieu.org.vn
                https://vjpm.vn
                http://vjfc.nifc.gov.vn
                https://vjs.ac.vn
                http://vutm.edu.vn
                https://jcmhch.com
                http://www.yhth.vn
                https://sj.ctu.edu.vn
                https://radiology.com.vn
                https://vjol.info.vn
                http://www.vjph.vn

            Others websites:
                https://vietanhsongngu.com
                https://baosongngu.com
                https://sachsongngu.top
                https://thuvienphapluat.vn
                http://vbpl.vn
                http://automation.net
                http://tapchixaydungbxd.vn
                https://duytan.edu.vn
                https://tapchikhcn.haui.edu.vn
                https://tapchivatuyentap.tlu.edu.vn
                http://tapchimoitruong.vn
                https://translations.launchpad.net
                https://translationproject.org
                https://issuu.com/
                https://lyricstranslate.com
                https://www.wikihow.com
                https://d2l.aivivn.com (for 2 books: 'Dive in to Deep Learning' & 'Machine Learning Yearning')

            Subtitles from the following channels on Youtube:
                - Khan Academy 
                - Ted Ed
                - Asap Science
                - Crash courses
                - GCP Grey
                - Veritasium
                - Vsauce

        3. Book downloads and aligned at paragraph-level from
            https://www.dtv-ebook.com
            https://www.pdfdrive.com
            https://vn1lib.org
            https://trinhdinhlinh.com
            https://sachvui.vn
            
        The paragraph-aligning algorithm is described here:
        https://github.com/vietai/SAT/blob/main/dp_algorithm.md