This is a repository of different scripts used at University of Lorraine to populate the collections of the OpenLRW API.
All these scripts are made for the Educational community, don't hesitate to contribute!
$ git clone https://github.com/xchopin/openlrw-scripts
$ cd OpenLRW-scripts/bootstrap
$ cp settings.yml.dist settings.yml ; vi settings.yml
To get the libraries you will need to have PIP package manager
-
$ wget https://bootstrap.pypa.io/get-pip.py -O /tmp/get-pip.py ; python /tmp/get-pip.py
-
⚠️ This library requires python-devel and openldap$ pip install python-ldap
-
$ pip install pyyaml
-
⚠️ This library requires mariadb-develyum install mariadb-devel
$ pip install mysqlclient
-
$ pip install openlrw
-v / --verbose : to print all the HTTP calls
-no-mail : to don't send any emails
-h / --help : help documentation
This script will import the users by using the LDAP database.
Clear then populate the collection (recommended for a new OpenLRW instance)
$ python collections/Users/LDAP/import_users.py --reset
Add the new users to the collection
$ python collections/Users/LDAP/import_users.py --update
The new information will be added into the
metadata
attribute of users.
$ python collections/Users/Apogee/import_civic_information.py --file path_to_your_file.csv
Aimed to a CRON use - Format: YYYYmmdd.csv - eg: Students_CS_20191030.csv; Students_Biology_20191030.csv
⚠ The repository of your source files has to be indicated in the settings file, at the civic_information_directory
attribute.
$ python collections/Users/Apogee/import_civic_information.py --file path_to_your_file.csv
CMP;LOGIN;YEAR;GENDER;CHILDREN;HANDICAP;HAS_SCHOLARSHIP;HAS_A_JOB;HAS_ADAPTED_STUDY_PLAN;CITY;BACCALAUREAT_YEAR;BACCALAUREAT_TYPE;BACCALAUREAT_ZIPCODE;BACCALAUREAT_HONOR
AB0;foo;1998;F;0;null;N;O;N;PARIS;2018;ES;054;P
AB0;bar;2000;F;0;null;N;O;N;NANCY;2017;ES;054;AB
AB0;foobar;1991;M;0;null;N;N;O;LYON;2009;S;093;P
This script will import the "logged-in" events (students only) by using log files
$ cd collections/Events/CAS/
$ cat /logs/cas_auth.log | /opt/logstash/bin/logstash --quiet -w10 -f authentication.conf
$ cd collections/Events/CAS/
$ sh authentications.sh
From a timestamp
$ python collections/Events/Moodle/import_events.py -t TIMESTAMP
From a timestamp to another one
$ python collections/Events/Moodle/import_events.py -t TIMESTAMP TIMESTAMP
Check the most recent event stored in the database and insert the newer ones
$ python collections/Events/Moodle/import_events.py -u
This script imports the Moodle classes, it uses a .txt file to allow you to active the class you want (other will be inactive).
It also checks duplicates so you can run it several times for updates.
⚠ The template file is located at data/Classes/active_classes.txt.dist
.
$ cp data/Classes/active_classes.txt.dist data/Classes/active_classes.txt
Then add the class id (one per line), you can add comments with the # character.
In order to set all your classes as active just let the file empty.
$ python collections/Classes/Moodle/import_classes.py
This script will clear the whole Class collection from MongoDB (it keeps the indices though), it performs only one HTTP DELETE request.
$ python collections/Classes/delete_all_classes.py
This script imports the Quizzes
, Active quizzes
and the Grades
from the Moodle database. Checking method is used for HTTP Post so you can use the script for populating and updating your collection.
The condition is made on Moodle's timemodified attribute
$ python collections/Results/Moodle/import_results.py --from TIMESTAMP
The condition is made on Moodle's timemodified attribute
$ python collections/Results/Moodle/import_results.py --from TIMESTAMP --to TIMESTAMP
Import only the results that are older to the last result in MongoDB
$ python collections/Results/Moodle/import_results.py -u
Import the results from an Apogée csv file; it also add LineItem objects if they don't exist. In order to link these new LineItems to a Moodle class you will have a to use a script (more details in LineItems section)
⚠ Two field are required in the settings file
results_directory
where you have to put files with the following format YYYYmmdd.csv eg: export1_20193010.csvlineitems_name_filepath
you have to indicate the absolute path of a csv file for the lineitems
foobar1;2019;3WLAEIMIA;200;initiale;1WIN12;800;UE1-AAAAA1-17.0-ADM
foobar2;2019;3WLAEIMIA;200;initiale;1WIN12;800;UE1-AAAAA1-12.0-ADM
foobar2;2019;3WLOEIMIA;200;initiale;1WIN12;800;UE1-AAAAA3-12.0-null
COD_ELP;LIB_ELP
AAAAA1;Discret Math
AAAAA2;Web applications with JavaEE
AAAAA3;Management
AAAAA4;Internship
Import all the results from the files containing the date of yesterday
$ python collections/Results/Apogee/import_results.py -l
Make a difference between the results of yesterday and the day before (for CRON)
$ python collections/Results/Apogee/import_results.py -u
This script imports the lineItems from Moodle and also checks duplicates, it means you can use it several times for updates.
Import the LineItems
$ python collections/LineItems/Moodle/import_lineitems.py
This script maps the LineItem objects created by the import_results.py
script. For each Moodle class that contains the classCode
key in metadata
it checks if it equals their sourcedId. Ita lso checks duplicates which means you can use it several times for updates.
Map the Apogée LineItems to Moodle classes
$ python collections/LineItems/Apogee/map_classes.py
This script imports the enrollments from Moodle with a timestamp argument (from)
Import the Enrollments
$ python collections/Enrollments/Moodle/import_enrollments.py --from TIMESTAMP
Update the Enrollments
$ python collections/Enrollments/Moodle/import_enrollments.py -u
This script will clear the whole Enrollment collection from MongoDB (it keeps the indices though), it performs only one HTTP DELETE request.
$ python collections/Enrollments/delete_all_enrollments.py
OpenLRW-scripts is made available under the terms of the Educational Community License, Version 2.0 (ECL-2.0).