why the loading of files is so slow?

[m1ci]

Hi,
I am loading a 165 MB file and the loading process takes ages. I would expect to be loaded in a matter of seconds.
Am I doing smth wrong or this is a normal?
Here is my code which is very simple:

import rdflib
g = rdflib.Graph()
g.load('data.ttl', format="turtle");

[white-gecko]

The file is from: https://en-word.net/static/english-wordnet-2020.ttl.gz

In my setup it took 4 minutes

time python import.py
python import.py  235,80s user 3,08s system 99% cpu 4:00,02 total

I think our turtle parser is not very good.

To make some testing easier I've changed the code to:

import sys
import rdflib
g = rdflib.Graph()
g.parse(sys.argv[1], format=sys.argv[2])

Also I converted the file with rapper (raptor2-utils) to n-triples

$ time rapper -i turtle wordnet.ttl > wordnet.nt
rapper: Parsing URI file:///tmp/milan/wordnet.ttl with parser turtle
rapper: Serializing with serializer ntriples
rapper: Parsing returned 3103323 triples
rapper -i turtle wordnet.ttl > wordnet.nt  10,66s user 0,98s system 99% cpu 11,653 total
$ time rapper -i turtle -o turtle wordnet.ttl > wordnet_rapper.ttl
rapper: Parsing URI file:///tmp/milan/wordnet.ttl with parser turtle
rapper: Serializing with serializer turtle
rapper: Parsing returned 3103323 triples
rapper -i turtle -o turtle wordnet.ttl > wordnet_rapper.ttl  32,84s user 0,84s system 99% cpu 33,712 total

(The last conversion is just from turtle to turtle, to see if there is any difference between the original serialization and the rapper serialization)

And compared parsing if the n-triples file with the turlte parser as well as with the n-triples parser

$ time python import.py wordnet.nt nt
python import.py wordnet.nt nt  140,03s user 4,35s system 97% cpu 2:28,76 total
$ time python import.py wordnet.nt ttl
python import.py wordnet.nt ttl  320,25s user 3,22s system 99% cpu 5:24,02 total
$ time python import.py wordnet_rapper.ttl ttl 
python import.py wordnet_rapper.ttl ttl  212,88s user 2,66s system 99% cpu 3:35,73 total
$ time python import.py wordnet.ttl ttl       
python import.py wordnet.ttl ttl  231,11s user 2,83s system 99% cpu 3:54,22 total

So we can see a lot of variation between the user and system times

Maybe somebody wants to do this comparison with Cython #1250 .

[white-gecko]

After doing some profiling, I suspect the uri_ref2 and skipSpace methods of the notation3.py

rdflib/rdflib/plugins/parsers/notation3.py

Line 1155 in 9f0f296

def uri_ref2(self, argstr, i, res):

rdflib/rdflib/plugins/parsers/notation3.py

Line 1239 in 9f0f296

def skipSpace(self, argstr, i):

[rchateauneu]

@white-gecko

Yes absolutely. Please see #1262

I have committed changed in notation3.py in my branch, I can send a pull request if you want,
To https://github.com/rchateauneu/rdflib.git
4289f0e..860f985 master -> master