Add support for vanilla q3

[brabiega]

At the moment only enchanced logs from OSP are supported.
The difference between log format in OSP and vanilla Q3 lies in:

• log format:

# vanilla q3 (minutes:seconds)
0:00 ClientConnect: 0

# osp (seconds.tenths of seconds)
0.4 ClientConnect: 0

• OSP introduces new event in lgos:

0.0 ServerTime: 20200319210429  21:04:29 (19 Mar 2020)

This is the only way to know when the match actually happened.
It's good to know when a match happens if single log file consists multiple matches.
Also thanks to server time quakestats currently ignores duplicates (e.g. the same log uploaded for parsing multiple times).

So there are two problems to solve for vanilla q3 support:

1. How to ignore matches from log file which was already parsed in the past?
2. What was the date and time when the match happened?

[brabiega]

New q3toql creates match guid from checksum of events of entire match. This should be enough to distinguish baseq3 matches

[juanmcasillas]

Hello, any news on that ? I'm trying to upload a q3 vainilla log ... and get a 200 ok. How to check some debug info?

[brabiega]

Hello, I'd probably have to clean up some dust.
Could you please provide me a log file from a vanilla q3 match?

[juanmcasillas]

Hello;

first of all, I create a password using quakestats set-admin-pwd quake3secret. Then, my settings.py file:

RAW_DATA_DIR = 'mongodb/var/tmp'

# secret token used for minimalistic auth, e.g. post new results
ADMIN_TOKEN = 'quake3secret'

# secret key used to encrypt cookies (see flask session docs)
SECRET_KEY = 'somesecret'

# mongo DB uri
MONGO_URI = 'mongodb://localhost:27017/quakestats'

I do: export QUAKESTATS_SETTINGS=/Archive/Src/quakestats/settings.py, starts mongodb, and run the flask app using: FLASK_APP="quakestats.web"; QUAKESTATS_SETTINGS=pwd/settings.py; twistd web --wsgi quakestats.web.app. The app is up and running good.

In the shell, and try to upload the server.logfile using this:

curl -X POST --form file=@mongodb/var/tmp/server.log --form token=quake3secret  127.0.0.1:8080/api/v2/upload

but I get a 500 internal server error. The file is in the proper place.

2023-04-17T09:37:55+0200 [twisted.python.log#info] "127.0.0.1" - - [17/Apr/2023:07:37:54 +0000] "POST /api/v2/upload HTTP/1.1" 500 290 "-" "curl/7.54.0"

If I try http://127.0.0.1:8080/ I can see the web pages, also I can login using admin/quake3secret.

I use plain q3 server. Thanks for your help,

server.log

[brabiega]

Hey there, would you mind trying version from this branch
https://github.com/brabiega/quakestats/tree/dev-br ?

You need to upload log file with hint baseq3
curl -X POST --form file=@/path/to/your/games.log --form token=adminsecrettoken host:port/api/v2/upload/baseq3

[juanmcasillas]

Hello.

Running it from command line: D:\Software\python39\Scripts\quakestats.exe process-q3-log I:\Games\quake3server\baseq3\server.log baseq3

  File "D:\Software\python39\lib\site-packages\quakestats-1.5.1-py3.9.egg\quakestats\sdk.py", line 89, in process_q3_log
    q3_game = self.q3parser.parse_game_log(game_log)
  File "D:\Software\python39\lib\site-packages\quakestats-1.5.1-py3.9.egg\quakestats\core\q3parser\api.py", line 41, in parse_game_log
    raise Exception(f"Unsupported mod {game_log.mod}")
Exception: Unsupported mod baseq3

using the POST method:

curl -X POST --form file=@I:\Games\quake3server\baseq3\server.log --form token=quake3secret localhost:8080/api/v2/upload/baseq3

<title>500 Internal Server Error</title>
<h1>Internal Server Error</h1>
<p>The server encountered an internal error and was unable to complete your request. Either the server is overloaded or there is an error in the application.</p>

I get the dev-br branch, delete the previous install, and do a python setup.py install

Thanks for your support.

PS I attach a server.log file
server.log

[brabiega]

Hello, are you sure you've installed proper version (from the branch)?

I just tested it using fresh virtualenv + code from the branch and it worked...

(venv) root@quakestats-dev:~/quakestats/run/test-data# quakestats process-q3-log server.log baseq3
2023-05-22 21:19:11,045 [DEBUG|quakestats.system.log]:  Logger initialized
2023-05-22 21:19:11,051 [DEBUG|quakestats.sdk]:  Processing match 0, 5fbffd00ee6f0d11f36666d2248d8da4
2023-05-22 21:19:11,053 [DEBUG|quakestats.sdk]:  Game 5fbffd00ee6f0d11f36666d2248d8da4 ignored
2023-05-22 21:19:11,053 [DEBUG|quakestats.sdk]:  Processing match 1, 2036a7ae16daf873f0b7c4d2c7d1b19c
2023-05-22 21:19:11,275 [INFO|quakestats.sdk]:  Storing game 2036a7ae16daf873f0b7c4d2c7d1b19c in datastore
2023-05-22 21:19:11,276 [DEBUG|quakestats.sdk]:  Processing match 2, 5cf077b578feada2e28ca01e865528c3
2023-05-22 21:19:11,363 [INFO|quakestats.sdk]:  Storing game 5cf077b578feada2e28ca01e865528c3 in datastore

[juanmcasillas]

I will try it this night. Thanks for your support.