ODD CUDA Reconstruction, main branch (2024.04.23.)

[krasznaa]

After the last ~2 weeks of developments, full-chain ODD reconstruction with CUDA is finally here. 🥳

Unfortunately it's not quite as perfect just yet as it should be. 😦

In order to get the code running, I had to generalize traccc::cuda::finding_algorithm a little. It was not exercised with a resizable measurement buffer so far, so it was doing an invalid access when I gave it a resizable container. (I had to deal with exactly this in #545 and #550 already, so it didn't take long to find the issue...)

Other than that, I "just" had to update seq_example_cuda.cpp to add track finding and track fitting to it. 🤔 With this PR's code, I now get:

[bash][atspot01]:traccc > ./out/build/sycl/bin/traccc_seq_example_cuda --detector-file=geometries/odd/odd_geometry_detray.json --use-detray-detector --digitization-file=geometries/odd/odd-digi-geometric-config.json --input-directory=odd/muon100GeV-geant4/ --input-events=3 --compare-with-cpu

Running Full Tracking Chain Using CUDA

>>> Detector Options <<<
  Detector file       : geometries/odd/odd_geometry_detray.json
  Material file       : 
  Surface rid file    : 
  Use detray::detector: yes
  Digitization file   : geometries/odd/odd-digi-geometric-config.json
>>> Input Data Options <<<
  Input data format             : csv
  Input directory               : odd/muon100GeV-geant4/
  Number of input events        : 3
  Number of input events to skip: 0
>>> Clusterization Options <<<
  Target cells per partition: 1024
>>> Track Seeding Options <<<
  None
>>> Track Finding Options <<<
  Track candidates range   : 3:100
  Maximum Chi2             : 30
  Maximum branches per step: 4294967295
>>> Track Propagation Options <<<
  Constraint step size : 3.40282e+38 [mm]
  Overstep tolerance   : -100 [um]
  Mask tolerance       : 15 [um]
  Search window        : 0 x 0
  Runge-Kutta tolerance: 0.0001
>>> Performance Measurement Options <<<
  Run performance checks: no
>>> Accelerator Options <<<
  Compare with CPU results: yes

WARNING: No material in detector
WARNING: No entries in volume finder
Detector check: OK
WARNING: No material in detector
WARNING: No entries in volume finder
Detector check: OK
WARNING: @traccc::io::csv::read_cells: 4500 duplicate cells found in /home/krasznaa/ATLAS/projects/traccc/traccc/data/odd/muon100GeV-geant4/event000000000-cells.csv
===>>> Event 0 <<<===
Number of spacepoints: 31587 (host), 31587 (device)
  Matching rate(s):
    - 98.1701% at 0.01% uncertainty
    - 99.503% at 0.1% uncertainty
    - 99.503% at 1% uncertainty
    - 99.503% at 5% uncertainty
Number of seeds: 5977 (host), 5977 (device)
  Matching rate(s):
    - 54.877% at 0.01% uncertainty
    - 93.4415% at 0.1% uncertainty
    - 98.6783% at 1% uncertainty
    - 99.1467% at 5% uncertainty
Number of track parameters: 5977 (host), 5977 (device)
  Matching rate(s):
    - 71.1226% at 0.01% uncertainty
    - 96.4196% at 0.1% uncertainty
    - 99.3642% at 1% uncertainty
    - 99.6319% at 5% uncertainty
Number of track candidates: 8145 (host), 4388 (device)
  Matching rate(s):
    - 33.2719% at 0.01% uncertainty
    - 61.5715% at 0.1% uncertainty
    - 64.4199% at 1% uncertainty
    - 64.7146% at 5% uncertainty
Number of track states: 8145 (host), 4388 (device)
  Matching rate(s):
    - 0.0245549% at 0.01% uncertainty
    - 1.01903% at 0.1% uncertainty
    - 18.7723% at 1% uncertainty
    - 40.356% at 5% uncertainty
WARNING: @traccc::io::csv::read_cells: 4170 duplicate cells found in /home/krasznaa/ATLAS/projects/traccc/traccc/data/odd/muon100GeV-geant4/event000000001-cells.csv
===>>> Event 1 <<<===
Number of spacepoints: 31923 (host), 31923 (device)
  Matching rate(s):
    - 98.2646% at 0.01% uncertainty
    - 99.5395% at 0.1% uncertainty
    - 99.5395% at 1% uncertainty
    - 99.5395% at 5% uncertainty
Number of seeds: 5829 (host), 5829 (device)
  Matching rate(s):
    - 54.3146% at 0.01% uncertainty
    - 93.3951% at 0.1% uncertainty
    - 98.9707% at 1% uncertainty
    - 99.4167% at 5% uncertainty
Number of track parameters: 5829 (host), 5829 (device)
  Matching rate(s):
    - 70.012% at 0.01% uncertainty
    - 96.1228% at 0.1% uncertainty
    - 99.3996% at 1% uncertainty
    - 99.7427% at 5% uncertainty
Number of track candidates: 7621 (host), 3719 (device)
  Matching rate(s):
    - 33.2371% at 0.01% uncertainty
    - 57.1447% at 0.1% uncertainty
    - 60.5301% at 1% uncertainty
    - 60.8188% at 5% uncertainty
Number of track states: 7621 (host), 3719 (device)
  Matching rate(s):
    - 0.0524866% at 0.01% uncertainty
    - 1.25968% at 0.1% uncertainty
    - 19.2363% at 1% uncertainty
    - 38.6301% at 5% uncertainty
WARNING: @traccc::io::csv::read_cells: 4711 duplicate cells found in /home/krasznaa/ATLAS/projects/traccc/traccc/data/odd/muon100GeV-geant4/event000000002-cells.csv
===>>> Event 2 <<<===
Number of spacepoints: 32088 (host), 32088 (device)
  Matching rate(s):
    - 98.2517% at 0.01% uncertainty
    - 99.5512% at 0.1% uncertainty
    - 99.5512% at 1% uncertainty
    - 99.5512% at 5% uncertainty
Number of seeds: 5789 (host), 5789 (device)
  Matching rate(s):
    - 55.7955% at 0.01% uncertainty
    - 93.4358% at 0.1% uncertainty
    - 98.9636% at 1% uncertainty
    - 99.4645% at 5% uncertainty
Number of track parameters: 5789 (host), 5789 (device)
  Matching rate(s):
    - 71.9468% at 0.01% uncertainty
    - 96.5797% at 0.1% uncertainty
    - 99.4818% at 1% uncertainty
    - 99.8273% at 5% uncertainty
Number of track candidates: 7186 (host), 3698 (device)
  Matching rate(s):
    - 35.4439% at 0.01% uncertainty
    - 59.435% at 0.1% uncertainty
    - 62.6078% at 1% uncertainty
    - 62.9557% at 5% uncertainty
Number of track states: 7186 (host), 3698 (device)
  Matching rate(s):
    - 0.0278319% at 0.01% uncertainty
    - 1.09936% at 0.1% uncertainty
    - 20.4843% at 1% uncertainty
    - 40.8572% at 5% uncertainty
==> Statistics ... 
- read    234267 cells from 33672 modules
- created (cpu)  95598 measurements     
- created (cpu)  95598 spacepoints     
- created (cuda) 95598 spacepoints     
- created  (cpu) 17595 seeds
- created (cuda) 17595 seeds
- found (cpu)    22952 tracks
- found (cuda)   11805 tracks
- fitted (cpu)   22952 tracks
- fitted (cuda)  11805 tracks
==>Elapsed times...
           File reading  (cpu)  451 ms
         Clusterization (cuda)  51 ms
   Spacepoint formation (cuda)  2 ms
         Clusterization  (cpu)  17 ms
   Spacepoint formation  (cpu)  1 ms
                Seeding (cuda)  12 ms
                Seeding  (cpu)  239 ms
           Track params (cuda)  2 ms
           Track params  (cpu)  3 ms
          Track finding (cuda)  484 ms
           Track finding (cpu)  6011 ms
          Track fitting (cuda)  181 ms
           Track fitting (cpu)  2001 ms
                     Wall time  9483 ms
[bash][atspot01]:traccc >

The excellent news is that the code techically runs. 😁 But CUDA finds way fewer tracks than the host does. And even the ones that it finds, have quite different fitted parameters than the ones found by the CPU. 🤔

@beomki-yeo, I hope that some of it is "just" due to some mistake I'm doing with some sort of configuration parameter. But I'll need some help. 😉

[beomki-yeo]

Gpu ans Cpu track finding do not produce the same result. There have been some changes in Cpu track finding only.
I am working on fixing it now so could you wait?

[krasznaa]

Gpu ans Cpu track finding do not produce the same result. There have been some changes in Cpu track finding only. I am working on fixing it now so could you wait?

Sure, I'll keep this as a draft. But you know... chop-chop... 😛

[beomki-yeo]

I don't know how long it is going to take, just go ahead.

[krasznaa]

Going ahead with this, to simplify the testing of #556 a little.