push stats on demand

[adRn-s]

Adressing #89

Not tested.

[WardDeb]

I left two specific comments in the code, but have two more general points:

How I'd understand it:
One runs BRB --stats --fid path/to/flowcell
-> don't run BRB.PushButton.GetResults, only phoneHome or telegraphHome where appropriate
One runs BRB without --stats / --fid and there is an unprocessed flow cell
-> run workflow as it's implemented now (GetResults workflow)
One runs BRB without --stats / --fid and there is no unprocessed flow cell
-> sleep

I only see a query to parkour under the stats is not None arm, am I misunderstanding the implementation maybe ?

Secondly, It should be easy to test a stats/fid implementation by comparing the json to the one pushed to parkour previously, it's probably safer to test this before just pushing these changes over.

[adRn-s]

You're not misunderstanding the implementation. So, yes. BRB without --stats, should be the original.

The idea, is that this one is there, left running as always. On top of it, no matter what that process is doing (if there's a new flowcell, or not, and it's just sleeping) we could run BRB with --stats and a FCID, to push those stats to Parkour.

---

Secondly, It should be easy to test a stats/fid implementation by comparing the json to the one pushed to parkour previously, it's probably safer to test this before just pushing these changes over.

I will try that! 👍🏽

[adRn-s]

The newly added functionality works. I didn't compare the json file(s) pushed to Parkour, but saw the emails in my inbox and they look coherent, and since we didn't change how the info is parsed or anything else; it should all be set.

The only thing left, perhaps, is testing that the old functionality is there without any new issues. It should be the case, but still.

---

As a side note, I have modified the "external data" print statement in PushButton because I was puzzled by it. I can't say that it's anything clearer now... For example, BigRedButton -c /home/pipegrp/adRn/brb.ini --stats --fcid="BH7WMYDRX5" outputs:

Logging into: /home/pipegrp/adRn/logs/brb/240715_A00931_0732_BH7WMYDRX5_lanes_1_2.log
GetResults with ignore=True, 3211_Holec_Hilgers is external data.
GetResults with ignore=True, 3237_Tripathy_Akhtar is external data.
GetResults with ignore=True, 3241_Förtsch_Akhtar is external data.

[WardDeb]

I still fail to understand the logic here, sorry..

stats not set -> only set a log file
stats set -> infer some lane status and set some other paths ?

Afterwards all the data gets processed anyway. For me this is effectively the same as running BRB twice ( with the added disadvantage that we are hardcoded for two lanes now).

If you don't have FID as an argument, but an actual path to a processed flowcell (which might be split per lane or not, but at least you don't have to infer it here), and subsequently have:
if not stats:
process_data
else:
if analysis was actually done already (because analysis worked, parkour was just dead) -> run phoneHome
else no analysis was actually done (because non-std libtype, or external data) -> run telegraphHome

[WardDeb]

Logging into: /home/pipegrp/adRn/logs/brb/240715_A00931_0732_BH7WMYDRX5_lanes_1_2.log
GetResults with ignore=True, 3211_Holec_Hilgers is external data.
GetResults with ignore=True, 3237_Tripathy_Akhtar is external data.
GetResults with ignore=True, 3241_Förtsch_Akhtar is external data.

My guess is the actual path constructed from the flow cells is wrong (does lanes_1_2 even exist, or we only have lanes_1 and lanes_2 as separate dirs). Perhaps the other comment clarifies some things, otherwise we could discuss this in person.