🎆 Only do BLE matching on the phone for unprocessed BLE Scans #1154
Conversation
`ble_sensed_mode` and `ble_sensed_summary` were added to sections and confirmed trips in e-mission/e-mission-server#965. We are going to need `ble_sensed_mode` to determine vehicle info. The section summaries (`ble_sensed_summary`, `cleaned_section_summary` and `inferred_section_summary`) are not used in unprocessed trips currently, but I am adding them so there is less of a gap between composite trips and unprocessed trips
Since we have matching on the server, processed sections will already have BLE sensed modes filled in. We no longer need to query for all BLE scans; only unprocessed ones (ie newer than pipeline end ts). Then while we are constructing unprocessed trips, the list of unprocessed BLE scans will be used to determine BLE sensed modes. We no longer need timelineBleMap and won't treat BLE scans like user inputs anymore. For 'confirmedMode', instead of using timelineBleMap, we will use the ble_sensed_mode of the primary section. New simple function in diaryHelper to deternmine the primary section.
I was just using the latest / master branch to test, but it should really be locked to a release
|
Unit tests passed but codecov failed |
|
We should just wait for ~ 30 mins and retry; I will do that |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1154 +/- ##
==========================================
+ Coverage 26.40% 26.47% +0.06%
==========================================
Files 114 114
Lines 4983 4986 +3
Branches 1064 1071 +7
==========================================
+ Hits 1316 1320 +4
+ Misses 3665 3664 -1
Partials 2 2
Flags with carried forward coverage won't be shown. Click here to find out more.
|
|
e-mission-server is fine to stay on 0.4.3 for now because it doesn't use This can be pushed out to fix the slowness on dfc-fermata, while I work on the other changes in parallel |
|
The pipeline is reset to include |
| start_ts: pipelineRange.start_ts, | ||
| start_ts: pipelineRange.end_ts, |
There was a problem hiding this comment.
note to self, this is not an error, this is the performance optimization!
| const confirmedModeFor = (tlEntry: TimelineEntry) => | ||
| timelineBleMap?.[tlEntry._id.$oid] || labelFor(tlEntry, 'MODE'); | ||
| const confirmedModeFor = (tlEntry: CompositeTrip) => | ||
| primarySectionForTrip(tlEntry)?.ble_sensed_mode || labelFor(tlEntry, 'MODE'); |
There was a problem hiding this comment.
nit, future fix: I think it would be worthwhile for you and Abby to sync up on how this should work across the public dashboard and the phone. The server side data does not have a confirmedMode; maybe it should add one. Also, I am not sure that confirmed_mode is the best name for this, since the ble_sensed_mode has not been confirmed by anybody.
There was a problem hiding this comment.
Agreed, this is not the long term solution.
"Confirmed mode" was something temporary I threw together for the alpha. When I first added it, it actually represented the BLE sensed mode OR the labeled mode, depending on what was available.
For dfc-fermata there are no mode labels and I think it makes more sense to use something like primary_ble_sensed_mode. will coordinate this with Abby once I know what adjustments we are making to the section summaries
I do think something like "confirmed mode" will be useful in the app dashboard and for analysis. I'm envisioning future programs that could theoretically use BLE sensing in addition to mode+purpsoe labels – the BLE sensed trips will not need manual user input since we already "confirmed" the mode when we matched it to a beacon.
When we do viz/ analysis we will want one property that encompasses both cases: i) determined by BLE or ii) manually confirmed
fill in new BLE fields & update types
ble_sensed_modeandble_sensed_summarywere added to sections and confirmed trips in e-mission/e-mission-server#965.We are going to need
ble_sensed_modeto determine vehicle info.The section summaries (
ble_sensed_summary,cleaned_section_summaryandinferred_section_summary) are not used in unprocessed trips currently, but I am adding them so there is less of a gap between composite trips and unprocessed tripsuse only unprocessed BLE scans in label tab
Since we have matching on the server, processed sections will already have BLE sensed modes filled in. We no longer need to query for all BLE scans; only unprocessed ones (ie newer than pipeline end ts).
Then while we are constructing unprocessed trips, the list of unprocessed BLE scans will be used to determine BLE sensed modes.
We no longer need timelineBleMap and won't treat BLE scans like user inputs anymore. For 'confirmedMode', instead of using timelineBleMap, we will use the ble_sensed_mode of the primary section. New simple function in diaryHelper to deternmine the primary section.