-
Notifications
You must be signed in to change notification settings - Fork 25
Moderation
Experts in killer whale bioacoustics collaborate to ensure that predictions from the OrcaHello real time inference system are validated as efficiently as possible. The goal is semi-automated notifications that Southern Resident Killer Whales are audible (and therefore inferred to be present near an Orcasound hydrophone) as rapidly as possible, but also with high reliability (i.e. a false positive rate near zero).
During beta testing in 2020-2023, the moderators were:
- David Bain
- Scott Veirs
- Val Veirs
In 2023, we would like to add more moderators, ideally in a range of time zones around the globe. That way, if most of us are sleeping here in the Pacific Time zone, a domain expert on the other side of the planet might greatly reduce our collective response time! If you'd like to help, please read through the rest of this section and all of this wiki page first. Then send an email to info@orcasound.net or join the Orcasound Slack to volunteer.
If you're an expert at listening for orcas, we'd welcome your help as a moderator. At a minimum, you should be able to differentiate between Southern Resident and Bigg's killer whales, humpbacks, and other common Salish Sea sounds. Ideally, you should also be able to infer which pod is the source of SRKW calls, and be familiar with 5-10 of the most common calls in the SRKW repertoire.
As of late 2022, moderators are notified of recent detections via a Mailman discussion list. You can subscribe to this listserv here -- lists.orcasound.net/listinfo.cgi/orcahello-detection-orcasound.net
To moderate a candidate, you will need to be added as an OrcaHello moderator. An Orcasound administrator with Azure access can help do this for you.
Anyone can receive these notification emails, but many of them are false positives, so if you aren't a moderator you may want to receive a daily digest, or you not want to subscribe at all. If you're interested in confirmed acoustic detections, you may instead want to subscribe via other Orcasound notification channels.
This is the recommended procedure (to maximize reliability and minimize latency):
- As soon as possible after receiving a notification as a moderator, navigate to the OrcaHello moderator UI and authenticate.
- Review the most recent candidate(s), or -- if many false positives occurred recently -- sort to review the highest-confidence recent candidate.
- If you find a true positive, check "Yes"
- Add labels (using the procedure and standards described below)
- Add comments (a brief synopsis or notes beyond what the labels provide)
- Hit submit (which will issue a notification email to subscribers)
- At a minimum for true positives, add pod(s) and any individuals known to be present (e.g. from visual IDs). Ideally, also label any call types that you recognize within the candidate.
- For false positives, or for "unknown" candidates (for which you cannot confidently assign a sound source), add labels for at least one of the model predictions (to characterize what caused the false positives) and any other signals of interest you hear in the candidate. Ideally, any label you add will be consistent with existing cloud of OrcaHello tags and any new labels that you create will aspire to be consistent with the annotation efforts of the broader Orcasound and bioacoustic communities (e.g onomatopoeic labels for humpback sounds).
If you are unsure if a label exists yet for a particular sound, you can browse the Orcasound signal dictionary for OrcaHello candidates (Google spreadsheet with ~100 tags in these broad classes: Biophony, Anthrophony, Geophony, General, and Systemic). To hear examples of particular sounds that have been labeled, browse the Tags section of the OrcaHello Dashboard (takes a while to load, especially if you filter for All time -- which you should to see all available samples for a particular tag ). In the same spreadsheet, you will find other labels used by other annotation efforts, like HALLO and the BC Hydrophone Network, with which Orcasound is coordinating in pursuit of common metadata and annotation standards for orca bioacoustics.
To efficiently improve the OrcaHello system, as a moderator you can create Github issues in the aifororcas-livesystem repo for bugs or other problems you experience. For each issue, please use a label to characterize it for the dev team, e.g. "bug" (something's not working) and/or "moderatorportal."
Simple rules when deciding whether to file an bug/issue:
- If there was ever an "event" with whales of any kind but you did not receive an email - please file an issue.
- Create separate issues for separate hydrophones. For example, if on 11/2/21 there were 2 independent events on Bush Point and Port Townsend and you did not receive an email for either, please file 2 different issues.
- Create separate issues for different events happening on the same day. For example, if there were 2 different events -- one at 9 AM and one at 9 PM -- on the same hydrophone on the same day, file 2 separate issues.
- When in doubt, file separate issues!
It is not useful to file new issues under existing issues unless we are sure that we have root-caused it correctly. This adds to confusion.
What is useful when filing an issue is a full description of the bug with facts and pictures, minus any conjecture.
We'll work on creating an issue template later, but for now consider including, at least:
- Which hydrophone?
- When did the KW event start?
- When did the KW event end?
- What was the nature of the event? Faint calls, killer whales/ humpback?
- Was the container running? (If you have this info?)
You can also create issues with ideas for feature requests or enhancements to the existing moderator portal UI. Please use the "enhancement" label for these issues.