Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Discover action types and pass Sensor through action call #109

Merged
merged 91 commits into from
Jan 27, 2024
Merged
Show file tree
Hide file tree
Changes from 66 commits
Commits
Show all changes
91 commits
Select commit Hold shift + click to select a range
5e28250
sigan monitor
dboulware Jan 8, 2024
bb42996
Move sigan monitor to status.
dboulware Jan 8, 2024
588fb6f
fix signal_analyzer_monitor.
dboulware Jan 8, 2024
eba51a9
Create core module to initialize the StatusMonitor and SiganMonitor
dboulware Jan 9, 2024
8a3b2c2
typo fix.
dboulware Jan 9, 2024
71da125
rename initialization import.
dboulware Jan 9, 2024
b2022e4
add test for sigan registration.
dboulware Jan 9, 2024
9a7309c
Increment version.
dboulware Jan 9, 2024
5909cd6
Restore functionality. Add gps registration and tests.
dboulware Jan 10, 2024
f2bce4c
Remove sigan and gps registration signals and core module. Update act…
dboulware Jan 11, 2024
5fcb348
remove register sigan and gps signal use.
dboulware Jan 11, 2024
2e489c2
fix import of status_monitor
dboulware Jan 11, 2024
027e23a
remove old tests and fix test_status_handler.
dboulware Jan 11, 2024
5327022
fix test_status_handler
dboulware Jan 11, 2024
42d2767
remove register sigan signal refs.
dboulware Jan 11, 2024
023f554
pass gps and sigan to actions call method instead of instantiating ac…
dboulware Jan 12, 2024
d827a57
Remove sigan from discover
dboulware Jan 12, 2024
a9b0b78
remove sigan and gps from action constructors.
dboulware Jan 12, 2024
46ff311
remove erroneous ref to sigan in logger action.
dboulware Jan 12, 2024
ace3525
remove sigan and gps from sea action init.
dboulware Jan 12, 2024
faba66f
remove gps/sigan ref from action inits.
dboulware Jan 12, 2024
af98a15
fix tests.
dboulware Jan 12, 2024
3960349
run pre-commit
dboulware Jan 12, 2024
a727559
fixed tests.
dboulware Jan 13, 2024
6843b97
add missing sensor class.
dboulware Jan 13, 2024
eec0a47
add initialization code.
dboulware Jan 13, 2024
a054081
move initialization utilities back to scos-sensor.
dboulware Jan 13, 2024
fbc3400
remove erroneous self.sensor_definition
dboulware Jan 13, 2024
f2ce96e
comment out preselector settings
dboulware Jan 13, 2024
37b67a7
Remove calibration initialization. Update sigan constructor to option…
dboulware Jan 13, 2024
bd0689f
Don't import cals from actions.calibration
dboulware Jan 13, 2024
f4371cc
add is_default property to Calibration. Remove cal imports in y_facto…
dboulware Jan 13, 2024
6f6505e
Move status_registration_handler to scos-sensor. Add register_signal_…
dboulware Jan 13, 2024
63bf60a
remove import
dboulware Jan 13, 2024
e174253
use dataclass for is_default.
dboulware Jan 14, 2024
7184c4b
set is_default in Calibration creation.
dboulware Jan 14, 2024
1030344
correct sensor_definition reference in Sensor.
dboulware Jan 14, 2024
58d7fff
correct refs to signal_analyzer in actions.
dboulware Jan 14, 2024
8f17a92
add sensor in call to configure preselector.
dboulware Jan 14, 2024
d23ea9f
use sensor capabilities to set location in data product action.
dboulware Jan 14, 2024
4d4fb43
correct ref to switches and location.
dboulware Jan 14, 2024
4abce51
Add register sensor signal. Use sensor.location in sigmf metadata in …
dboulware Jan 14, 2024
fae07d4
pass sensor to global sensor metadata method.
dboulware Jan 14, 2024
0dba721
pass sensor to capture_diagnostics
dboulware Jan 14, 2024
a9bb6b3
add type hints to Sensor
dboulware Jan 14, 2024
946a14a
add type hints to SignalAnalyzerInterface
dboulware Jan 14, 2024
8410505
fix dict type hints and update tests.
dboulware Jan 14, 2024
a2cf95c
Update readme and add type hint for Sensor to the __call__ method.
dboulware Jan 15, 2024
a8107f4
Used passed in sensor in create_global_sensor_metadata
dboulware Jan 15, 2024
b3961b9
update calibration action for sensor changes.
dboulware Jan 15, 2024
d569dd9
pass preselector to get_temperature and get_linear_enr. Update type h…
dboulware Jan 15, 2024
a42c536
debugging
dboulware Jan 15, 2024
3538051
store calibration file path in Calibration.
dboulware Jan 15, 2024
3c020d6
Use SCOS_SENSOR_GIT_TAG from environment.
dboulware Jan 15, 2024
d15a7ef
set sigan and sensor calibration files from env.
dboulware Jan 15, 2024
3128ba8
Don't use django settings.
dboulware Jan 15, 2024
5f4b3cc
fix tests.
dboulware Jan 15, 2024
25ce95d
Remove SENSOR_CALIBRATION_FILE And SIGAN_CALIBRATION_FILE
dboulware Jan 15, 2024
2661313
add MOCK_SIGAN_RANDOM setting.
dboulware Jan 16, 2024
80784aa
Consistent naming and README updates
aromanielloNTIA Jan 16, 2024
165efe5
add and update action base class type hints
aromanielloNTIA Jan 16, 2024
07d0201
remove unused testing code
aromanielloNTIA Jan 16, 2024
acd3e68
add missing json import
aromanielloNTIA Jan 16, 2024
d529c64
remove unused SENSOR_DEFINITION_FILE setting
aromanielloNTIA Jan 16, 2024
54567d3
add and update action class type hints and imports
aromanielloNTIA Jan 16, 2024
719d1b1
Update sample_debug.py
aromanielloNTIA Jan 16, 2024
ecd054c
add missing parameter to docstring
aromanielloNTIA Jan 16, 2024
df4589b
remove unused gps parameter
aromanielloNTIA Jan 16, 2024
71a2007
update type hints and run isort
aromanielloNTIA Jan 16, 2024
62ed6a2
more specific type hint for power_cycle_sigan
aromanielloNTIA Jan 16, 2024
489f829
remove redundant call to Path()
aromanielloNTIA Jan 16, 2024
fb3e310
update type hints
aromanielloNTIA Jan 16, 2024
5d32721
Remove sensor parameter from configure_preselector and get_sigmf_buil…
dboulware Jan 16, 2024
a562915
remove dead code.
dboulware Jan 16, 2024
cd1a205
fix get_sigmf_builder call in test action
aromanielloNTIA Jan 16, 2024
cd74577
Remove sensor from call to get_sigmf_builder.
dboulware Jan 16, 2024
fabccfe
Merge branch 'discover_action_types' of https://github.com/NTIA/scos-…
dboulware Jan 16, 2024
8a4f69e
Remove StatusMonitor (functionality replaced with singleton in scos-s…
dboulware Jan 17, 2024
e35274f
Remove status package and move start_time into Sensor.
dboulware Jan 17, 2024
e5f5ac2
Fix tests.
dboulware Jan 17, 2024
83f7dc9
Use property setters in Sensor init.
dboulware Jan 17, 2024
d8d45e4
Add test_sensor. Remove dead code.
dboulware Jan 17, 2024
ee5ebb4
Revert unnecessary change in MockSignalAnalyzer recompute_sensor_cali…
dboulware Jan 17, 2024
d3c169d
Remove DEFAULT_CALIBRATION_FILE from settings.
dboulware Jan 17, 2024
027fde8
Add optional switches dictionary to SignalAnalyzerInterface
dboulware Jan 18, 2024
4d3e5b5
update type hint in SignalAnalyzerInterface
dboulware Jan 18, 2024
acfdb8f
pass switches to power cycle sigan.
dboulware Jan 18, 2024
46c9f61
add load_switches
dboulware Jan 18, 2024
0e82996
Remove load switches.
dboulware Jan 21, 2024
0581b95
Default signal_analyzer to None and add optional switches to MockSign…
dboulware Jan 22, 2024
74b3760
Require signal analyzer and capabilities in Sensor constructor. Add d…
dboulware Jan 26, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
62 changes: 30 additions & 32 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,28 +36,32 @@ architecture.

## Overview of Repo Structure

- `scos_actions/actions`: This includes the base Action class, signals, and the following
- `scos_actions/actions`: This includes base Action classes and the following
common action classes:
- `acquire_single_freq_fft`: performs FFTs and calculates mean, median, min, max, and
sample statistics at a single center frequency.
- `acquire_single_freq_tdomain_iq`: acquires IQ data at a single center frequency.
- `acquire_stepped_freq_tdomain_iq`: acquires IQ data at multiple center frequencies.
- `calibrate_y_factor`: performs calibration using the Y-Factor method.
- `sync_gps`: gets GPS location and syncs the host to GPS time
- `monitor_sigan`: ensures a signal analyzer is available and is able to maintain a
connection to the computer.
- `sync_gps`: gets GPS location and syncs the host to GPS time
- `scos_actions/calibration`: This includes an interface for sensor calibration data
- `scos_actions/configs/actions`: This folder contains the YAML files with the parameters
used to initialize the actions described above.
- `scos_actions/discover`: This includes the code to read YAML files and make actions
available to scos-sensor.
- `scos_actions/hardware`: This includes the signal analyzer interface and GPS interface
used by the actions and the mock signal analyzer. The signal analyzer interface is
intended to represent universal functionality that is common across all signal
analyzers. The specific implementations of the signal analyzer interface for
particular signal analyzers are provided in separate repositories like
available to SCOS Sensor.
- `scos_actions/hardware`: This includes the signal analyzer and GPS interfaces used by
actions and the mock signal analyzer. The signal analyzer interface represents functionality
common to all signal analyzers. Specific implementations of the signal analyzer interface
for particular signal analyzers are provided in separate repositories like
[scos-usrp](https://github.com/NTIA/scos-usrp).
- `scos_actions/metadata`: This includes the `SigMFBuilder` class and related metadata
structures used to generate [SigMF](https://github.com/SigMF/SigMF)-compliant metadata.
- `scos_actions/signal_processing`: This contains various common signal processing
routines which are used in actions.
- `scos_actions/status`: This provides a class to register objects with the SCOS Sensor
status endpoint.

## Running in SCOS Sensor

Expand Down Expand Up @@ -176,14 +180,11 @@ actions = {
"sync_gps": SyncGps(gps),
}

yaml_actions, yaml_test_actions = init(sigan=sigan, yaml_dir=ACTION_DEFINITIONS_DIR)
yaml_actions, yaml_test_actions = init(yaml_dir=ACTION_DEFINITIONS_DIR)

actions.update(yaml_actions)
```

Pass the implementation of the signal analyzer interface and the directory where the
YAML files are located to the `init` method.

If no existing action class meets your needs, see [Writing Custom Actions](
#writing-custom-actions).

Expand Down Expand Up @@ -300,11 +301,13 @@ You're done.
sensor owner wants the sensor to be able to *do*. At a lower level, they are simply
Python classes with a special method `__call__`. Actions use [Django Signals](
<https://docs.djangoproject.com/en/3.1/topics/signals/>) to provide data and results to
scos-sensor.
SCOS Sensor.

Start by looking at the [`Action` base class](scos_actions/actions/interfaces/action.py).
It includes some logic to parse a description and summary out of the action class's
docstring, and a `__call__` method that must be overridden.
docstring, and a `__call__` method that must be overridden. Actions are only instantiated
with parameters. The signal analyzer implementation will be passed to the action at
execution time through the __call__ method's Sensor object.

A new custom action can inherit from the existing action classes to reuse and build
upon existing functionality. A [`MeasurementAction` base class](scos_actions/actions/interfaces/measurement_action.py),
Expand All @@ -318,25 +321,21 @@ enables SCOS Sensor to do something with the results of the action. This could r
from storing measurement data to recycling a Docker container or to fixing an unhealthy
connection to the signal analyzer. You can see the available signals in
[`scos_actions/signals.py`](scos_actions/signals.py).
The following signals are currently offered:
The following signals are currently offered for actions:

- `measurement_action_completed` - signal expects task_id, data, and metadata
- `location_action_completed` - signal expects latitude and longitude
- `trigger_api_restart` - triggers a restart of the API docker container (where
scos-sensor runs)
SCOS Sensor runs)

New signals can be added. However, corresponding signal handlers must be added to
scos-sensor to receive the signals and process the results.
SCOS Sensor to receive the signals and process the results.

##### Adding custom action to SCOS Actions

A custom action meant to be re-used by other plugins can live in SCOS Actions. It can
be instantiated using a YAML file, or directly in the `actions` dictionary in the
`discover/__init__.py` module. This can be done in SCOS Actions with a mock signal
analyzer. Plugins supporting other hardware would need to import the action from
SCOS Actions. Then it can be instantiated in that plugin’s actions dictionary in its
discover module, or in a YAML file living in that plugin (as long as its discover
module includes the required code to parse the YAML files).
`discover/__init__.py` module.

##### Adding system or hardware specific custom action

Expand All @@ -349,7 +348,7 @@ above.
### Supporting a Different Signal Analyzer

[scos_usrp](https://github.com/NTIA/scos-usrp) adds support for the Ettus B2xx line of
signal analyzers to `scos-sensor`. Follow these instructions to add support for
signal analyzers to SCOS Sensor. Follow these instructions to add support for
another signal analyzer with a Python API.

- Create a new repository called `scos-[signal analyzer name]`.
Expand All @@ -372,24 +371,23 @@ another signal analyzer with a Python API.
custom actions that are unique to the hardware. See [Adding Actions](#adding-actions)
subsection above.
- In the new repository, add a `discover/__init__.py` file. This should contain a
dictionary called `actions` with a key of action name and a value of action object.
dictionary called `actions` with keys of action names and values of action instances.
If the repository also includes new action implementations, it should also expose a
dictionary named `action_classes` with keys of actions names and values of action classes.
You can use the [init()](scos_actions/discover/__init__.py) and/or the
[load_from_yaml()](scos_actions/discover/yaml.py) methods provided in this repository
to look for YAML files and initialize actions. These methods allow you to pass your
new signal analyzer object to the action's constructor. You can use the existing
to look for YAML files and initialize actions. You can use the existing
action classes [defined in this repository](scos_actions/actions/__init__.py) or
[create custom actions](#writing-custom-actions). If the signal analyzer supports
calibration, you should also add a `get_last_calibration_time()` method to
`discover/__init__.py` to enable the status endpoint to report the last calibration
time.
[create custom actions](#writing-custom-actions).

If your signal analyzer doesn't have a Python API, you'll need a Python wrapper that
calls out to your signal analyzer's available API and reads the samples back into
Python. Libraries such as [SWIG](http://www.swig.org/) can automatically generate
Python wrappers for programs written in C/C++.

The next step in supporting a different signal analyzer is to create a class that
inherits from the [GPSInterface](scos_actions/hardware/gps_iface.py) abstract class.
inherits from the [GPSInterface](scos_actions/hardware/gps_iface.py) abstract class if
the signal analyzer includes GPS capabilities.
Then add the `sync_gps` and `monitor_sigan` actions to your `actions` dictionary,
passing the gps object to the `SyncGps` constructor, and the signal analyzer object to
the `MonitorSignalAnalyzer` constructor. See the example in the [Adding Actions
Expand All @@ -407,7 +405,7 @@ specific drivers are required for your signal analyzer, you can attempt to link
within the package or create a docker image with the necessary files. You can host the
docker image as a [GitHub package](
<https://docs.github.com/en/free-pro-team@latest/packages/using-github-packages-with-your-projects-ecosystem/configuring-docker-for-use-with-github-packages>
). Then, when running scos-sensor, set the environment variable
). Then, when running SCOS Sensor, set the environment variable
`BASE_IMAGE=<image tag>`.

## License
Expand Down
9 changes: 5 additions & 4 deletions sample_debug.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@

from scos_actions.actions.acquire_single_freq_fft import SingleFrequencyFftAcquisition
from scos_actions.hardware.mocks.mock_sigan import MockSignalAnalyzer
from scos_actions.hardware.sensor import Sensor
from scos_actions.signals import measurement_action_completed

parameters = {
Expand All @@ -17,6 +18,7 @@
"nffts": 300,
"nskip": 0,
"classification": "UNCLASSIFIED",
"calibration_adjust": False,
}
schedule_entry_json = {
"name": "test_m4s_multi_1",
Expand Down Expand Up @@ -50,10 +52,9 @@ def callback(sender, **kwargs):

measurement_action_completed.connect(callback)

action = SingleFrequencyFftAcquisition(
parameters=parameters, sigan=MockSignalAnalyzer(randomize_values=True)
)
action(schedule_entry_json, 1)
action = SingleFrequencyFftAcquisition(parameters)
sensor = Sensor(signal_analyzer=MockSignalAnalyzer(randomize_values=True))
action(sensor, schedule_entry_json, 1)
print("metadata:")
print(json.dumps(_metadata, indent=4))
print("finished")
2 changes: 1 addition & 1 deletion scos_actions/__init__.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "7.1.0"
__version__ = "8.0.0"
Loading