Custom DB with Custom tables to handle developer data.

[tcgoetz]

I'm looking into adding a custom DB class with a CustomTable class that can take its table layout from a JSON file and can be used to make tables from develop (actually any) fields.

[mkffl]

Hi,
That functionality would be super useful. My main question is around where file processing should happen - in the CustomClass or later.

As per the example share here #75 the json files can be very nested i.e. target fields are buried many layers into the json. Should the file then be processed inside the python class?

I wouldn't mind doing this in sqlite, but I am not sure if it supports json. In other database systems, I'd just cast the document as Json and create a view that explodes/flattens the nested structure.

Apologies if this is a solved problem, I am just starting to get familiar with the code.

[tcgoetz]

SQLite isn't a document DB so putting the whole JSON document into the DB isn't an option.

I'm looking at adding two configurable systems:

1. A DB extension system that allows defining additional DBs and tables.
2. A mapping system that allows mapping of any FIT message/field to any DB/table.

Do you have a example FIT file and the developer fields you would like in a table? The JSON file you linked in the other thread is not accessible. I'd like a FIT file so I can runt he code against it and see it populate a table.

[mkffl]

That makes sense and both options would work, albeit with different trade-offs.

I have shared a dummy version of the json content in the other thread but I will also share fit files.

[tcgoetz]

@mkffl First pass of plugin support with example plugin for HRV is checked in to the develop branch. Please check it out and give feedback. Copy hrv_plugin.py from GarminDB\Plugins to ~/HealthData/Plugins/. and run some of your files through. Look in the activities DB for new tables hrv_records and hrv_sessions.

[mkffl]

I can see the data in the hrv_records and hrv_sessions tables, that's awesome. The summary values are the same as in the portal.
Do you plan to make changes to the plug-in interface? I would like to try and write one for another Garmin app.

[tcgoetz]

I just pushed the next version to develop. Some general cleanup of the plugin code and the hrv plugin now creates a db view.

I have some more to do before I push it to the master branch and release it.

What type of plugin do you want to write? For another App with dev fields that creates activity FIT files? IF so, then your good to start with the understanding that things may still change some. I haven't added the hooks for other types of plugins yet. If this is a different kind of plugin, what are you looking to do?

[mkffl]

Yes, it's another plug-in for sqlite using another custom app that creates FIT files. I am assessing another HRV app that includes more information. The plug-in interface makes sense at first sight but I will probably have to delve into other parts of the code to make it work. If the plug-in interface is relatively stable I may write some documentation for any future usage / other users. Otherwise I will hold off until you merge with the master branch. Thanks

…

On Fri, Jan 8, 2021 at 2:11 AM Tom Goetz ***@***.***> wrote: I just pushed the next version to develop. Some general cleanup of the plugin code and the hrv plugin now creates a db view. I have some more to do before I push it to the master branch and release it. What type of plugin do you want to write? For another App with dev fields that creates activity FIT files? IF so, then your good to start with the understanding that things may still change some. I haven't added the hooks for other types of plugins yet. If this is a different kind of plugin, what are you looking to do? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#95 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACOSDR3VQDJDZQHOZ43CSW3SYZSV7ANCNFSM4VR6TBCA> .

-- Michel Kiffel +44 7918 936 319

[tcgoetz]

I have a couple more things to do:

• add lazy init of plugins so that tables and views are only created for plugins that have matching files
• convert the existing built in support for the FBB Elliptical app to a plugin
• convert the Plugins directory in the source tree to a submodule
• look at how the Plugins directory will work with a binary distribution
• add support for plugins processing lap data

[tcgoetz]

@mkffl It's ready for you to try out. Feedback welcome. Instructions here: https://github.com/tcgoetz/GarminDbPlugins/tree/develop

[mkffl]

It works fine. I no longer need to manually copy the plugin py file into ~/HealthData/Plugins/, probably bc you turned plugins into a submodule. To add another plug-in based on another app, should I take fbb_hrv_plugin.py as a template and update it with what I see in the other app fitfile?

Most of the structure makes sense, with a few excpetions like _application_id, a bytearray that I am not sure where to find it.

[tcgoetz]

To add another plug-in based on another app, should I take fbb_hrv_plugin.py as a template and update it with what I see in the other app fitfile?

Yes. Though depending on what your new plugin is going to do, one of the other plugins may be a better starting point.

Most of the structure makes sense, with a few excpetions like _application_id, a bytearray that I am not sure where to find it.

_application_id is the Connect IQ App id. That’s what I’m using to match files generated by apps to the app plugin for most of the plugins. You can find it by parsing the FIT file and looking at the parse log. Put you fit file in the test file directory and run make test_fit_file.TestFitFile.test_parse_activity in the test directory. See https://github.com/tcgoetz/GarminDbPlugins/tree/develop