-
Notifications
You must be signed in to change notification settings - Fork 202
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Table decompression #407
Comments
This will require some community discussion - this decompress for apps is being considered for removal as it has been identified as a (possibly) un-required feature. The problem is that the decompress happens to a RAMDISK so you simply trade a slight NVRAM savings for significant increase in RAM use, as the table is now effectively loaded into RAM twice. In many flight systems the RAM is just as constrained, if not more so, than the NVRAM storage. Can you identify a specific use case for this (i.e. a flight computer where the NVRAM is constrained such that this would be a requirement)? |
Note that the decompression functionality is transitioning out of cFE to a library (likely called FS_lib), so the FM app will retain it's capability to decompress (and actually untar is also getting added). The dependency of cFE on decompress is being removed, so "automatic" decompression as part of load isn't supported. The plan is to continue to support decompress with FM, then you can load the app/lib/table. It just won't be automatically done within cFE. The use case for us is to handle transferring many large tables to a cFS based system as tar.gz, then decompressing and untaring on board. A major motivation of moving decompress (and untar) into a library is to make it easier for users to replace with whatever decompression algorithm they'd like. |
There could be two requirements actually,
As an example 270KB CF(CFDP App) (rtems/sparc,stripped) task could be zipped to 70KB (26% compression rate). This is usually better for tables due to repetitive parts like Space Packet Header (ex;sch_def_msgtbl). In projects, I have encountered an average ~1/10 compression rate for tables. Also, ATS and RTS tables loaded daily to some spacecraft, so 1/10 compression is very good for effective operational time usage. On the other hand, implementing this feature with FM will cause at least three constraints or situations in order to load compressed lib, app and tables.
IMHO, FM app should have compression and decompression features. Especially, file compression will be very useful for DS app output files. However, compression & decompression functions can be implemented in OSAL(FS), since some operating systems have these features inherently (posix,rtems). There could be some flag to disable this feature for unsupported RTOS. ES and TBL apps could use these functions to load zipped app, libs, tables, if RTOS supports. FM can use OSAL functions or any other loaded comp-decomp library like FS_lib with hooks or options. |
True (along with the core).
True.
Plan is FM would need to decompress before the load command to ES. ES would not have any linkage to decompress or FM. Could be a scripted set of startup commands. Compressed file system is also a valid approach for operating systems that support it(below the cFS level). There's other approaches outside cFS if there are unique system restrictions (decompress into ram as part of boot allows you to compress everything, etc).
Decompress is being maintained in FM app. Compression (as a library addition) could certainly be considered, and I encourage open source contribution of a cFS/Flight friendly implementation.
Note that built in support is frequently not flight software friendly (recursive, unbounded timing, dynamic memory use, non-portable, etc). There is the shell command from ES already if you want to evoke built in tools on a supporting OS.
Planned approach is to abstract via FS_lib, and maintain a common set of API's and allow projects to easily swap out for other implementations. |
Note there are projects that as part of boot (or any time prior to loading cFS) decompress the entire file system (or parts like image/data/tables/calibration/etc). Typically using whatever built in or custom decompression a project desires. Allows for maximum compression, project customization, and independence from cFS. This is the preferred approach vs decompressing at load within ES. For anything loaded after starting cFS, FS can be used to decompress. |
Is your feature request related to a problem? Please describe.
Compressed tables could save space in NV-RAMs.
Describe the solution you'd like
The decompression feature (implemented already for apps) can be extended to tables.
Describe alternatives you've considered
None.
Additional context
N/A
Requester Info
Onur TURHAN
The text was updated successfully, but these errors were encountered: