Faster retrieval of component control statements, show control titles #1570

gregelin · 2021-04-23T11:30:22Z

Retrieve component control statement much faster by getting
just related catalog_control_as_dict for statement. Also
add statement property to get control title. Display control title
on component control pages.

Important performance improvement!

Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages.

system_tags aren't used anywhere

davidpofo

lgtm

* Filter SAR Deployment form dropdown to System's deployments * Filter SAR Deployment form dropdown to System's deployments * Create a simple SAR server to generate test data Create a simple System Assessment Result web server generating JSON data object listing assessment summary results that can be added to a system's assessment. This provides a synthentic provider of SAR results data from which we can test the UI for consuming SAR data. * Better SAR displays, wrap SAR item data Polish SAR summary page - use details tag, other improvements. Polish SAR list page - include deployment name. Wrap SAR test item to start to make more realistic SAR data. Display assessment name in Assessment model admin list. * Add sar_etl.py example middlware app * Use project-base.html in deployment detail page. Misc improvements to sar generator and middleware. Include description value for SAR wrapper. * Remove hardcoded data from SAR middleware pipeline Remove hardcoded values of system id and deployment id from SAR middleware pipeline. Pass as many values as possible in SAR wrapper object. Properly handle UUID to string and back again Post assessment to system without a deployment ID Display project id, system id on project pages. * Support Wazuh SCA results in SAR pipeline Add `tools/simple_sar_server/wazuh_etl.py` to support Wazuh SCA results in SAR pipeline. Display all summary values of a SAR result for each inventory item using a loop and table. * Improve styling of inventory-item results display * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 3 * attempting multiple catalog output * dont want install py changes * Add clear links for forgot password, change password * Update stub_app for input, components Update stub_app used by complianca_app command or generating compliance app to include "input" and "output" section; and to have folders for templates, utils, and components. Add in exceprtion for integrity error to eventually support gracefully handling. * Closes #1547 Display impact level on project page. Implement impact level as a statement about a system of type "fisma_impact_level" with imoact level in body. This continues idea that statements are factual observations about a system. Statement must be associated with system.root_element and be consumed by root_element. * note as with any change to the document data to be rendered need to press the refresh documents button to see changes. Removing cache removal function. * spelling * Need default of None if there are no control catalogs present * also need try/except for test_render_markdown_to_text since there are no projects for this test. * ISPGBSS-208 Added new fields to system settings and created migrations * make sure to send catalog_key for sid_class form_value * checking for statement sid class * Add tests for fisma_impact_level_display * need to filter statements by sid class as well. some optimization on catalog retrieval * Add StatementTypeEnum Enumarate different type of statements * modularizing some common data retrievals. * Added empty my_component.json file * only save once at the end of all possible changes * added better logging to find bad templates faster * Report OSCAL component schema validation error to standout Provide better error reporting on import component schema validation. Report actual validation error to standout. * Merge v0.9.3.4 into develop * ISPGBSS-208 ISPGBSS-261 Refactored previous changes to implement timeout through environment variables instead of db based setting. * implemented a full example of multiple catalogs in a new govready-qfiles-startpack app source multi-catalog. * remove extra prints * put "Manage Import Records" button last Rearrange Create | Import | Manage component buttons * Immediately assign change project perms to user starting project This fixes a bug where non-admin users were not assigned permissions to change a project started by the user until after several project modification steps were performed. The fix assigns non-admin project change permissions right after project and system is created and before further actions are made to project, such as setting baseline controls. * Update CHANGELOG * Set system fisma_impact_level smt as question action with new set/get methods Set system fisma_impact_level root_element statement as part of question action to set baseline. Since baselines could have more names than just "Low", "Moderate", and "High", we only set fisma_impact_level if baseline is one of those three. This might need to improve in the future and make setting fisma_impact_level a separate question action from setting baselines. Created set/get methods for fisma_impact_level in System model to make sure only one statement for the system with statement_type of fisma_impact_level existed. Also fixed POAM stat counting to only count POAMs associated with current system. Existing POAM count was not filtering on current system. * Update CHANGELOG * Update CHANGELOG.md spelling * Update navbar.html * Update project.html all caps no brakes * fixing fisma display test (#1566) * fixes a few typos in HTML templates (#1565) * fix filtering for components on adding a component search box. (#1567) * Docker Local Development implementation (#1555) * wip * Local development rework based on deployments repo * adding +x * added action for running and some clean up and docs * Added docker engine check * added another message at end * more faq * changelog * added selenium & chrome support * document quick fix * regression * readme change * added var * Update dev_env/run.py Co-authored-by: davidpofo <dampofo@umd.edu> * space Co-authored-by: davidpofo <dampofo@umd.edu> * ISPGBSS-208 ISPGBSS-261 Added test for session setting ping * Fixed a bug where ssh files weren't brought over on restart * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Faster retrieval of component control statements, show control titles (#1570) * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Update element_detail_tabs.html system_tags aren't used anywhere Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * enums need to match case of value. adding smt id to logs * use enums' value to cover the expected values everye where for statement type. * remove asserts comment * ISPGBSS-208 ISPGBSS-261 Used requirements_txt_updated.sh to update requirements.txt as per PR review. * Display control catalog guidance text in `details` tag Display control catalog guidance text in `details` tag next to component control implementation statements. Also clean up some html on component detail pages, move some styles to main `govready-q.css` file. * Search component library by tag, make component tags clickable * Better notifification when project smt differs from certified Better notify users when project implementation statement differs from certified by displaying notice in third column control detail pages. Improve language notifying users that project implementation statement differs from certified. Only difference notice is clickable now. * ISPGBSS-261 Fixed issues related to PR review. * Da/last min develop (#1582) * Add new options to runner.py environment. PR 1572 missed changelog. also change log for PR 1535 was missed for the last release * also mention nplusone. * commenting out session security for now until issue with js is fixed. * unnecessary return * fixing statement types * a few more statement type fixes * removing feature that is a WIP * adding some time to avoid the erroneous failures. * more sleep * skipping bad integration tests. (#1585) * update version and changelog Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Greg Elin <gregelin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com>

* Release/v0.9.3.5 (#1586) * Filter SAR Deployment form dropdown to System's deployments * Filter SAR Deployment form dropdown to System's deployments * Create a simple SAR server to generate test data Create a simple System Assessment Result web server generating JSON data object listing assessment summary results that can be added to a system's assessment. This provides a synthentic provider of SAR results data from which we can test the UI for consuming SAR data. * Better SAR displays, wrap SAR item data Polish SAR summary page - use details tag, other improvements. Polish SAR list page - include deployment name. Wrap SAR test item to start to make more realistic SAR data. Display assessment name in Assessment model admin list. * Add sar_etl.py example middlware app * Use project-base.html in deployment detail page. Misc improvements to sar generator and middleware. Include description value for SAR wrapper. * Remove hardcoded data from SAR middleware pipeline Remove hardcoded values of system id and deployment id from SAR middleware pipeline. Pass as many values as possible in SAR wrapper object. Properly handle UUID to string and back again Post assessment to system without a deployment ID Display project id, system id on project pages. * Support Wazuh SCA results in SAR pipeline Add `tools/simple_sar_server/wazuh_etl.py` to support Wazuh SCA results in SAR pipeline. Display all summary values of a SAR result for each inventory item using a loop and table. * Improve styling of inventory-item results display * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 3 * attempting multiple catalog output * dont want install py changes * Add clear links for forgot password, change password * Update stub_app for input, components Update stub_app used by complianca_app command or generating compliance app to include "input" and "output" section; and to have folders for templates, utils, and components. Add in exceprtion for integrity error to eventually support gracefully handling. * Closes #1547 Display impact level on project page. Implement impact level as a statement about a system of type "fisma_impact_level" with imoact level in body. This continues idea that statements are factual observations about a system. Statement must be associated with system.root_element and be consumed by root_element. * note as with any change to the document data to be rendered need to press the refresh documents button to see changes. Removing cache removal function. * spelling * Need default of None if there are no control catalogs present * also need try/except for test_render_markdown_to_text since there are no projects for this test. * ISPGBSS-208 Added new fields to system settings and created migrations * make sure to send catalog_key for sid_class form_value * checking for statement sid class * Add tests for fisma_impact_level_display * need to filter statements by sid class as well. some optimization on catalog retrieval * Add StatementTypeEnum Enumarate different type of statements * modularizing some common data retrievals. * Added empty my_component.json file * only save once at the end of all possible changes * added better logging to find bad templates faster * Report OSCAL component schema validation error to standout Provide better error reporting on import component schema validation. Report actual validation error to standout. * Merge v0.9.3.4 into develop * ISPGBSS-208 ISPGBSS-261 Refactored previous changes to implement timeout through environment variables instead of db based setting. * implemented a full example of multiple catalogs in a new govready-qfiles-startpack app source multi-catalog. * remove extra prints * put "Manage Import Records" button last Rearrange Create | Import | Manage component buttons * Immediately assign change project perms to user starting project This fixes a bug where non-admin users were not assigned permissions to change a project started by the user until after several project modification steps were performed. The fix assigns non-admin project change permissions right after project and system is created and before further actions are made to project, such as setting baseline controls. * Update CHANGELOG * Set system fisma_impact_level smt as question action with new set/get methods Set system fisma_impact_level root_element statement as part of question action to set baseline. Since baselines could have more names than just "Low", "Moderate", and "High", we only set fisma_impact_level if baseline is one of those three. This might need to improve in the future and make setting fisma_impact_level a separate question action from setting baselines. Created set/get methods for fisma_impact_level in System model to make sure only one statement for the system with statement_type of fisma_impact_level existed. Also fixed POAM stat counting to only count POAMs associated with current system. Existing POAM count was not filtering on current system. * Update CHANGELOG * Update CHANGELOG.md spelling * Update navbar.html * Update project.html all caps no brakes * fixing fisma display test (#1566) * fixes a few typos in HTML templates (#1565) * fix filtering for components on adding a component search box. (#1567) * Docker Local Development implementation (#1555) * wip * Local development rework based on deployments repo * adding +x * added action for running and some clean up and docs * Added docker engine check * added another message at end * more faq * changelog * added selenium & chrome support * document quick fix * regression * readme change * added var * Update dev_env/run.py Co-authored-by: davidpofo <dampofo@umd.edu> * space Co-authored-by: davidpofo <dampofo@umd.edu> * ISPGBSS-208 ISPGBSS-261 Added test for session setting ping * Fixed a bug where ssh files weren't brought over on restart * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Faster retrieval of component control statements, show control titles (#1570) * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Update element_detail_tabs.html system_tags aren't used anywhere Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * enums need to match case of value. adding smt id to logs * use enums' value to cover the expected values everye where for statement type. * remove asserts comment * ISPGBSS-208 ISPGBSS-261 Used requirements_txt_updated.sh to update requirements.txt as per PR review. * Display control catalog guidance text in `details` tag Display control catalog guidance text in `details` tag next to component control implementation statements. Also clean up some html on component detail pages, move some styles to main `govready-q.css` file. * Search component library by tag, make component tags clickable * Better notifification when project smt differs from certified Better notify users when project implementation statement differs from certified by displaying notice in third column control detail pages. Improve language notifying users that project implementation statement differs from certified. Only difference notice is clickable now. * ISPGBSS-261 Fixed issues related to PR review. * Da/last min develop (#1582) * Add new options to runner.py environment. PR 1572 missed changelog. also change log for PR 1535 was missed for the last release * also mention nplusone. * commenting out session security for now until issue with js is fixed. * unnecessary return * fixing statement types * a few more statement type fixes * removing feature that is a WIP * adding some time to avoid the erroneous failures. * more sleep * skipping bad integration tests. (#1585) * update version and changelog Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Greg Elin <gregelin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> * Fix for handling unset SESSION timeout params (#1587) * Fix to handle unset SESSION timeout params Fixes missing key error in settings.py for `SESSION_SECURITY...` params when param missing. Testing was done only in dev env where DEBUG was always set. * Update CHANGELOG, VERSION Co-authored-by: Greg Elin <greg.elin@govready.com> * Restore `common-tab-count` styles (#1588) * Restore `common-tab-count` styles Restore a `common-tab-count` and `component-tab-count` styles accidentally not added to `govready-q.css` style sheet. * Update requirements.txt Co-authored-by: Greg Elin <greg.elin@govready.com> * Fix cmpt search result, smt count on action, session timeout err Fix various issues. Fix component search results returning multiple copies. Fix Session Timeout showing 500 error after being away from site. Fix control count on actions where too many control statements were trying to be added instead of statements of control implementation prototypes. * Reload component detail page after adding control statement Have page reload after adding control statement to a component in the library to avoid non-feedback to user and user having to refresh the page. Also fix ImportExportProjectTests test * Upgrade to Django 3.2.3 Upgrade to Django 3.2.3 to correct for Snyk indicated vulnerability in Django 3.1.8 https://snyk.io/vuln/SNYK-PYTHON-DJANGO-1279042 Remove documentation-related m2r and sphinx related packages from requirements.in. * Use 'controls' instead of 'control' in controls/apps.py Co-authored-by: davidpofo <dampofo@umd.edu> Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> Co-authored-by: GovReady-Q <govready-q@govready.com> Co-authored-by: Greg Elin <Greg Elin>

* Filter SAR Deployment form dropdown to System's deployments * Filter SAR Deployment form dropdown to System's deployments * Create a simple SAR server to generate test data Create a simple System Assessment Result web server generating JSON data object listing assessment summary results that can be added to a system's assessment. This provides a synthentic provider of SAR results data from which we can test the UI for consuming SAR data. * Better SAR displays, wrap SAR item data Polish SAR summary page - use details tag, other improvements. Polish SAR list page - include deployment name. Wrap SAR test item to start to make more realistic SAR data. Display assessment name in Assessment model admin list. * Add sar_etl.py example middlware app * Use project-base.html in deployment detail page. Misc improvements to sar generator and middleware. Include description value for SAR wrapper. * Remove hardcoded data from SAR middleware pipeline Remove hardcoded values of system id and deployment id from SAR middleware pipeline. Pass as many values as possible in SAR wrapper object. Properly handle UUID to string and back again Post assessment to system without a deployment ID Display project id, system id on project pages. * Support Wazuh SCA results in SAR pipeline Add `tools/simple_sar_server/wazuh_etl.py` to support Wazuh SCA results in SAR pipeline. Display all summary values of a SAR result for each inventory item using a loop and table. * Improve styling of inventory-item results display * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 3 * attempting multiple catalog output * dont want install py changes * Add clear links for forgot password, change password * Update stub_app for input, components Update stub_app used by complianca_app command or generating compliance app to include "input" and "output" section; and to have folders for templates, utils, and components. Add in exceprtion for integrity error to eventually support gracefully handling. * Closes #1547 Display impact level on project page. Implement impact level as a statement about a system of type "fisma_impact_level" with imoact level in body. This continues idea that statements are factual observations about a system. Statement must be associated with system.root_element and be consumed by root_element. * note as with any change to the document data to be rendered need to press the refresh documents button to see changes. Removing cache removal function. * spelling * Need default of None if there are no control catalogs present * also need try/except for test_render_markdown_to_text since there are no projects for this test. * ISPGBSS-208 Added new fields to system settings and created migrations * make sure to send catalog_key for sid_class form_value * checking for statement sid class * Add tests for fisma_impact_level_display * need to filter statements by sid class as well. some optimization on catalog retrieval * Add StatementTypeEnum Enumarate different type of statements * modularizing some common data retrievals. * Added empty my_component.json file * only save once at the end of all possible changes * added better logging to find bad templates faster * Report OSCAL component schema validation error to standout Provide better error reporting on import component schema validation. Report actual validation error to standout. * Merge v0.9.3.4 into develop * ISPGBSS-208 ISPGBSS-261 Refactored previous changes to implement timeout through environment variables instead of db based setting. * implemented a full example of multiple catalogs in a new govready-qfiles-startpack app source multi-catalog. * remove extra prints * put "Manage Import Records" button last Rearrange Create | Import | Manage component buttons * Immediately assign change project perms to user starting project This fixes a bug where non-admin users were not assigned permissions to change a project started by the user until after several project modification steps were performed. The fix assigns non-admin project change permissions right after project and system is created and before further actions are made to project, such as setting baseline controls. * Update CHANGELOG * Set system fisma_impact_level smt as question action with new set/get methods Set system fisma_impact_level root_element statement as part of question action to set baseline. Since baselines could have more names than just "Low", "Moderate", and "High", we only set fisma_impact_level if baseline is one of those three. This might need to improve in the future and make setting fisma_impact_level a separate question action from setting baselines. Created set/get methods for fisma_impact_level in System model to make sure only one statement for the system with statement_type of fisma_impact_level existed. Also fixed POAM stat counting to only count POAMs associated with current system. Existing POAM count was not filtering on current system. * Update CHANGELOG * Update CHANGELOG.md spelling * Update navbar.html * Update project.html all caps no brakes * fixing fisma display test (#1566) * fixes a few typos in HTML templates (#1565) * fix filtering for components on adding a component search box. (#1567) * Docker Local Development implementation (#1555) * wip * Local development rework based on deployments repo * adding +x * added action for running and some clean up and docs * Added docker engine check * added another message at end * more faq * changelog * added selenium & chrome support * document quick fix * regression * readme change * added var * Update dev_env/run.py Co-authored-by: davidpofo <dampofo@umd.edu> * space Co-authored-by: davidpofo <dampofo@umd.edu> * ISPGBSS-208 ISPGBSS-261 Added test for session setting ping * Fixed a bug where ssh files weren't brought over on restart * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Faster retrieval of component control statements, show control titles (#1570) * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Update element_detail_tabs.html system_tags aren't used anywhere Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * enums need to match case of value. adding smt id to logs * use enums' value to cover the expected values everye where for statement type. * remove asserts comment * ISPGBSS-208 ISPGBSS-261 Used requirements_txt_updated.sh to update requirements.txt as per PR review. * Display control catalog guidance text in `details` tag Display control catalog guidance text in `details` tag next to component control implementation statements. Also clean up some html on component detail pages, move some styles to main `govready-q.css` file. * Left align text in project action buttons * Improve project pages appearance: descrease action button width and left align text; widen from 9 to 10 columns main content. * Update CHANGELOG * Search component library by tag, make component tags clickable * Better notifification when project smt differs from certified Better notify users when project implementation statement differs from certified by displaying notice in third column control detail pages. Improve language notifying users that project implementation statement differs from certified. Only difference notice is clickable now. * Add DB page for AppInput * Add input_type to AppInputs * ISPGBSS-261 Fixed issues related to PR review. * Da/last min develop (#1582) * Add new options to runner.py environment. PR 1572 missed changelog. also change log for PR 1535 was missed for the last release * also mention nplusone. * commenting out session security for now until issue with js is fixed. * unnecessary return * fixing statement types * a few more statement type fixes * removing feature that is a WIP * adding some time to avoid the erroneous failures. * more sleep * skipping bad integration tests. (#1585) * missing statetypeenum import * added SECURITY_IMPACT_LEVEL to statements. Created a components enum * Ge/develop test 0516 (#1595) * Release/v0.9.3.5 (#1586) * Filter SAR Deployment form dropdown to System's deployments * Filter SAR Deployment form dropdown to System's deployments * Create a simple SAR server to generate test data Create a simple System Assessment Result web server generating JSON data object listing assessment summary results that can be added to a system's assessment. This provides a synthentic provider of SAR results data from which we can test the UI for consuming SAR data. * Better SAR displays, wrap SAR item data Polish SAR summary page - use details tag, other improvements. Polish SAR list page - include deployment name. Wrap SAR test item to start to make more realistic SAR data. Display assessment name in Assessment model admin list. * Add sar_etl.py example middlware app * Use project-base.html in deployment detail page. Misc improvements to sar generator and middleware. Include description value for SAR wrapper. * Remove hardcoded data from SAR middleware pipeline Remove hardcoded values of system id and deployment id from SAR middleware pipeline. Pass as many values as possible in SAR wrapper object. Properly handle UUID to string and back again Post assessment to system without a deployment ID Display project id, system id on project pages. * Support Wazuh SCA results in SAR pipeline Add `tools/simple_sar_server/wazuh_etl.py` to support Wazuh SCA results in SAR pipeline. Display all summary values of a SAR result for each inventory item using a loop and table. * Improve styling of inventory-item results display * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 3 * attempting multiple catalog output * dont want install py changes * Add clear links for forgot password, change password * Update stub_app for input, components Update stub_app used by complianca_app command or generating compliance app to include "input" and "output" section; and to have folders for templates, utils, and components. Add in exceprtion for integrity error to eventually support gracefully handling. * Closes #1547 Display impact level on project page. Implement impact level as a statement about a system of type "fisma_impact_level" with imoact level in body. This continues idea that statements are factual observations about a system. Statement must be associated with system.root_element and be consumed by root_element. * note as with any change to the document data to be rendered need to press the refresh documents button to see changes. Removing cache removal function. * spelling * Need default of None if there are no control catalogs present * also need try/except for test_render_markdown_to_text since there are no projects for this test. * ISPGBSS-208 Added new fields to system settings and created migrations * make sure to send catalog_key for sid_class form_value * checking for statement sid class * Add tests for fisma_impact_level_display * need to filter statements by sid class as well. some optimization on catalog retrieval * Add StatementTypeEnum Enumarate different type of statements * modularizing some common data retrievals. * Added empty my_component.json file * only save once at the end of all possible changes * added better logging to find bad templates faster * Report OSCAL component schema validation error to standout Provide better error reporting on import component schema validation. Report actual validation error to standout. * Merge v0.9.3.4 into develop * ISPGBSS-208 ISPGBSS-261 Refactored previous changes to implement timeout through environment variables instead of db based setting. * implemented a full example of multiple catalogs in a new govready-qfiles-startpack app source multi-catalog. * remove extra prints * put "Manage Import Records" button last Rearrange Create | Import | Manage component buttons * Immediately assign change project perms to user starting project This fixes a bug where non-admin users were not assigned permissions to change a project started by the user until after several project modification steps were performed. The fix assigns non-admin project change permissions right after project and system is created and before further actions are made to project, such as setting baseline controls. * Update CHANGELOG * Set system fisma_impact_level smt as question action with new set/get methods Set system fisma_impact_level root_element statement as part of question action to set baseline. Since baselines could have more names than just "Low", "Moderate", and "High", we only set fisma_impact_level if baseline is one of those three. This might need to improve in the future and make setting fisma_impact_level a separate question action from setting baselines. Created set/get methods for fisma_impact_level in System model to make sure only one statement for the system with statement_type of fisma_impact_level existed. Also fixed POAM stat counting to only count POAMs associated with current system. Existing POAM count was not filtering on current system. * Update CHANGELOG * Update CHANGELOG.md spelling * Update navbar.html * Update project.html all caps no brakes * fixing fisma display test (#1566) * fixes a few typos in HTML templates (#1565) * fix filtering for components on adding a component search box. (#1567) * Docker Local Development implementation (#1555) * wip * Local development rework based on deployments repo * adding +x * added action for running and some clean up and docs * Added docker engine check * added another message at end * more faq * changelog * added selenium & chrome support * document quick fix * regression * readme change * added var * Update dev_env/run.py Co-authored-by: davidpofo <dampofo@umd.edu> * space Co-authored-by: davidpofo <dampofo@umd.edu> * ISPGBSS-208 ISPGBSS-261 Added test for session setting ping * Fixed a bug where ssh files weren't brought over on restart * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Faster retrieval of component control statements, show control titles (#1570) * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Update element_detail_tabs.html system_tags aren't used anywhere Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * enums need to match case of value. adding smt id to logs * use enums' value to cover the expected values everye where for statement type. * remove asserts comment * ISPGBSS-208 ISPGBSS-261 Used requirements_txt_updated.sh to update requirements.txt as per PR review. * Display control catalog guidance text in `details` tag Display control catalog guidance text in `details` tag next to component control implementation statements. Also clean up some html on component detail pages, move some styles to main `govready-q.css` file. * Search component library by tag, make component tags clickable * Better notifification when project smt differs from certified Better notify users when project implementation statement differs from certified by displaying notice in third column control detail pages. Improve language notifying users that project implementation statement differs from certified. Only difference notice is clickable now. * ISPGBSS-261 Fixed issues related to PR review. * Da/last min develop (#1582) * Add new options to runner.py environment. PR 1572 missed changelog. also change log for PR 1535 was missed for the last release * also mention nplusone. * commenting out session security for now until issue with js is fixed. * unnecessary return * fixing statement types * a few more statement type fixes * removing feature that is a WIP * adding some time to avoid the erroneous failures. * more sleep * skipping bad integration tests. (#1585) * update version and changelog Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Greg Elin <gregelin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> * Fix for handling unset SESSION timeout params (#1587) * Fix to handle unset SESSION timeout params Fixes missing key error in settings.py for `SESSION_SECURITY...` params when param missing. Testing was done only in dev env where DEBUG was always set. * Update CHANGELOG, VERSION Co-authored-by: Greg Elin <greg.elin@govready.com> * Restore `common-tab-count` styles (#1588) * Restore `common-tab-count` styles Restore a `common-tab-count` and `component-tab-count` styles accidentally not added to `govready-q.css` style sheet. * Update requirements.txt Co-authored-by: Greg Elin <greg.elin@govready.com> * Fix cmpt search result, smt count on action, session timeout err Fix various issues. Fix component search results returning multiple copies. Fix Session Timeout showing 500 error after being away from site. Fix control count on actions where too many control statements were trying to be added instead of statements of control implementation prototypes. * Reload component detail page after adding control statement Have page reload after adding control statement to a component in the library to avoid non-feedback to user and user having to refresh the page. Also fix ImportExportProjectTests test * Upgrade to Django 3.2.3 Upgrade to Django 3.2.3 to correct for Snyk indicated vulnerability in Django 3.1.8 https://snyk.io/vuln/SNYK-PYTHON-DJANGO-1279042 Remove documentation-related m2r and sphinx related packages from requirements.in. * Use 'controls' instead of 'control' in controls/apps.py Co-authored-by: davidpofo <dampofo@umd.edu> Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> Co-authored-by: GovReady-Q <govready-q@govready.com> Co-authored-by: Greg Elin <Greg Elin> * Add management command to export as OSCAL or CSV. * In Django 3.2.0+ on need to set type of auto-created primary keys in settings. (could also be done explicitly in each model) * optimize imports * set/get methods for an element's security objectives * first instead of brackets * adding frontend for a component's security objectives section * this is implemented for elements. should be SSP instead... * implemented security objectives for systems. reverting element version * test * get project directly * need an element and system * ele and system * added comp state/type to element model, some system sec info to comp gen. * test * Add Django mgt cmd importcomponents to batch import components Updates to generation of components to improve adherence to OSCAL specification by removing certain keys when value for keys is None. Added new parameter `existing_import_record` to importing and creating components to group multiple imports under the same import record. * Make Elements.description TextField and required Change Elements.description to TextField and make required. Modify component edit modal to use a textarea for description field. Fixes to show error on problem saving edit to library component * Use self.element.tags.exists() * Send proper error message when editing component * Temporarily remove controls/migrations/0051_auto * Add back better controls/migrations/0051_auto * fixed control and sectioning for components in system * check for security impact level statement when updating. Readding retrieval of security impact levels. todo on separate form for levels. * adding component_state and component_type to system component and component library display. including component_metadata template to keep styling consistent * pulling in some of the information from statement about system. Rest would come from questionnaires. system_information_types is not used at all. * a todo for fisma impact level renaming * added project_security_objs_edit to edit security objectives separately from project editing. * update changelog and some wording in the modal * Move action-button styles from inline to style section * two views/urls for editing component state and type * adding component type and state to ElementForm * adding just the display of the state and type to component library components not the ability to change. * Da/quick insert (#1601) * make sure component_type not element_type is exported * ssp versions should be floats not integers. Information types needs a uuid * adding empty placeholders for the required keys. * using updated for component version * party-uuids is still a todo * categorizations is still a todo * parties is still a todo * Fix system ctl detail page err; Improve creating smt from prototypes (#1602) Refactor creating system control statements from component library prototype statements when adding a component from the library to a system and reduce by an order a magnitude the time it takes to add a component to system. Rename smt.create_instance_from_prototype to smt.create_system_control_smt_from_component_prototype_smt Fix bug breaking rendering of system's control detail page by removing an errant login_required decorator on a function. Add test for system control page. Will add test(s) for system control detail page. Co-authored-by: Greg Elin <greg.elin@govready.com> * Automatically clear, refresh output document content downloading docs Performnce of document generation now sufficiently fast to not require cache and manual "Refresh documents" button. * remove comments. changelog * Fixed an issue where statement didn't exist while exporting to oscal (#1605) * Fixed an issue where statement didn't exist while exporting to oscal * Update CHANGELOG Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Greg Elin <greg.elin@govready.com> * Align Delete section on project settings (#1604) Co-authored-by: Greg Elin <greg.elin@govready.com> * Update CHANGELOG * Ge/file upload extensions (#1607) * Accepts file uploads with capitalized extensions, e.g. ".JPG". Adjust file upload validator to recognize capitalized extensions and also recognizes ".jpeg" in addition to ".jpg". * Add tests for validating uppercase extensions on file uploads * Add test fixture data Co-authored-by: Greg Elin <greg.elin@govready.com> * Batch update cntl impl smts when component_statement changes Implemented a faster way to update status of system controls. When user sets a system component state to "operational" all statements associated with that component for the system get their status set to "Implemented". Similarly, setting component’s state to "planned" batch sets all component statements for that system to "Planned", and "under-development" sets component statements to "Partially Implemented". Display system component component_state and component_type when component is listed for a system. * export a projects ssp control implementations with export form (#1611) * export a projects ssp control implementations with export form * remove comments * Correct slugify import * Security update Python 3.2.4 due to https://snyk.io/vuln/SNYK-PYTHON-DJANGO-1298665 * Polish SSP control CSV export form Co-authored-by: Greg Elin <greg.elin@govready.com> * Add 'Create a template' button to template library (#1610) Co-authored-by: Greg Elin <greg.elin@govready.com> * Content-Security-Policy header permit images (*), videos youtube, vimeo * Force controls csv to download to browser * 'Back' link to question to take user to previous question (#1612) * 'Back' link to question to take user to previous question * Update guidedmodules/views.py Refactor pulling back_url into project_form Co-authored-by: davidpofo <dampofo@umd.edu> * Improve back-button styling Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: davidpofo <dampofo@umd.edu> Co-authored-by: Greg Elin <greg.elin@govready.com> * WIP: Side-by-side comparison of components (#1620) * created checkbox and form for submitting components for comparison. created rough start for displaying differences between prime component and rest * for now just implementing two comparison * click to read full text after 50 chars * styling and added Control part * displaying comparisons for x number of component statements against the prime component. Styling and abstracted out the comparison block into an included template * check for pid * removing detail/summary not really necessary * Condense comparison listings into rows of a single table Co-authored-by: Greg Elin <Greg Elin> * Update CHANGELOG.md * Rename 'compare' column to 'select' in component library (#1626) Co-authored-by: Greg Elin <Greg Elin> * Bump VERSION, CHANGELOG Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Greg Elin <gregelin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> Co-authored-by: GovReady-Q <govready-q@govready.com>

* Filter SAR Deployment form dropdown to System's deployments * Filter SAR Deployment form dropdown to System's deployments * Create a simple SAR server to generate test data Create a simple System Assessment Result web server generating JSON data object listing assessment summary results that can be added to a system's assessment. This provides a synthentic provider of SAR results data from which we can test the UI for consuming SAR data. * Better SAR displays, wrap SAR item data Polish SAR summary page - use details tag, other improvements. Polish SAR list page - include deployment name. Wrap SAR test item to start to make more realistic SAR data. Display assessment name in Assessment model admin list. * Add sar_etl.py example middlware app * Use project-base.html in deployment detail page. Misc improvements to sar generator and middleware. Include description value for SAR wrapper. * Remove hardcoded data from SAR middleware pipeline Remove hardcoded values of system id and deployment id from SAR middleware pipeline. Pass as many values as possible in SAR wrapper object. Properly handle UUID to string and back again Post assessment to system without a deployment ID Display project id, system id on project pages. * Support Wazuh SCA results in SAR pipeline Add `tools/simple_sar_server/wazuh_etl.py` to support Wazuh SCA results in SAR pipeline. Display all summary values of a SAR result for each inventory item using a loop and table. * Improve styling of inventory-item results display * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 3 * attempting multiple catalog output * dont want install py changes * Add clear links for forgot password, change password * Update stub_app for input, components Update stub_app used by complianca_app command or generating compliance app to include "input" and "output" section; and to have folders for templates, utils, and components. Add in exceprtion for integrity error to eventually support gracefully handling. * Closes #1547 Display impact level on project page. Implement impact level as a statement about a system of type "fisma_impact_level" with imoact level in body. This continues idea that statements are factual observations about a system. Statement must be associated with system.root_element and be consumed by root_element. * note as with any change to the document data to be rendered need to press the refresh documents button to see changes. Removing cache removal function. * spelling * Need default of None if there are no control catalogs present * also need try/except for test_render_markdown_to_text since there are no projects for this test. * ISPGBSS-208 Added new fields to system settings and created migrations * make sure to send catalog_key for sid_class form_value * checking for statement sid class * Add tests for fisma_impact_level_display * need to filter statements by sid class as well. some optimization on catalog retrieval * Add StatementTypeEnum Enumarate different type of statements * modularizing some common data retrievals. * Added empty my_component.json file * only save once at the end of all possible changes * added better logging to find bad templates faster * Report OSCAL component schema validation error to standout Provide better error reporting on import component schema validation. Report actual validation error to standout. * Merge v0.9.3.4 into develop * ISPGBSS-208 ISPGBSS-261 Refactored previous changes to implement timeout through environment variables instead of db based setting. * implemented a full example of multiple catalogs in a new govready-qfiles-startpack app source multi-catalog. * remove extra prints * put "Manage Import Records" button last Rearrange Create | Import | Manage component buttons * Immediately assign change project perms to user starting project This fixes a bug where non-admin users were not assigned permissions to change a project started by the user until after several project modification steps were performed. The fix assigns non-admin project change permissions right after project and system is created and before further actions are made to project, such as setting baseline controls. * Update CHANGELOG * Set system fisma_impact_level smt as question action with new set/get methods Set system fisma_impact_level root_element statement as part of question action to set baseline. Since baselines could have more names than just "Low", "Moderate", and "High", we only set fisma_impact_level if baseline is one of those three. This might need to improve in the future and make setting fisma_impact_level a separate question action from setting baselines. Created set/get methods for fisma_impact_level in System model to make sure only one statement for the system with statement_type of fisma_impact_level existed. Also fixed POAM stat counting to only count POAMs associated with current system. Existing POAM count was not filtering on current system. * Update CHANGELOG * Update CHANGELOG.md spelling * Update navbar.html * Update project.html all caps no brakes * fixing fisma display test (#1566) * fixes a few typos in HTML templates (#1565) * fix filtering for components on adding a component search box. (#1567) * Docker Local Development implementation (#1555) * wip * Local development rework based on deployments repo * adding +x * added action for running and some clean up and docs * Added docker engine check * added another message at end * more faq * changelog * added selenium & chrome support * document quick fix * regression * readme change * added var * Update dev_env/run.py Co-authored-by: davidpofo <dampofo@umd.edu> * space Co-authored-by: davidpofo <dampofo@umd.edu> * ISPGBSS-208 ISPGBSS-261 Added test for session setting ping * Fixed a bug where ssh files weren't brought over on restart * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Faster retrieval of component control statements, show control titles (#1570) * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Update element_detail_tabs.html system_tags aren't used anywhere Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * enums need to match case of value. adding smt id to logs * use enums' value to cover the expected values everye where for statement type. * remove asserts comment * ISPGBSS-208 ISPGBSS-261 Used requirements_txt_updated.sh to update requirements.txt as per PR review. * Display control catalog guidance text in `details` tag Display control catalog guidance text in `details` tag next to component control implementation statements. Also clean up some html on component detail pages, move some styles to main `govready-q.css` file. * Left align text in project action buttons * Improve project pages appearance: descrease action button width and left align text; widen from 9 to 10 columns main content. * Update CHANGELOG * Search component library by tag, make component tags clickable * Better notifification when project smt differs from certified Better notify users when project implementation statement differs from certified by displaying notice in third column control detail pages. Improve language notifying users that project implementation statement differs from certified. Only difference notice is clickable now. * Add DB page for AppInput * Add input_type to AppInputs * ISPGBSS-261 Fixed issues related to PR review. * Da/last min develop (#1582) * Add new options to runner.py environment. PR 1572 missed changelog. also change log for PR 1535 was missed for the last release * also mention nplusone. * commenting out session security for now until issue with js is fixed. * unnecessary return * fixing statement types * a few more statement type fixes * removing feature that is a WIP * adding some time to avoid the erroneous failures. * more sleep * skipping bad integration tests. (#1585) * missing statetypeenum import * added SECURITY_IMPACT_LEVEL to statements. Created a components enum * Ge/develop test 0516 (#1595) * Release/v0.9.3.5 (#1586) * Filter SAR Deployment form dropdown to System's deployments * Filter SAR Deployment form dropdown to System's deployments * Create a simple SAR server to generate test data Create a simple System Assessment Result web server generating JSON data object listing assessment summary results that can be added to a system's assessment. This provides a synthentic provider of SAR results data from which we can test the UI for consuming SAR data. * Better SAR displays, wrap SAR item data Polish SAR summary page - use details tag, other improvements. Polish SAR list page - include deployment name. Wrap SAR test item to start to make more realistic SAR data. Display assessment name in Assessment model admin list. * Add sar_etl.py example middlware app * Use project-base.html in deployment detail page. Misc improvements to sar generator and middleware. Include description value for SAR wrapper. * Remove hardcoded data from SAR middleware pipeline Remove hardcoded values of system id and deployment id from SAR middleware pipeline. Pass as many values as possible in SAR wrapper object. Properly handle UUID to string and back again Post assessment to system without a deployment ID Display project id, system id on project pages. * Support Wazuh SCA results in SAR pipeline Add `tools/simple_sar_server/wazuh_etl.py` to support Wazuh SCA results in SAR pipeline. Display all summary values of a SAR result for each inventory item using a loop and table. * Improve styling of inventory-item results display * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 3 * attempting multiple catalog output * dont want install py changes * Add clear links for forgot password, change password * Update stub_app for input, components Update stub_app used by complianca_app command or generating compliance app to include "input" and "output" section; and to have folders for templates, utils, and components. Add in exceprtion for integrity error to eventually support gracefully handling. * Closes #1547 Display impact level on project page. Implement impact level as a statement about a system of type "fisma_impact_level" with imoact level in body. This continues idea that statements are factual observations about a system. Statement must be associated with system.root_element and be consumed by root_element. * note as with any change to the document data to be rendered need to press the refresh documents button to see changes. Removing cache removal function. * spelling * Need default of None if there are no control catalogs present * also need try/except for test_render_markdown_to_text since there are no projects for this test. * ISPGBSS-208 Added new fields to system settings and created migrations * make sure to send catalog_key for sid_class form_value * checking for statement sid class * Add tests for fisma_impact_level_display * need to filter statements by sid class as well. some optimization on catalog retrieval * Add StatementTypeEnum Enumarate different type of statements * modularizing some common data retrievals. * Added empty my_component.json file * only save once at the end of all possible changes * added better logging to find bad templates faster * Report OSCAL component schema validation error to standout Provide better error reporting on import component schema validation. Report actual validation error to standout. * Merge v0.9.3.4 into develop * ISPGBSS-208 ISPGBSS-261 Refactored previous changes to implement timeout through environment variables instead of db based setting. * implemented a full example of multiple catalogs in a new govready-qfiles-startpack app source multi-catalog. * remove extra prints * put "Manage Import Records" button last Rearrange Create | Import | Manage component buttons * Immediately assign change project perms to user starting project This fixes a bug where non-admin users were not assigned permissions to change a project started by the user until after several project modification steps were performed. The fix assigns non-admin project change permissions right after project and system is created and before further actions are made to project, such as setting baseline controls. * Update CHANGELOG * Set system fisma_impact_level smt as question action with new set/get methods Set system fisma_impact_level root_element statement as part of question action to set baseline. Since baselines could have more names than just "Low", "Moderate", and "High", we only set fisma_impact_level if baseline is one of those three. This might need to improve in the future and make setting fisma_impact_level a separate question action from setting baselines. Created set/get methods for fisma_impact_level in System model to make sure only one statement for the system with statement_type of fisma_impact_level existed. Also fixed POAM stat counting to only count POAMs associated with current system. Existing POAM count was not filtering on current system. * Update CHANGELOG * Update CHANGELOG.md spelling * Update navbar.html * Update project.html all caps no brakes * fixing fisma display test (#1566) * fixes a few typos in HTML templates (#1565) * fix filtering for components on adding a component search box. (#1567) * Docker Local Development implementation (#1555) * wip * Local development rework based on deployments repo * adding +x * added action for running and some clean up and docs * Added docker engine check * added another message at end * more faq * changelog * added selenium & chrome support * document quick fix * regression * readme change * added var * Update dev_env/run.py Co-authored-by: davidpofo <dampofo@umd.edu> * space Co-authored-by: davidpofo <dampofo@umd.edu> * ISPGBSS-208 ISPGBSS-261 Added test for session setting ping * Fixed a bug where ssh files weren't brought over on restart * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Faster retrieval of component control statements, show control titles (#1570) * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Update element_detail_tabs.html system_tags aren't used anywhere Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * enums need to match case of value. adding smt id to logs * use enums' value to cover the expected values everye where for statement type. * remove asserts comment * ISPGBSS-208 ISPGBSS-261 Used requirements_txt_updated.sh to update requirements.txt as per PR review. * Display control catalog guidance text in `details` tag Display control catalog guidance text in `details` tag next to component control implementation statements. Also clean up some html on component detail pages, move some styles to main `govready-q.css` file. * Search component library by tag, make component tags clickable * Better notifification when project smt differs from certified Better notify users when project implementation statement differs from certified by displaying notice in third column control detail pages. Improve language notifying users that project implementation statement differs from certified. Only difference notice is clickable now. * ISPGBSS-261 Fixed issues related to PR review. * Da/last min develop (#1582) * Add new options to runner.py environment. PR 1572 missed changelog. also change log for PR 1535 was missed for the last release * also mention nplusone. * commenting out session security for now until issue with js is fixed. * unnecessary return * fixing statement types * a few more statement type fixes * removing feature that is a WIP * adding some time to avoid the erroneous failures. * more sleep * skipping bad integration tests. (#1585) * update version and changelog Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Greg Elin <gregelin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> * Fix for handling unset SESSION timeout params (#1587) * Fix to handle unset SESSION timeout params Fixes missing key error in settings.py for `SESSION_SECURITY...` params when param missing. Testing was done only in dev env where DEBUG was always set. * Update CHANGELOG, VERSION Co-authored-by: Greg Elin <greg.elin@govready.com> * Restore `common-tab-count` styles (#1588) * Restore `common-tab-count` styles Restore a `common-tab-count` and `component-tab-count` styles accidentally not added to `govready-q.css` style sheet. * Update requirements.txt Co-authored-by: Greg Elin <greg.elin@govready.com> * Fix cmpt search result, smt count on action, session timeout err Fix various issues. Fix component search results returning multiple copies. Fix Session Timeout showing 500 error after being away from site. Fix control count on actions where too many control statements were trying to be added instead of statements of control implementation prototypes. * Reload component detail page after adding control statement Have page reload after adding control statement to a component in the library to avoid non-feedback to user and user having to refresh the page. Also fix ImportExportProjectTests test * Upgrade to Django 3.2.3 Upgrade to Django 3.2.3 to correct for Snyk indicated vulnerability in Django 3.1.8 https://snyk.io/vuln/SNYK-PYTHON-DJANGO-1279042 Remove documentation-related m2r and sphinx related packages from requirements.in. * Use 'controls' instead of 'control' in controls/apps.py Co-authored-by: davidpofo <dampofo@umd.edu> Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> Co-authored-by: GovReady-Q <govready-q@govready.com> Co-authored-by: Greg Elin <Greg Elin> * Add management command to export as OSCAL or CSV. * In Django 3.2.0+ on need to set type of auto-created primary keys in settings. (could also be done explicitly in each model) * optimize imports * set/get methods for an element's security objectives * first instead of brackets * adding frontend for a component's security objectives section * this is implemented for elements. should be SSP instead... * implemented security objectives for systems. reverting element version * test * get project directly * need an element and system * ele and system * added comp state/type to element model, some system sec info to comp gen. * test * Add Django mgt cmd importcomponents to batch import components Updates to generation of components to improve adherence to OSCAL specification by removing certain keys when value for keys is None. Added new parameter `existing_import_record` to importing and creating components to group multiple imports under the same import record. * Make Elements.description TextField and required Change Elements.description to TextField and make required. Modify component edit modal to use a textarea for description field. Fixes to show error on problem saving edit to library component * Use self.element.tags.exists() * Send proper error message when editing component * Temporarily remove controls/migrations/0051_auto * Add back better controls/migrations/0051_auto * fixed control and sectioning for components in system * check for security impact level statement when updating. Readding retrieval of security impact levels. todo on separate form for levels. * adding component_state and component_type to system component and component library display. including component_metadata template to keep styling consistent * pulling in some of the information from statement about system. Rest would come from questionnaires. system_information_types is not used at all. * a todo for fisma impact level renaming * added project_security_objs_edit to edit security objectives separately from project editing. * update changelog and some wording in the modal * Move action-button styles from inline to style section * two views/urls for editing component state and type * adding component type and state to ElementForm * adding just the display of the state and type to component library components not the ability to change. * Work inprogress * Work inprogress * Da/quick insert (#1601) * make sure component_type not element_type is exported * ssp versions should be floats not integers. Information types needs a uuid * adding empty placeholders for the required keys. * using updated for component version * party-uuids is still a todo * categorizations is still a todo * parties is still a todo * Fix system ctl detail page err; Improve creating smt from prototypes (#1602) Refactor creating system control statements from component library prototype statements when adding a component from the library to a system and reduce by an order a magnitude the time it takes to add a component to system. Rename smt.create_instance_from_prototype to smt.create_system_control_smt_from_component_prototype_smt Fix bug breaking rendering of system's control detail page by removing an errant login_required decorator on a function. Add test for system control page. Will add test(s) for system control detail page. Co-authored-by: Greg Elin <greg.elin@govready.com> * Automatically clear, refresh output document content downloading docs Performnce of document generation now sufficiently fast to not require cache and manual "Refresh documents" button. * remove comments. changelog * Fixed an issue where statement didn't exist while exporting to oscal (#1605) * Fixed an issue where statement didn't exist while exporting to oscal * Update CHANGELOG Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Greg Elin <greg.elin@govready.com> * Align Delete section on project settings (#1604) Co-authored-by: Greg Elin <greg.elin@govready.com> * Update CHANGELOG * Ge/file upload extensions (#1607) * Accepts file uploads with capitalized extensions, e.g. ".JPG". Adjust file upload validator to recognize capitalized extensions and also recognizes ".jpeg" in addition to ".jpg". * Add tests for validating uppercase extensions on file uploads * Add test fixture data Co-authored-by: Greg Elin <greg.elin@govready.com> * Batch update cntl impl smts when component_statement changes Implemented a faster way to update status of system controls. When user sets a system component state to "operational" all statements associated with that component for the system get their status set to "Implemented". Similarly, setting component’s state to "planned" batch sets all component statements for that system to "Planned", and "under-development" sets component statements to "Partially Implemented". Display system component component_state and component_type when component is listed for a system. * More okta changes * export a projects ssp control implementations with export form (#1611) * export a projects ssp control implementations with export form * remove comments * Correct slugify import * Security update Python 3.2.4 due to https://snyk.io/vuln/SNYK-PYTHON-DJANGO-1298665 * Polish SSP control CSV export form Co-authored-by: Greg Elin <greg.elin@govready.com> * Add 'Create a template' button to template library (#1610) Co-authored-by: Greg Elin <greg.elin@govready.com> * Content-Security-Policy header permit images (*), videos youtube, vimeo * quick fix for auth * quick fix for auth * quick fix for auth * quick fix for auth * quick fix for auth * Force controls csv to download to browser * quick fix for auth * quick fix for auth * test * test * test * test * test * last fix and vuln update for django * last fix and vuln update for django * 'Back' link to question to take user to previous question (#1612) * 'Back' link to question to take user to previous question * Update guidedmodules/views.py Refactor pulling back_url into project_form Co-authored-by: davidpofo <dampofo@umd.edu> * Improve back-button styling Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: davidpofo <dampofo@umd.edu> Co-authored-by: Greg Elin <greg.elin@govready.com> * WIP: Side-by-side comparison of components (#1620) * created checkbox and form for submitting components for comparison. created rough start for displaying differences between prime component and rest * for now just implementing two comparison * click to read full text after 50 chars * styling and added Control part * displaying comparisons for x number of component statements against the prime component. Styling and abstracted out the comparison block into an included template * check for pid * removing detail/summary not really necessary * Condense comparison listings into rows of a single table Co-authored-by: Greg Elin <Greg Elin> * Update CHANGELOG.md * Rename 'compare' column to 'select' in component library (#1626) Co-authored-by: Greg Elin <Greg Elin> * Remove portfolio selection modal from Start a Project process Start projects in user's default portfolio to reduce the clicks starting a project. Use the User.create_default_portfolio_if_missing method consistently to consistently create the user portfolio default portfolio. Remove the PortfolioSignupForm because registration is no longer used in registering users. Remove the '"project_form": AddProjectForm(request.user ...' passed into many templates because the navbar start app option no longer brings up the portfolio select modal. * Update tests for default portfolio * Bump VERSION, CHANGELOG * Update CHANGELOG VERSION * Add button, form to add AppSource via upload of zip file Add button, form to App Store to provide front-end UI for admininstrators to add an AppSource by uploading a zip file. This simplifies setting up an AppSource for first time users. Implementation only validates that uploaded directory is a zip file, does not check if uploaded zip file is valid AppSource directory structure. Implementation assumes apps are in the 'apps' directory. * Link to library version of component from a system's selected control component * Display systems using a component (#1618) * Display systems using a component Add method controls.element.consuming_systems to produce list of systems consuming (e.g., containing) the element. Add tab to component library component detail page to display list of systems containing the component. Also, always display OSCAL tab in component library for component detail (rather than conditional on 'enable_experimental_opencontrol' parameter). * Show component system count in tab, better projects.exists query * Replace list compression with query filters Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * Ge/fulltext search (#1631) * Add fulltext smt search to component library search * Note fulltext search in CHANGELOG Co-authored-by: Greg Elin <Greg Elin> * check if we are in a portfolio when starting a project. If so then use that portfolio and not the default for the user. * fixed a bug where Elements of type system were shown in the selected components for a project * Addressing github issue 1630 in group id matching. fixed a bug where Elements of type system were shown in the selected components for a project. * try/except to still do the component search for non-Postgres users. (#1633) * Add a 'blank' project with no questions useful for batch project creation (#1634) Co-authored-by: Greg Elin <Greg Elin> * Add 'blank' compliance app to first_run * Fix version in CHANGELOG Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> Co-authored-by: GovReady-Q <govready-q@govready.com>

* fixing fisma display test (#1566) * fixes a few typos in HTML templates (#1565) * fix filtering for components on adding a component search box. (#1567) * Docker Local Development implementation (#1555) * wip * Local development rework based on deployments repo * adding +x * added action for running and some clean up and docs * Added docker engine check * added another message at end * more faq * changelog * added selenium & chrome support * document quick fix * regression * readme change * added var * Update dev_env/run.py Co-authored-by: davidpofo <dampofo@umd.edu> * space Co-authored-by: davidpofo <dampofo@umd.edu> * ISPGBSS-208 ISPGBSS-261 Added test for session setting ping * Fixed a bug where ssh files weren't brought over on restart * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Faster retrieval of component control statements, show control titles (#1570) * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Update element_detail_tabs.html system_tags aren't used anywhere Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * enums need to match case of value. adding smt id to logs * use enums' value to cover the expected values everye where for statement type. * remove asserts comment * ISPGBSS-208 ISPGBSS-261 Used requirements_txt_updated.sh to update requirements.txt as per PR review. * Display control catalog guidance text in `details` tag Display control catalog guidance text in `details` tag next to component control implementation statements. Also clean up some html on component detail pages, move some styles to main `govready-q.css` file. * Left align text in project action buttons * Improve project pages appearance: descrease action button width and left align text; widen from 9 to 10 columns main content. * Update CHANGELOG * Search component library by tag, make component tags clickable * Better notifification when project smt differs from certified Better notify users when project implementation statement differs from certified by displaying notice in third column control detail pages. Improve language notifying users that project implementation statement differs from certified. Only difference notice is clickable now. * Add DB page for AppInput * Add input_type to AppInputs * ISPGBSS-261 Fixed issues related to PR review. * Da/last min develop (#1582) * Add new options to runner.py environment. PR 1572 missed changelog. also change log for PR 1535 was missed for the last release * also mention nplusone. * commenting out session security for now until issue with js is fixed. * unnecessary return * fixing statement types * a few more statement type fixes * removing feature that is a WIP * adding some time to avoid the erroneous failures. * more sleep * skipping bad integration tests. (#1585) * missing statetypeenum import * added SECURITY_IMPACT_LEVEL to statements. Created a components enum * Clean up mixed tabs/spaces * Loosen git url regex to allow ssh:// and other urls * Missed some tabs in repatching make_widget() in render() * Ge/develop test 0516 (#1595) * Release/v0.9.3.5 (#1586) * Filter SAR Deployment form dropdown to System's deployments * Filter SAR Deployment form dropdown to System's deployments * Create a simple SAR server to generate test data Create a simple System Assessment Result web server generating JSON data object listing assessment summary results that can be added to a system's assessment. This provides a synthentic provider of SAR results data from which we can test the UI for consuming SAR data. * Better SAR displays, wrap SAR item data Polish SAR summary page - use details tag, other improvements. Polish SAR list page - include deployment name. Wrap SAR test item to start to make more realistic SAR data. Display assessment name in Assessment model admin list. * Add sar_etl.py example middlware app * Use project-base.html in deployment detail page. Misc improvements to sar generator and middleware. Include description value for SAR wrapper. * Remove hardcoded data from SAR middleware pipeline Remove hardcoded values of system id and deployment id from SAR middleware pipeline. Pass as many values as possible in SAR wrapper object. Properly handle UUID to string and back again Post assessment to system without a deployment ID Display project id, system id on project pages. * Support Wazuh SCA results in SAR pipeline Add `tools/simple_sar_server/wazuh_etl.py` to support Wazuh SCA results in SAR pipeline. Display all summary values of a SAR result for each inventory item using a loop and table. * Improve styling of inventory-item results display * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 2 * Improve styling of inventory-item results display 3 * attempting multiple catalog output * dont want install py changes * Add clear links for forgot password, change password * Update stub_app for input, components Update stub_app used by complianca_app command or generating compliance app to include "input" and "output" section; and to have folders for templates, utils, and components. Add in exceprtion for integrity error to eventually support gracefully handling. * Closes #1547 Display impact level on project page. Implement impact level as a statement about a system of type "fisma_impact_level" with imoact level in body. This continues idea that statements are factual observations about a system. Statement must be associated with system.root_element and be consumed by root_element. * note as with any change to the document data to be rendered need to press the refresh documents button to see changes. Removing cache removal function. * spelling * Need default of None if there are no control catalogs present * also need try/except for test_render_markdown_to_text since there are no projects for this test. * ISPGBSS-208 Added new fields to system settings and created migrations * make sure to send catalog_key for sid_class form_value * checking for statement sid class * Add tests for fisma_impact_level_display * need to filter statements by sid class as well. some optimization on catalog retrieval * Add StatementTypeEnum Enumarate different type of statements * modularizing some common data retrievals. * Added empty my_component.json file * only save once at the end of all possible changes * added better logging to find bad templates faster * Report OSCAL component schema validation error to standout Provide better error reporting on import component schema validation. Report actual validation error to standout. * Merge v0.9.3.4 into develop * ISPGBSS-208 ISPGBSS-261 Refactored previous changes to implement timeout through environment variables instead of db based setting. * implemented a full example of multiple catalogs in a new govready-qfiles-startpack app source multi-catalog. * remove extra prints * put "Manage Import Records" button last Rearrange Create | Import | Manage component buttons * Immediately assign change project perms to user starting project This fixes a bug where non-admin users were not assigned permissions to change a project started by the user until after several project modification steps were performed. The fix assigns non-admin project change permissions right after project and system is created and before further actions are made to project, such as setting baseline controls. * Update CHANGELOG * Set system fisma_impact_level smt as question action with new set/get methods Set system fisma_impact_level root_element statement as part of question action to set baseline. Since baselines could have more names than just "Low", "Moderate", and "High", we only set fisma_impact_level if baseline is one of those three. This might need to improve in the future and make setting fisma_impact_level a separate question action from setting baselines. Created set/get methods for fisma_impact_level in System model to make sure only one statement for the system with statement_type of fisma_impact_level existed. Also fixed POAM stat counting to only count POAMs associated with current system. Existing POAM count was not filtering on current system. * Update CHANGELOG * Update CHANGELOG.md spelling * Update navbar.html * Update project.html all caps no brakes * fixing fisma display test (#1566) * fixes a few typos in HTML templates (#1565) * fix filtering for components on adding a component search box. (#1567) * Docker Local Development implementation (#1555) * wip * Local development rework based on deployments repo * adding +x * added action for running and some clean up and docs * Added docker engine check * added another message at end * more faq * changelog * added selenium & chrome support * document quick fix * regression * readme change * added var * Update dev_env/run.py Co-authored-by: davidpofo <dampofo@umd.edu> * space Co-authored-by: davidpofo <dampofo@umd.edu> * ISPGBSS-208 ISPGBSS-261 Added test for session setting ping * Fixed a bug where ssh files weren't brought over on restart * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Faster retrieval of component control statements, show control titles (#1570) * Faster retrieval of component cntl smts, show cntl titles Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages. * Update element_detail_tabs.html system_tags aren't used anywhere Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * enums need to match case of value. adding smt id to logs * use enums' value to cover the expected values everye where for statement type. * remove asserts comment * ISPGBSS-208 ISPGBSS-261 Used requirements_txt_updated.sh to update requirements.txt as per PR review. * Display control catalog guidance text in `details` tag Display control catalog guidance text in `details` tag next to component control implementation statements. Also clean up some html on component detail pages, move some styles to main `govready-q.css` file. * Search component library by tag, make component tags clickable * Better notifification when project smt differs from certified Better notify users when project implementation statement differs from certified by displaying notice in third column control detail pages. Improve language notifying users that project implementation statement differs from certified. Only difference notice is clickable now. * ISPGBSS-261 Fixed issues related to PR review. * Da/last min develop (#1582) * Add new options to runner.py environment. PR 1572 missed changelog. also change log for PR 1535 was missed for the last release * also mention nplusone. * commenting out session security for now until issue with js is fixed. * unnecessary return * fixing statement types * a few more statement type fixes * removing feature that is a WIP * adding some time to avoid the erroneous failures. * more sleep * skipping bad integration tests. (#1585) * update version and changelog Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Greg Elin <gregelin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> * Fix for handling unset SESSION timeout params (#1587) * Fix to handle unset SESSION timeout params Fixes missing key error in settings.py for `SESSION_SECURITY...` params when param missing. Testing was done only in dev env where DEBUG was always set. * Update CHANGELOG, VERSION Co-authored-by: Greg Elin <greg.elin@govready.com> * Restore `common-tab-count` styles (#1588) * Restore `common-tab-count` styles Restore a `common-tab-count` and `component-tab-count` styles accidentally not added to `govready-q.css` style sheet. * Update requirements.txt Co-authored-by: Greg Elin <greg.elin@govready.com> * Fix cmpt search result, smt count on action, session timeout err Fix various issues. Fix component search results returning multiple copies. Fix Session Timeout showing 500 error after being away from site. Fix control count on actions where too many control statements were trying to be added instead of statements of control implementation prototypes. * Reload component detail page after adding control statement Have page reload after adding control statement to a component in the library to avoid non-feedback to user and user having to refresh the page. Also fix ImportExportProjectTests test * Upgrade to Django 3.2.3 Upgrade to Django 3.2.3 to correct for Snyk indicated vulnerability in Django 3.1.8 https://snyk.io/vuln/SNYK-PYTHON-DJANGO-1279042 Remove documentation-related m2r and sphinx related packages from requirements.in. * Use 'controls' instead of 'control' in controls/apps.py Co-authored-by: davidpofo <dampofo@umd.edu> Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> Co-authored-by: GovReady-Q <govready-q@govready.com> Co-authored-by: Greg Elin <Greg Elin> * Add management command to export as OSCAL or CSV. * In Django 3.2.0+ on need to set type of auto-created primary keys in settings. (could also be done explicitly in each model) * optimize imports * set/get methods for an element's security objectives * first instead of brackets * adding frontend for a component's security objectives section * this is implemented for elements. should be SSP instead... * implemented security objectives for systems. reverting element version * test * get project directly * need an element and system * ele and system * added comp state/type to element model, some system sec info to comp gen. * test * Add Django mgt cmd importcomponents to batch import components Updates to generation of components to improve adherence to OSCAL specification by removing certain keys when value for keys is None. Added new parameter `existing_import_record` to importing and creating components to group multiple imports under the same import record. * Make Elements.description TextField and required Change Elements.description to TextField and make required. Modify component edit modal to use a textarea for description field. Fixes to show error on problem saving edit to library component * Use self.element.tags.exists() * Send proper error message when editing component * Temporarily remove controls/migrations/0051_auto * Add back better controls/migrations/0051_auto * fixed control and sectioning for components in system * check for security impact level statement when updating. Readding retrieval of security impact levels. todo on separate form for levels. * adding component_state and component_type to system component and component library display. including component_metadata template to keep styling consistent * pulling in some of the information from statement about system. Rest would come from questionnaires. system_information_types is not used at all. * a todo for fisma impact level renaming * added project_security_objs_edit to edit security objectives separately from project editing. * update changelog and some wording in the modal * Move action-button styles from inline to style section * two views/urls for editing component state and type * adding component type and state to ElementForm * adding just the display of the state and type to component library components not the ability to change. * changelog * FISMA IMPACT LEVEL is now SECURITY SENSITIVITY LEVEL * Work inprogress * Work inprogress * Da/quick insert (#1601) * make sure component_type not element_type is exported * ssp versions should be floats not integers. Information types needs a uuid * adding empty placeholders for the required keys. * using updated for component version * party-uuids is still a todo * categorizations is still a todo * parties is still a todo * Fix system ctl detail page err; Improve creating smt from prototypes (#1602) Refactor creating system control statements from component library prototype statements when adding a component from the library to a system and reduce by an order a magnitude the time it takes to add a component to system. Rename smt.create_instance_from_prototype to smt.create_system_control_smt_from_component_prototype_smt Fix bug breaking rendering of system's control detail page by removing an errant login_required decorator on a function. Add test for system control page. Will add test(s) for system control detail page. Co-authored-by: Greg Elin <greg.elin@govready.com> * Automatically clear, refresh output document content downloading docs Performnce of document generation now sufficiently fast to not require cache and manual "Refresh documents" button. * remove comments. changelog * Fixed an issue where statement didn't exist while exporting to oscal (#1605) * Fixed an issue where statement didn't exist while exporting to oscal * Update CHANGELOG Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> Co-authored-by: Greg Elin <greg.elin@govready.com> * Align Delete section on project settings (#1604) Co-authored-by: Greg Elin <greg.elin@govready.com> * Update CHANGELOG * Ge/file upload extensions (#1607) * Accepts file uploads with capitalized extensions, e.g. ".JPG". Adjust file upload validator to recognize capitalized extensions and also recognizes ".jpeg" in addition to ".jpg". * Add tests for validating uppercase extensions on file uploads * Add test fixture data Co-authored-by: Greg Elin <greg.elin@govready.com> * Batch update cntl impl smts when component_statement changes Implemented a faster way to update status of system controls. When user sets a system component state to "operational" all statements associated with that component for the system get their status set to "Implemented". Similarly, setting component’s state to "planned" batch sets all component statements for that system to "Planned", and "under-development" sets component statements to "Partially Implemented". Display system component component_state and component_type when component is listed for a system. * More okta changes * export a projects ssp control implementations with export form (#1611) * export a projects ssp control implementations with export form * remove comments * Correct slugify import * Security update Python 3.2.4 due to https://snyk.io/vuln/SNYK-PYTHON-DJANGO-1298665 * Polish SSP control CSV export form Co-authored-by: Greg Elin <greg.elin@govready.com> * Add 'Create a template' button to template library (#1610) Co-authored-by: Greg Elin <greg.elin@govready.com> * Content-Security-Policy header permit images (*), videos youtube, vimeo * quick fix for auth * quick fix for auth * quick fix for auth * quick fix for auth * quick fix for auth * Force controls csv to download to browser * quick fix for auth * quick fix for auth * test * test * test * test * test * last fix and vuln update for django * last fix and vuln update for django * 'Back' link to question to take user to previous question (#1612) * 'Back' link to question to take user to previous question * Update guidedmodules/views.py Refactor pulling back_url into project_form Co-authored-by: davidpofo <dampofo@umd.edu> * Improve back-button styling Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: davidpofo <dampofo@umd.edu> Co-authored-by: Greg Elin <greg.elin@govready.com> * WIP: Side-by-side comparison of components (#1620) * created checkbox and form for submitting components for comparison. created rough start for displaying differences between prime component and rest * for now just implementing two comparison * click to read full text after 50 chars * styling and added Control part * displaying comparisons for x number of component statements against the prime component. Styling and abstracted out the comparison block into an included template * check for pid * removing detail/summary not really necessary * Condense comparison listings into rows of a single table Co-authored-by: Greg Elin <Greg Elin> * Update CHANGELOG.md * Rename 'compare' column to 'select' in component library (#1626) Co-authored-by: Greg Elin <Greg Elin> * Remove portfolio selection modal from Start a Project process Start projects in user's default portfolio to reduce the clicks starting a project. Use the User.create_default_portfolio_if_missing method consistently to consistently create the user portfolio default portfolio. Remove the PortfolioSignupForm because registration is no longer used in registering users. Remove the '"project_form": AddProjectForm(request.user ...' passed into many templates because the navbar start app option no longer brings up the portfolio select modal. * Update tests for default portfolio * Bump VERSION, CHANGELOG * Update CHANGELOG VERSION * Add button, form to add AppSource via upload of zip file Add button, form to App Store to provide front-end UI for admininstrators to add an AppSource by uploading a zip file. This simplifies setting up an AppSource for first time users. Implementation only validates that uploaded directory is a zip file, does not check if uploaded zip file is valid AppSource directory structure. Implementation assumes apps are in the 'apps' directory. * Link to library version of component from a system's selected control component * Display systems using a component (#1618) * Display systems using a component Add method controls.element.consuming_systems to produce list of systems consuming (e.g., containing) the element. Add tab to component library component detail page to display list of systems containing the component. Also, always display OSCAL tab in component library for component detail (rather than conditional on 'enable_experimental_opencontrol' parameter). * Show component system count in tab, better projects.exists query * Replace list compression with query filters Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: davidpofo <dampofo@umd.edu> * Ge/fulltext search (#1631) * Add fulltext smt search to component library search * Note fulltext search in CHANGELOG Co-authored-by: Greg Elin <Greg Elin> * check if we are in a portfolio when starting a project. If so then use that portfolio and not the default for the user. * fixed a bug where Elements of type system were shown in the selected components for a project * Addressing github issue 1630 in group id matching. fixed a bug where Elements of type system were shown in the selected components for a project. * Add YAML intermediary file for CMMC * try/except to still do the component search for non-Postgres users. (#1633) * Add a 'blank' project with no questions useful for batch project creation (#1634) Co-authored-by: Greg Elin <Greg Elin> * td not th * Polish security objective ui * Avoid errors when project has no root_task set * Better project name when no root task set * Align project name when listing project with no root task * Support CMMC ver 1 OSCAL catalog * Fix typo * Add 'blank' compliance app to first_run * Append '-dev' to version number * Legacy Statements added as statements for import * Updating regex * Del size limit on speedyssp img upload * updated column for imp statements * Fix test shipped catalogs count * td not th * Revert "td not th" This reverts commit e7e8b9c * these values are safe * removing extra differences obj. * safe and efficiency * adding select/deselect all. checkbox container wrap. * control structure for compare button toggle * Maintain sort order of compare_list otherwise Django will order ascending * adding change component button to change what the prime component of comparison is. Still has work todo * changed to allow user passed in for parsing * remove commented out code from template * Add UI for legacy statement display. Also fix StatementTypeEnum. (#1644) * [WIP] UI to display legacy control impl smts Create a conditional display of legacy control implementation statements in control editor page. Also widen width of display of editor control statements to 1250px. * Improve display of legacy statement * StatementTypeEnum fixes. Closes #1643 Set all `StatementTypeEnum.<LABEL>.value` to `StatementTypeEnum.<LABEL>.name` in order for relevant label/term to show up in Django database admin interface. Set component library detail page Systems tab to not be inactive thereby removing the content from the System tab showing up on the Control Implementation Statements tab. Update controls.tests. Co-authored-by: Greg Elin <greg.elin@govready.com> * Update CHANGELOG * fixing styling of portfolio table * using django guardian ObjectPermissionChecker to prefetch permissions. Directly check permissions, to avoid N+1 query of perms with get_obj_perms * hide_registration revert * formatting for sid * Use StatementEnum.*.name value * removing change component comparison button for now. * implemented persistent storage of checks by changing value in hidden input with jquery. Clears storage after clicking compare button or deselecting all * Use one import record for entire file * More OSCALize id fixes. Proper Create/Update/Del of smts * Display other_statement count on confirm import delete * test test_portfolio_projects * Add project, system.root_element to import_record Add project and system.root_element to import_record in order to auto delete the project and system (and root element) when the import process for importing legacy control impl smts also creates the project. * name not value for statement enums * Sometimes there are not parameters and that is okay it is caught by the try/except block. * Captialize mission for test * name not value for enums * captial impact... Impact * testing parse for version * missed one get * is_prerelease not dev release * using is_prerelease works for checking dev * need to force login as authenticated user and then reset login * url * snyk update to avoid SQL injection vuln found in Django 3.2.4 * check if previously checked and if so then don't hide compare button. * fix conflicting migrations detected * systems-security-sensitivity-level * Fix controls/0052 StatementTypeEnum migration (#1648) Co-authored-by: Greg Elin <Greg Elin> * Update CHANGELOG.md (#1647) * Configure users on install New govready_users parameter in local/environment.json to create sample users on install. * Add Wazuh collection form to Assessments page (#1651) * Add Wazuh info via end-user form * Create SecurityService class to represent Security Service Create `sec_srvc.SecurityService` class to represent a security service from which data could be collected. Add form to Assessments page to collect info from Wazuh SecurityService. * Fix sec_srvc.py * Abstracted and made a few improvements * Fix uuid error * Fix testing for fields Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> * checking for dev user creation pw. Create reg users not admin. * Add CMMC baselines, assign baselines (#1649) * Improve CMMC links, add OSCAL methods for link content Improve CMMC catalog links to link to NIST 800-53 in GovReady. Add methods to OSCAL catalog, get control_part, guidance links Add get_control_part_by_name, get_control_guidance_links, get_guidance_related_links_by_value_in_href, and get_guidance_related_links_text_by_value_in_href to make getting link content easier. * Display related controls as links in control guidance * Properly assign CMMC baselines * Remove debugging print statements * Fix typo * Properly use StatementTypeEnum when saving smts Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Greg Elin <Greg Elin> * first_run finishing touch * Fix assessment summary link to wazuh (#1653) * Add Wazuh info via end-user form * Create SecurityService class to represent Security Service Create `sec_srvc.SecurityService` class to represent a security service from which data could be collected. Add form to Assessments page to collect info from Wazuh SecurityService. * Fix sec_srvc.py * Abstracted and made a few improvements * Fix uuid error * Fix testing for fields * Fix assessment summary link to wazuh Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Alexander Ward <alexander.ward1@gmail.com> * Da/dropnfill (#1654) * adding drag and fill for component import * client-side filetype checking * client-side file size checking 5MB max. missing div. * missing listed catalog for CMMC. Delete extra migration. adjust test for drag-n-fill. import_project_submit for import project view/test * json_content not id_file * spelling * del * hot fix for external catalogs * Remove baseline controls based on control's catalog_key. Fixes failure to remove controls from another catalog when resetting baselines. (#1655) Co-authored-by: Greg Elin <greg.elin@govready.com> * Update SpeedSSP ssp template for multiple catalogs (#1656) Co-authored-by: Greg Elin <greg.elin@govready.com> * Update admin.py Readded AppInput * Update CHANGELOG.md Co-authored-by: Peter Kaminski <peter.kaminski@govready.com> Co-authored-by: alexanderward <alexander.ward1@gmail.com> Co-authored-by: Azhar Mian <azharem@gmail.com> Co-authored-by: Greg Elin <greg.elin@govready.com> Co-authored-by: Greg Elin <gregelin@govready.com> Co-authored-by: Greg Elin <Greg Elin> Co-authored-by: mike <mike.guelfi@gmail.com> Co-authored-by: GovReady-Q <govready-q@govready.com> Co-authored-by: Mike Guelfi <52943685+mguelfi@users.noreply.github.com>

Faster retrieval of component cntl smts, show cntl titles

1f4d695

Retrieve component control statement much faster by getting just related catalog_control_as_dict for statement. Also add statement property to get control title. Display control title on component control pages.

gregelin added bug better technical debt GRC feature Feature related to Governance, Risk, and Compliance capabilities labels Apr 23, 2021

gregelin requested a review from davidpofo April 23, 2021 11:30

gregelin assigned davidpofo Apr 23, 2021

Update element_detail_tabs.html

400b797

system_tags aren't used anywhere

davidpofo approved these changes Apr 23, 2021

View reviewed changes

gregelin merged commit 11fb374 into develop Apr 23, 2021

davidpofo deleted the ge/faster-cmpt-smts branch April 23, 2021 14:23

mguelfi mentioned this pull request Apr 29, 2021

Release/v0.9.3.5 (#1586) mguelfi/govready-q#1

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Faster retrieval of component control statements, show control titles #1570

Faster retrieval of component control statements, show control titles #1570

gregelin commented Apr 23, 2021

davidpofo left a comment

Faster retrieval of component control statements, show control titles #1570

Faster retrieval of component control statements, show control titles #1570

Conversation

gregelin commented Apr 23, 2021

davidpofo left a comment

Choose a reason for hiding this comment