Safety critical systems adaptations

[neerdoc]

I believe that doorstop should be improved to allow the use in safety-critical domains. The discussions have been ongoing for some time, and I do believe that there is somewhat of a consensus that it should be possible, after all, it does intend to replace one of the major players in the safety-critical requirements domain!

This issue tries to gather all of the discussions around this:

• Doorstep and ISO 26262 / EN 61508 #509
• Child with multiple parents #292
• Doorstop and aviation industry? #309
• Questions on scalability of doorstop #430
• Add support for digital signatures of items #375
• Add option to check derived requirements have rationale #142
• Add a way to create a traceability report. #100

If the decision is made to improve doorstop to the point of it actually being possible to use as-is in a safety-critical project, a number of changes should be done. Anyone with experience in the area should feel free to add to the list:

• Multiple parents - requirements and/or texts can have multiple parents, so this should be allowed.
• History - an easy way to extract the complete history of a single requirement. This would solve many traceability requirements, e.g., DO-178.
• Diff publish - Publish a difference report between two git-hashes (not tags since they can be changed).
• Look over hash calculation - make sure that everything that should be tracked for changes is included in the hash calculation, even custom attributes.
• Rules to check that different attributes are within limits.
• Add test suites specifically designed to show compliance to different standards. NOTE: There is a possibility that different standards will have different requirements on doorstop. If so, this can be solved with customized builds and test suites.
• Include text from links - Reduces duplication in documentation. E.g., often in a DO-178 project, you would need to have multiple documents containing the same text/paragraph to make them reviewable, but then you instead risk differentiating (and thus introducing errors) by writing the same text in the requirements specification, the design specification, the test specification, the verification report, etc.
• Link attributes - such as text as include text, enabled-by, roles, etc.
• Document outline - there should be an easier way of organizing a complete document than changing the level of each and everyone of the items. I like the approach suggested by @tangoalx in RFC: "One document - one markdown file" (instead of multiple YAML files) #401, although I believe it should be simplified into just a list of items. The text of each individual item should still be stored in a separate yaml-file. The level-attribute might not be required then either.
• Reviewed tag improvements - multiple reviewers, with at least roles, dates and effort.
• Look over speed - if used in a large project, we are likely talking about (at least) tens-of-thousands of items. Can doorstop handle that within a reasonable time? Setup some benchmarks and evaluate, e.g., validation time, publish time, etc.
• Improved GUI - If needed. I personally think that editing in a text editor is fine, but for a more widespread adoptation in existing organisations I too see the value of a GUI. What about using doorhole instead of a build-in GUI? I.e., remove GUI in doorstop and refer to doorhole.
• Remember last added item - Currently, a removed item can be re-added. This must not be allowed to fulfill DO-178 and other standards. Reason: UUID functionality, each requirement must be uniquely identifiable. The common way of doing this is by name, e.g., REQ-499, and if that can change to mean something else years down the road that will be bad.
• git hash on publish command - All publish commands should support showing the git hash of the commit. This opens for external reviews of generated PDFs.
• Add rationale to all normative items.

[alexkm]

I think this should also link to #471

Multiple parents - requirements and/or texts can have multiple parents, so this should be allowed.

I would certainly find this useful, particularly coupled with "link attributes" later on in your list.

History - an easy way to extract the complete history of a single requirement. / Diff publish - Publish a difference report between two git-hashes

Should this not be achieved using your version control system rather than doorstop itself? The doorstop items are all individually tracked files.

make sure that everything that should be tracked for changes is included in the hash calculation, even custom attributes

This can be done already in the .doorstop.yml for a document. You can set custom attributes to be included in the hash:

attributes: 
  reviewed:
    - rationale
    - type
  defaults:
    type: requirement
    rationale:
    notes:
  publish:
    - rationale
    - notes

Rules to check that different attributes are within limits.

This can be done already with a validation script using Validation Hooks.

git hash on publish command

Doorstop currently supports more than just git. It would be a shame to limit this to git only. Perhaps a hook to add a calculated field or environment variable to published doc instead? Could then call a script to grab changeset id, or any other relevant detail.

Add rationale to all normative items.

I think this should be optional, and can be achieved already with custom attributes in .doorstop.yml and enforced with a validation script.

[ckolumbus]

Hi everyone interested in "multiple parents": i've implemented a suggestion that mimics the behavior that we use within the healthcare environment #569 .

Could you please review the proposal and let me know whether this would solve this for you scenarios as well?

[jmvillalba]

@neerdoc Great post. I'm reading it slowly. Really interesting.

@ckolumbus I'm going to check your multiple parents solution.

I think that this post is the future of doorstop. @jacebrowning have the requirement that the implementation don't break the validation and publishing implementation.

If @jacebrowning let us know if the @ckolumbus implementation concept it's fine for him it will be a great support in order to concentrate efforts.

Best regards

[ckolumbus]

@jmvillalba if you have suggestions please let me know. I'm happy to identify the best possible solution together 👍🏽

[RoggeHaj]

@neerdoc

History - an easy way to extract the complete history of a single requirement. This would solve many traceability requirements, e.g., DO-178.

What part of DO-178 requires the complete history for a requirement? I was under the impression that you only need to be able to provide the versions relevant for specific reviews.

[neerdoc]

@RoggeHaj

What part of DO-178 requires the complete history for a requirement? I was under the impression that you only need to be able to provide the versions relevant for specific reviews.

I was perhaps a bit sweeping in my statement. You are correct in that pure traceability between requirements, design, implementation, tests and binary is only required in a specific review. But if you take in the larger picture with change control with baselines and traceability therein, it is clear (at least to me) that perserving the history of a requirement is one of the easier ways to achieve this.

As an example, if you have done a CDR for a software and then find a change that needs to be implemented due to whatever reason, then if you have a complete history of the changes and can prove that, the new CDR must only handle the change and all items it affects. If you do not have the difference, the new CDR must handle everthing.

Another example is requirement gliding. Especially for large organizations with software that are around for many years, it will be crucial to be able to trace changes backwards in time to when a requirement was changed, by whom, from what and why.

A third example would be to prove life-cycle data points for a software.

So, to summarize: In my head I tend to handle traceability, change management and life cycle data as one as they are easily controlled in the same manner; as an example, by the complete history of all items.

[robtaylor]

@neerdoc: Would it be possible to make a minimal list of requirements for FuSa use of Doorstop? This list seems pretty maximal and has a few "wish list" items. A minimal set of "must haves" could help move this on.