Draft: support clearing reports and CLIs in SBOM and in new command "bom downloadattachments"

[gernot-h]

First draft for #24

[gernot-h]

@tngraf, while this is far from complete, I would like to ask for your opinion about storing the attachment information in the SBOM.

For now, project createbom adds external references for the attachments like these:

        {
          "url": "CLIXML_libxml2_2.9.4+dfsg1-2.2+deb9u3-debian-combined.zip_2020-11-09_12_40_19.xml",
          "comment": "component license information (local copy), sw360Id: 0546f2...",
          "type": "other",
          "hashes": [
            {
              "alg": "SHA-1",
              "content": "b272c58e300713207fc963ea5a792bc00835f730"
            }
          ]
        },
        {
          "url": "libxml2_2.9.4+dfsg1-2.2+deb9u3-debian-combined.zip_clearing_report_Mon_Nov_09_11_2020_12_22_58.docx",
          "comment": "clearing report (local copy), sw360Id: 0546f3...",
          "type": "other",
          "hashes": [
            {
              "alg": "SHA-1",
              "content": "6de0b76d42c3ef9f8aa7254bdb868625385df30f"
            }
          ]
        }

I think it's a good idea to store the sw360Id of the attachments in the SBOM so I don't need to search for the file for later bom downloadattachments. I'm however unsure about how to store it. As first try, I add it to the comment string per external reference, see example above.

And the same problem applies to later download step, where I'd also like to store the per-attachment information like createdBy, checkedBy, createdOn etc. so I can filter and document them for later upload steps.

In project getlicenseinfo you store them in a proprietary JSON format like this:

    {
      "ComponentName": "aufs-tools 4.1+20161219",
      "CliFile": "folder/CLIXML_aufs-tools_1:4.1+20161219-1-debian-combined.tar.bz2_2022-05-12_10:43:09.xml",
      "CreatedBy": "user@siemens.com",
      "CreatedOn": "2022-05-12",
      "CheckedBy": "admin@siemens.com",
      "CheckedTeam": "AA",
      "CheckStatus": "ACCEPTED"
    }

This however won't work if I have multiple CLI and report attachments.

So I can think about the following options:

• create a separate output file with another proprietary JSON format to hold all attachment metadata
• add all the metadata to the SBOM. As far as I can see, you cannot add custom properties to external references, so I would need to think about some way to store them either in comments there (ugly!) or in some flat variant in the component's custom properties

All the options sound a bit ugly to me... :)

[gernot-h]

I integrated the current implementation of "project createbom" and "bom downloadattachments" into some internal attachment import scripts (https://code.siemens.com/linux/distro-clearing/debian) where it seems to work well.

So I consider this part as "implemented" (tm), but I'm still unsure about the way how to store additional data for the attachments. I now went for creating a proprietary JSON "control file" with similar syntax to the output of "project getlicenseinfo" as described above.

Another possibility would be to create some kind of "extended BOM" adding the clearing reports and CLI files as own CycloneDX components of type "file" and linking to them using the bom-ref mechanism, inspired by the discussion in CycloneDX/specification#135 (comment). Then I could add the interesting data as CycloneDX properties for these components.

[gernot-h]

@tngraf and just had a quick look on the MR and decided to continue with the two-file approach. Open TODOs:

• improve documentation for new commands, including difference between "bom downloadsources" and "bom downloadattachments"
• add parameters to "bom downloadsources" to specify type of download (CLI, Report and probably also SOURCE and SOURCE_SELF attachments)
• add parameter to "project createbom" to specify control file
• add "project createbom" to main command overview
• keep "project GetLicenseInfo", but mark it as deprecated and replaced by new commands