Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failed to scan Debian Docker image #1

Closed
pombredanne opened this issue Sep 11, 2020 · 1 comment · Fixed by #3
Closed

Failed to scan Debian Docker image #1

pombredanne opened this issue Sep 11, 2020 · 1 comment · Fixed by #3
Labels
bug Something isn't working

Comments

@pombredanne
Copy link
Member

While scanning https://hub.docker.com/_/mongo with docker.py pipeline we get:

        "01-debian-agpl-sspl-mongo-latest.tar"
    ],
    "next_run": null,
    "runs": [
        {
            "url": "http://127.0.0.1:8001/api/runs/c9337b56-04b6-45c4-a1e8-aa82c85edb19/",
            "pipeline": "scanpipe/pipelines/docker.py",
            "description": "A pipeline to analyze a Docker image.",
            "project": "http://127.0.0.1:8001/api/projects/28f6738c-f2f8-49d9-8299-ab6ff9e9987f/",
            "uuid": "c9337b56-04b6-45c4-a1e8-aa82c85edb19",
            "run_id": "1599755581029165",
            "created_date": "2020-09-10T16:32:58.966347Z",
            "task_id": "7771f598-6b51-434e-b663-96b09b4488e1",
            "task_start_date": "2020-09-10T16:32:59.017750Z",
            "task_end_date": "2020-09-10T16:38:01.455060Z",
            "task_exitcode": 1,
            "task_output": [
                "Validating your flow...",
                "    The graph looks good!",
                "Running pylint...",
                "    Pylint is happy!",
                "2020-09-10 16:33:01.031 Workflow starting (run-id 1599755581029165):",
                "2020-09-10 16:33:01.035 [1599755581029165/start/1 (pid 26030)] Task is starting.",
                "2020-09-10 16:33:01.990 [1599755581029165/start/1 (pid 26030)] Task finished successfully.",
                "2020-09-10 16:33:01.995 [1599755581029165/extract_images/2 (pid 26037)] Task is starting.",
                "2020-09-10 16:33:04.129 [1599755581029165/extract_images/2 (pid 26037)] Task finished successfully.",
                "2020-09-10 16:33:04.134 [1599755581029165/extract_layers/3 (pid 26043)] Task is starting.",
                "2020-09-10 16:33:05.773 [1599755581029165/extract_layers/3 (pid 26043)] Task finished successfully.",
                "2020-09-10 16:33:05.838 [1599755581029165/find_images_linux_distro/4 (pid 26049)] Task is starting.",
                "2020-09-10 16:33:06.812 [1599755581029165/find_images_linux_distro/4 (pid 26049)] Task finished successfully.",
                "2020-09-10 16:33:06.816 [1599755581029165/collect_images_information/5 (pid 26055)] Task is starting.",
                "2020-09-10 16:33:07.782 [1599755581029165/collect_images_information/5 (pid 26055)] Task finished successfully.",
                "2020-09-10 16:33:07.786 [1599755581029165/collect_and_create_codebase_resources/6 (pid 26062)] Task is starting.",
                "2020-09-10 16:33:54.536 [1599755581029165/collect_and_create_codebase_resources/6 (pid 26062)] Task finished successfully.",
                "2020-09-10 16:33:54.541 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)] Task is starting.",
                "2020-09-10 16:38:00.696 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)] <flow DockerPipeline step collect_and_create_system_packages> failed:",
                "2020-09-10 16:38:00.697 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     Internal error",
                "2020-09-10 16:38:00.697 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)] Traceback (most recent call last):",
                "2020-09-10 16:38:01.054 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/metaflow/cli.py\", line 883, in main",
                "2020-09-10 16:38:01.054 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     start(auto_envvar_prefix='METAFLOW', obj=state)",
                "2020-09-10 16:38:01.054 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/click/core.py\", line 829, in __call__",
                "2020-09-10 16:38:01.054 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     return self.main(args, kwargs)",
                "2020-09-10 16:38:01.054 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/click/core.py\", line 782, in main",
                "2020-09-10 16:38:01.054 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     rv = self.invoke(ctx)",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/click/core.py\", line 1259, in invoke",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     return _process_result(sub_ctx.command.invoke(sub_ctx))",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/click/core.py\", line 1066, in invoke",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     return ctx.invoke(self.callback, ctx.params)",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/click/core.py\", line 610, in invoke",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     return callback(args, kwargs)",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/click/decorators.py\", line 33, in new_func",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     return f(get_current_context().obj, args, kwargs)",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/metaflow/cli.py\", line 444, in step",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     max_user_code_retries)",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/metaflow/task.py\", line 394, in run_step",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     self._exec_step_function(step_func)",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/metaflow/task.py\", line 47, in _exec_step_function",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     step_function()",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"scanpipe/pipelines/docker.py\", line 87, in collect_and_create_system_packages",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     docker_pipes.scan_image_for_system_packages(self.project, image)",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/scanpipe/pipes/docker.py\", line 105, in scan_image_for_system_packages",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     for i, (purl, package, layer) in enumerate(installed_packages):",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/container_inspector/image.py\", line 329, in get_installed_packages",
                "2020-09-10 16:38:01.055 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     for purl, package in layer.get_installed_packages(packages_getter):",
                "2020-09-10 16:38:01.056 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/scanpipe/pipes/debian.py\", line 16, in package_getter",
                "2020-09-10 16:38:01.056 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     for package in packages:",
                "2020-09-10 16:38:01.056 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/packagedcode/debian.py\", line 211, in get_installed_packages",
                "2020-09-10 16:38:01.056 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     for package in parse_status_file(base_status_file_loc, distro=distro):",
                "2020-09-10 16:38:01.056 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]   File \"/tmp/scancode.io/lib/python3.6/site-packages/packagedcode/debian.py\", line 228, in parse_status_file",
                "2020-09-10 16:38:01.056 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)]     raise FileNotFoundError('[Errno 2] No such file or directory: {}'.format(repr(location)))",
                "2020-09-10 16:38:01.056 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)] FileNotFoundError: [Errno 2] No such file or directory: '/tmp/scancode.io/var/projects/sspl-28f6738c/codebase/01-debian-agpl-sspl-mongo-latest.tar-extract/6f90c94ad68f6b08882985f9884f3154469709ca8af796d52726ac7562f7ff1c/var/lib/dpkg/status'",
                "2020-09-10 16:38:01.056 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)] ",
                "2020-09-10 16:38:01.057 [1599755581029165/collect_and_create_system_packages/7 (pid 26104)] Task failed.",
                "2020-09-10 16:38:01.057 Workflow failed.",
                "2020-09-10 16:38:01.057 Terminating 0 active tasks...",
                "2020-09-10 16:38:01.057 Flushing logs...",
                "    Step failure:",
                "    Step collect_and_create_system_packages (task-id 7) failed.",
                ""
            ],
            "execution_time": 302
@pombredanne pombredanne added the bug Something isn't working label Sep 11, 2020
@pombredanne
Copy link
Member Author

@tdruez aboutcode-org/scancode-toolkit#2224 is a fix for this.
It seems to work OK and I will push a new rc2 of scancode anytime

pombredanne added a commit that referenced this issue Sep 11, 2020
@pombredanne
Use thew newest ScanCode TK version #1
1126472 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
pombredanne added a commit that referenced this issue Sep 12, 2020
@pombredanne
Update Changelog for #1
b592aad 
Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>
@pombredanne pombredanne mentioned this issue Sep 12, 2020
Merged
tdruez added a commit that referenced this issue Sep 14, 2020
@tdruez
Exclude the "tool_version" from the reference data unit test #1
35dd99e 
Signed-off-by: Thomas Druez <tdruez@nexb.com>
@tdruez tdruez closed this as completed in #3 Sep 14, 2020
tdruez added a commit that referenced this issue Sep 14, 2020
@pombredanne @tdruez
Update scancode-toolkit to newest version #1 (#3)
b7cdeb3 
* Use thew newest ScanCode TK version #1

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

* Update Changelog for #1

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

* Bump version to 1.0.1

Signed-off-by: Philippe Ombredanne <pombredanne@nexb.com>

* Exclude the "tool_version" from the reference data unit test #1

Signed-off-by: Thomas Druez <tdruez@nexb.com>

Co-authored-by: Thomas Druez <tdruez@nexb.com>
@mjherzog mjherzog mentioned this issue Mar 27, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update scancode-toolkit to newest version #1 aboutcode-org/scancode.io
1 participant
@pombredanne