Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

bug(pom): wrong inheritance of version and scope from root DepManagement in parent dependencies #7539

Closed
DmitriyLewen opened this issue Sep 18, 2024 Discussed in #7537 · 15 comments · Fixed by #7541
Closed

bug(pom): wrong inheritance of version and scope from root DepManagement in parent dependencies #7539

DmitriyLewen opened this issue Sep 18, 2024 Discussed in #7537 · 15 comments · Fixed by #7541
Assignees
@DmitriyLewen
Labels
kind/bug Categorizes issue or PR as related to a bug.
Milestone
v0.57.0

Comments

@DmitriyLewen
Copy link
Contributor

Description

We need to correctly inherit some fields from the root DepManagement in parent dependencies:

  • always use version from root DepManagement and don't use properties from parent.
  • overwrite scope only if parent dependency doesn't use scope

Example:
root pom:

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>

    <parent>
        <groupId>org.example</groupId>
        <artifactId>example-parent</artifactId>
        <version>4.0.0</version>
        <relativePath>./parent/pom.xml</relativePath>
    </parent>

    <groupId>com.example</groupId>
    <artifactId>child-depManagement-in-parent</artifactId>
    <version>1.0.0</version>


    <properties>
        <api.version>1.0.1</api.version>
        <api2.version>1.0.2</api2.version>
    </properties>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>org.example</groupId>
                <artifactId>example-api</artifactId>
                <version>${api.version}</version>
                <scope>runtime</scope>
            </dependency>
            <dependency>
                <groupId>org.example</groupId>
                <artifactId>example-api2</artifactId>
                <version>${api2.version}</version>
                <scope>runtime</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>
</project>

Parent pom:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

    <modelVersion>4.0.0</modelVersion>

    <groupId>org.example</groupId>
    <artifactId>example-parent</artifactId>
    <version>4.0.0</version>
    <packaging>pom</packaging>

    <properties>
        <api.version>4.0.1</api.version>
    </properties>

    <dependencies>
        <dependency>
            <groupId>org.example</groupId>
            <artifactId>example-api</artifactId>
            <version>${api.version}</version>
            <scope>compile</scope>
        </dependency>
        <dependency>
            <groupId>org.example</groupId>
            <artifactId>example-api2</artifactId>
        </dependency>
    </dependencies>
</project>

mvn result:

[INFO] com.example:child-depManagement-in-parent:jar:1.0.0
[INFO] +- org.example:example-api:jar:1.0.1:compile
[INFO] \- org.example:example-api2:jar:1.0.2:runtime

Discussed in #7537
@DmitriyLewen DmitriyLewen added the kind/bug Categorizes issue or PR as related to a bug. label Sep 18, 2024
@DmitriyLewen DmitriyLewen self-assigned this Sep 18, 2024
@DmitriyLewen DmitriyLewen mentioned this issue Sep 18, 2024
Merged
7 tasks
@DmitriyLewen
Copy link
Contributor Author

always use version from root DepManagement and don't use properties from parent.
overwrite scope only if parent dependency doesn't use scope

After investigation I realized that this is incorrect.
For parents we need to inherit DependencyManagement and Properties as non-rootDepManagements.
In this case we will be correctly overwrite empty scopes/version, use properties, etc.

@knqyf263 knqyf263 added this to the v0.57.0 milestone Sep 30, 2024
@knqyf263
Copy link
Collaborator

I wanted to play with maven, but faced an error.

cd $GOPATH/src/github.com/aquasecurity/trivy/pkg/dependency/parser/java/pom/testdata
docker run --rm -it -v $PWD:/testdata maven:3-amazoncorretto bash
cp /testdata/settings/global/conf/settings.xml /usr/share/maven/conf/settings.xml
sed -i 's/testdata/\/testdata/g' /usr/share/maven/conf/settings.xml
cd inherit-scopes-in-parents-from-root/
mvn dependency:list
...
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.121 s
[INFO] Finished at: 2024-10-16T08:16:45Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project inherit-scopes-in-parents-from-root: Could not resolve dependencies for project com.example:inherit-scopes-in-parents-from-root:jar:0.1.0
[ERROR] dependency: org.example:example-nested-scope-runtime:jar:1.0.0 (runtime)
[ERROR]         Could not find artifact org.example:example-nested-scope-runtime:jar:1.0.0 in central (https://repo.maven.apache.org/maven2)
[ERROR] dependency: org.example:example-scope-runtime:jar:2.0.0 (compile)
[ERROR]         Could not find artifact org.example:example-scope-runtime:jar:2.0.0 in central (https://repo.maven.apache.org/maven2)
[ERROR] dependency: org.example:example-api-runtime:jar:3.0.0 (runtime)
[ERROR]         Could not find artifact org.example:example-api-runtime:jar:3.0.0 in central (https://repo.maven.apache.org/maven2)

I ran maven with --debug and it seems like the repository configuration is successfully loaded.

[DEBUG] Reading global settings from /usr/share/maven/conf/settings.xml
[DEBUG] Reading user settings from /root/.m2/settings.xml
[DEBUG] Reading global toolchains from /usr/share/maven/conf/toolchains.xml
[DEBUG] Reading user toolchains from /root/.m2/toolchains.xml
[DEBUG] Using local repository at /testdata/repository
[DEBUG] Using manager EnhancedLocalRepositoryManager with priority 10.0 for /testdata/repository

How do you test maven?

@DmitriyLewen
Copy link
Contributor Author

I just copied org folder to .m2 directory.

@knqyf263
Copy link
Collaborator 

cd /Users/teppei/src/github.com/aquasecurity/trivy/pkg/dependency/parser/java/pom/testdata
docker run --rm -it -v $PWD:/testdata maven:3-amazoncorretto bash
cp -r /testdata/repository/* /root/.m2/repository/
cd /testdata/inherit-scopes-in-parents-from-root/
mvn dependency:list

Hmm... it didn't help either.

@DmitriyLewen
Copy link
Contributor Author

mvn dependecy:tree works:

bash-4.2# mvn dependency:tree
[INFO] Scanning for projects...
[INFO] 
[INFO] ----------< com.example:inherit-scopes-in-parents-from-root >-----------
[INFO] Building inherit-scopes-in-parents-from-root 0.1.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.example:example-api-runtime:jar:3.0.0 is missing, no dependency information available
[WARNING] The POM for org.example:example-api-compile:jar:3.0.0 is missing, no dependency information available
[WARNING] The POM for org.example:example-api-empty:jar:3.0.0 is missing, no dependency information available
[INFO] 
[INFO] --- dependency:3.7.0:tree (default-cli) @ inherit-scopes-in-parents-from-root ---
[INFO] com.example:inherit-scopes-in-parents-from-root:jar:0.1.0
[INFO] +- org.example:example-nested-scope-runtime:jar:1.0.0:runtime
[INFO] |  \- org.example:example-scope-runtime:jar:2.0.0:compile
[INFO] |     \- org.example:example-api-runtime:jar:3.0.0:runtime
[INFO] +- org.example:example-nested-scope-compile:jar:1.0.0:compile
[INFO] |  \- org.example:example-scope-compile:jar:2.0.0:runtime
[INFO] |     \- org.example:example-api-compile:jar:3.0.0:test
[INFO] \- org.example:example-nested-scope-empty:jar:1.0.0:test
[INFO]    \- org.example:example-scope-empty:jar:2.0.0:test
[INFO]       \- org.example:example-api-empty:jar:3.0.0:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.395 s
[INFO] Finished at: 2024-10-16T08:54:41Z
[INFO] ------------------------------------------------------------------------

@knqyf263
Copy link
Collaborator

What??? Shouldn't dependency:list and dependency:tree be the same for resolving dependencies? It's really weird...

@knqyf263
Copy link
Collaborator

knqyf263 commented Oct 16, 2024 •
edited
Loading

Even if maven cannot resolve dependencies, mvn dependency:tree seems to say "BUILD SUCCESS"...

[INFO] --------------------------------[ jar ]---------------------------------
[WARNING] The POM for org.example:example-nested-scope-runtime:jar:1.0.0 is missing, no dependency information available
[WARNING] The POM for org.example:example-nested-scope-compile:jar:1.0.0 is missing, no dependency information available
[WARNING] The POM for org.example:example-nested-scope-empty:jar:1.0.0 is missing, no dependency information available
[INFO]
[INFO] --- dependency:3.7.0:tree (default-cli) @ inherit-scopes-in-parents-from-root ---
[INFO] com.example:inherit-scopes-in-parents-from-root:jar:0.1.0
[INFO] +- org.example:example-nested-scope-runtime:jar:1.0.0:runtime
[INFO] +- org.example:example-nested-scope-compile:jar:1.0.0:compile
[INFO] \- org.example:example-nested-scope-empty:jar:1.0.0:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.542 s
[INFO] Finished at: 2024-10-16T09:03:01Z
[INFO] ------------------------------------------------------------------------

@knqyf263
Copy link
Collaborator

This output also shows warnings.

@knqyf263
Copy link
Collaborator

example-api-xxx doesn't exist under the repository directory.

@DmitriyLewen
Copy link
Contributor Author

I thought there was no point in adding these dependencies because they were at the top of the tree (I didn't know that mvn dependency:list doesn't work because of this).

@knqyf263
Copy link
Collaborator

I thought there was no point in adding these dependencies because they were at the top of the tree (I didn't know that mvn dependency:list doesn't work because of this).

In addition to missing dependencies, dependency:list seems to require a jar file. I've created an empty jar, and it made dependency:list succeed.

bash-4.2# mvn dependency:list
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------< com.example:happy >--------------------------
[INFO] Building happy 1.0.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
Downloading from central: https://repo.maven.apache.org/maven2/org/example/example-api/1.7.30/example-api-1.7.30.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.656 s
[INFO] Finished at: 2024-10-16T09:31:28Z
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal on project happy: Could not resolve dependencies for project com.example:happy:jar:1.0.0
[ERROR] dependency: org.example:example-api:jar:1.7.30 (compile)
[ERROR]         Could not find artifact org.example:example-api:jar:1.7.30 in central (https://repo.maven.apache.org/maven2)
[ERROR]
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
bash-4.2# touch /root/.m2/repository/org/example/example-api/1.7.30/example-api-1.7.30.jar
bash-4.2# mvn dependency:list
[INFO] Scanning for projects...
[INFO]
[INFO] -------------------------< com.example:happy >--------------------------
[INFO] Building happy 1.0.0
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- dependency:3.7.0:list (default-cli) @ happy ---
[INFO] Can't extract module name from example-api-1.7.30.jar: zip file is empty
[INFO]
[INFO] The following files have been resolved:
[INFO]    org.example:example-api:jar:1.7.30:compile
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  0.527 s
[INFO] Finished at: 2024-10-16T09:32:19Z
[INFO] ------------------------------------------------------------------------

@knqyf263
Copy link
Collaborator

I thought there was no point in adding these dependencies because they were at the top of the tree

Yes, I think so, too, but our test cases don't work with dependency:list, so I'm a little worried that we are testing Maven's behavior correctly.

@DmitriyLewen
Copy link
Contributor Author

I think using mvn dependency:tree is enough (also less hassle (no need to create jar file and dependencies for which you don't need children).
But let me know if you find important differences.

@knqyf263
Copy link
Collaborator

I think using mvn dependency:tree is enough

We don't know whether mvn dependency:tree works in the same way as mvn install or mvn dependency:list. Ideally, we should update our tests to work with dependency:list. However, it's a big effort. We can defer it until we find a problem or someone complains.

@DmitriyLewen
Copy link
Contributor Author

However, it's a big effort. We can defer it until we find a problem or someone complains.

I agree with you. I will check new changes i will make using dependency:list.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DmitriyLewen DmitriyLewen

Labels
kind/bug Categorizes issue or PR as related to a bug.
Projects
Trivy Roadmap
Status: No status
Milestone
v0.57.0
2 participants
@knqyf263 @DmitriyLewen