-
Notifications
You must be signed in to change notification settings - Fork 7
Analysis-target filtering #140
Conversation
src/App/Fossa/Analyze.hs
Outdated
[ Bundler.discover', | ||
Cargo.discover', | ||
Carthage.discover', | ||
Cocoapods.discover', | ||
Gradle.discover', | ||
Rebar3.discover', | ||
Gomodules.discover', | ||
Godep.discover', | ||
Setuptools.discover', | ||
Maven.discover', | ||
Leiningen.discover', | ||
Composer.discover', | ||
Cabal.discover', | ||
Stack.discover', | ||
Yarn.discover', | ||
Npm.discover', | ||
Scala.discover', | ||
RPM.discover', | ||
RepoManifest.discover' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have we decided on a name? Are we sticking with discover'
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just renamed to discover
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Needs some items addressed as already discussed, but overall PR is in good shape. Last remaining bits after those changes is to confirm that all strategies have been migrated, check pathfinder still works (with licenses), check the fork safety issue you mentioned, and do a VERY short write up of how we tackle the easy discovery tests (in a later follow-up PR on the test repo, of course).
Additionally, this PR should probably update the GH actions workflow to fix our licensing issues. Since the cabal strategy has been already migrated, there should be no blocker there.
This is a big one.
Summary
See the design doc for additional context.
User-facing additions
fossa analyze --filter ANALYSIS-TARGET
This allows for project-wide filtering or analysis-target-specific filtering. When more than one filter is specified, an analysis-target that passes any of the filters will be analyzed
fossa list-targets
This command lists valid analysis-targets that can be used with
--filter
.Example output:
Implementation notes
strategy-focused pipeline
Historically, we've had a very "strategy"-focused analysis pipeline, where we have a bunch of possibly-related strategies, that don't know about each other, that all go try to find and analyze projects independently.
For example, a yarn-lock strategy that only does static analysis on yarn.lock would be run independently of a yarn-command-running strategy. We'd pick the "best" strategy's dependency graph when deduplicating projects.
This has a couple of downsides:
As part of these changes, we move toward a more project-focused analysis (e.g., "yarn"). This is critical for allowing users to select projects as analysis-targets
subprojects
Subprojects are not a well-supported notion in our current datatypes, leading to the hacky workaround of a
SubprojectType
dependency type. We need to reify subprojects as a first-class construct, to both eliminate that tech debt, but also to allow users to select subprojects as analysis-targets.These changes reify "BuildTargets" as part of the project, which are typically subprojects -- but in the future, things like bazel build targets will fit there too.
Bonus improvements
Future tickets
Features
list-targets