fossas · elldritch · Aug 2, 2021 · Jul 20, 2021 · Jul 21, 2021 · Jul 24, 2021
diff --git a/Changelog.md b/Changelog.md
@@ -1,5 +1,9 @@
 # Spectrometer Changelog
 
+## v2.12.3
+
+- Fixes an issue where unresolvable Gradle configurations would cause Gradle analysis to show no dependencies ([#292](https://github.com/fossas/spectrometer/pull/292)).
+
 ## v2.12.2
 
 - Supports poetry build backend of `poetry.masonry.api` ([#309](https://github.com/fossas/spectrometer/pull/309))
@@ -16,7 +20,7 @@
 
 ## v2.11.1
 
-- Support CPAN deps in the fossa-deps file ([#296](https://github.com/fossas/spectrometer/pull/296)) 
+- Support CPAN deps in the fossa-deps file ([#296](https://github.com/fossas/spectrometer/pull/296))
 
 ## v2.11.0
 

diff --git a/docs/quickreference/gradle.md b/docs/quickreference/gradle.md
diff --git a/docs/strategies.md b/docs/strategies.md
@@ -1,6 +1,73 @@
 # Analysis Strategies
 
-We support dependency analysis for the following languages/ecosystems:
+The CLI performs dependency analysis using a set of "strategies".
+
+Strategies define how to identify user projects and how to determine the dependencies of each project. Each strategy corresponds to roughly one tool, language, or ecosystem.
+
+<!-- omit in toc -->
+## Table of contents
+
+- [Anatomy of a strategy](#anatomy-of-a-strategy)
+  - [Analysis targets](#analysis-targets)
+  - [Discovery and analysis](#discovery-and-analysis)
+  - [Tactics](#tactics)
+- [Supported strategies](#supported-strategies)
+
+## Anatomy of a strategy
+
+### Analysis targets
+
+When the CLI identifies user projects to analyze, it groups these projects into "analysis targets". The exact semantics of an analysis target depends on the language, but usually a target is a single module, package, library, or program. Usually, an analysis target is equivalent to whatever you would create a dependency manifest file for.
+
+Analysis targets have exactly one type (which identifies which strategy is used to analyze the target) and produce exactly one dependency graph.
+
+A single Analysis (i.e. a single run of the CLI) includes every analysis target discovered in the folder being analyzed.
+
+### Discovery and analysis
+
+Strategies usually have two phases:
+
+1. A _discovery_ phase, to determine which user projects are in the folder being analyzed.
+2. An _analysis_ phase, where dependencies of discovered user projects are analyzed.
+
+_Discovery_ usually works by examining the file system for special files that indicate the presence of a project. For example, a `package.json` usually indicates the presence of an NPM project, or a `pom.xml` usually indicates the presence of a Maven project.
+
+_Analysis_ usually works by running through a series of _tactics_ for each strategy. For example, to analyze the dependencies of an NPM project, we might try to parse `package-lock.json` or execute `npm ls`.
+
+Each strategy defines its own logic for how it does discovery, and its own logic for which tactics it uses to analyze dependencies.
+
+### Tactics
+
+Strategies use one or more _tactics_ to analyze the dependencies or discovered projects.
+
+Each strategy defines its own tactics. Strategies generally have multiple tactics, and will automatically select between tactics depending on what information is available.
+
+Tactics tend to vary across two axes:
+
+1. What _requirements_ must be fulfilled for the tactic to succeed?
+2. What _structure_ does the tactic provide in its output?
+
+The relevant questions for _requirements_ are usually:
+
+1. Does this tactic perform dynamic or static analysis? Tactics that do dynamic analysis usually require a functioning build environment and a CI integration, while tactics that only do static analysis only require a copy of the source code.
+2. What files are necessary? Some tactics require files such as lockfiles that are not always present by default.
+3. What build environment is necessary? Some tactics require running after successful builds, or being able to install build plugins.
+
+The relevant questions for _structure_ are usually:
+
+1. Does this tactic provide all dependencies? Some tactics are only able to provide direct dependencies, installed dependencies, or some other subset of dependencies.
+2. Does this tactic provide a graph or a list? Some tactics don't provide edge information between dependencies.
+3. Does this tactic provide additional dependency metadata? Some tactics are able to tag dependencies with metadata, such as whether they're test dependencies.
+
+Each strategy's documentation contains information about its tactics and how it chooses between them.
+
+## Supported strategies
+
+<!--
+TODO: create a lookup table that categorizes these strategies by language or tool.
+-->
+
+The CLI supports the following strategies:
 
 - [clojure](strategies/golang.md) (lein)
 - [erlang](strategies/erlang.md) (rebar3)

diff --git a/docs/strategies/gradle.md b/docs/strategies/gradle.md
@@ -1,29 +1,188 @@
-# Gradle Analysis
+# Gradle
 
-When using gradle, users specify dependencies and plugins with a groovy DSL. This makes accurate dependency analysis particularly difficult: gradle projects tend to contain several (sub)projects with their own buildscripts, and plugins that add dependencies of their own.
+[Gradle](https://gradle.org/) is a polyglot tool mostly used by JVM projects. It's popular with Java and Android projects.
 
-To accurately determine the entire project/dependency structure, we use an
-initscript to inject our own `jsonDeps` task.
+Gradle users generally specify their builds using a `build.gradle` file (written in Groovy) or a `build.gradle.kts` file (written in Kotlin). These builds are specified as programs that must be dynamically evaluated.
 
-| Strategy   | Direct Deps | Deep Deps | Edges | Tags |
-| ---        | ---         | ---       | ---   | ---  |
-| initscript | ✅          | ✅        | ✅    |      |
+|           |                                                                           |
+| --------- | ------------------------------------------------------------------------- |
+| :warning: | This strategy requires dynamic analysis, which requires a CI integration. |
 
-## Requirements
+<!-- omit in toc -->
+## Table of contents
 
-Gradle project analysis requires at least one of:
+- [Concepts](#concepts)
+  - [Subprojects and configurations](#subprojects-and-configurations)
+  - [Gradle wrappers](#gradle-wrappers)
+- [Running Gradle](#running-gradle)
+- [Discovery](#discovery)
+- [Tactics](#tactics)
+  - [Tactic selection](#tactic-selection)
+  - [Gradle build plugin](#gradle-build-plugin)
+  - [Parsing `gradle :dependencies`](#parsing-gradle-dependencies)
+- [Debugging an integration](#debugging-an-integration)
+  - [Determining whether Gradle targets are detected](#determining-whether-gradle-targets-are-detected)
+  - [Manually checking Gradle dependency results](#manually-checking-gradle-dependency-results)
+  - [Debugging the "Gradle build plugin" tactic](#debugging-the-gradle-build-plugin-tactic)
+- [Manually specifying Gradle dependencies](#manually-specifying-gradle-dependencies)
 
-- A project that uses gradle wrappers (`gradlew`/`gradlew.bat`)
-- A locally-installed gradle
+## Concepts
 
-## Project Discovery
+### Subprojects and configurations
 
-Directories that contain a gradle buildscript, e.g., `build.gradle` or `build.gradle.kts`, are treated as gradle projects. Gradle buildscripts in a project's subdirectories are ignored.
+Most sizable Gradle builds organize their dependencies with two concepts: subprojects and configurations.
 
-## Analysis
+#### Subprojects
 
-1. Unpack our init script to a temporary directory
-2. Invoke the init script with `gradle jsonDeps -Ipath/to/init.gradle`
+_Subprojects_ are used when you have multiple "projects" in a single Gradle build (e.g. having multiple projects in a single repository managed by with single `settings.gradle` and one or more `build.gradle` files). Gradle calls these "multi-projects". Gradle multi-projects have one root project, and one or more subprojects.
 
-Currently, we only record the `default` configuration dependencies of each
-project. Subprojects are also included in the graph.
+A single subproject roughly corresponds to a single set of outputs. Hence, we treat subprojects as separate analysis targets.
+
+For details, see [Creating a multi-project build](https://docs.gradle.org/current/userguide/multi_project_builds.html#authoring-multi-project-builds).
+
+#### Configurations
+
+Within a single subproject, Gradle builds can declare dependencies for different "scopes" (i.e. different contexts, such as compilation, test execution, or runtime execution).
+
+Gradle calls these scopes _configurations_. Examples of configurations include `implementation`, `testRuntime`, or `compileClasspath`.
+
+Different subprojects can have different configurations. Configurations are specific to a subproject, although there are many common configurations (e.g. `compileClasspath`) that most subprojects have.
+
+For more details, see [What are dependency configurations](https://docs.gradle.org/current/userguide/declaring_dependencies.html#sec:what-are-dependency-configurations).
+
+#### Relationship to analysis targets
+
+Each pair of `(subproject, configuration)` corresponds to one dependency graph.
+
+Ideally, each `(subproject, configuration)` pair would be a separate analysis target. For technical reasons, the current implementation treats separate subprojects as analysis targets, and treats each subproject as the union of all of its configurations.
+
+In practice, this should not affect returned dependency results.
+
+### Gradle wrappers
+
+Instead of invoking `gradle` directly, most Gradle projects use a "Gradle wrapper", which is a shell script vendored in the project that selects and downloads the correct version of Gradle. See [The Gradle Wrapper](https://docs.gradle.org/current/userguide/gradle_wrapper.html) for details.
+
+Because of this, the Gradle analyzer has logic for selecting which `gradle` executable to actually use. See [Running Gradle](#running-gradle) for details.
+
+## Running Gradle
+
+This strategy requires dynamic analysis in its discovery phase (not just in the analysis phase). This is because we need to execute Gradle in order to list subprojects and evaluate `build.gradle` files.
+
+When executing Gradle for an analysis target at directory `ANALYSIS_TARGET_DIR`, the CLI will prefer (in order):
+
+1. `$ANALYSIS_TARGET_DIR/gradlew`
+2. `$ANALYSIS_TARGET_DIR/gradlew.bat`
+3. `gradle` (from `$PATH`)
+
+For more details, see [Gradle wrappers](#gradle-wrappers).
+
+In this documentation below, for brevity, we'll always refer to the selected tool as `gradle`.
+
+## Discovery
+
+This strategy discovers analysis targets by looking for files in the folder being analyzed whose names start with `build.gradle`. This matches both `build.gradle` as well as `build.gradle.kts` and Gradle build scripts in other Gradle-supported languages (`build.gradle.*`).
+
+It then executes `gradle projects` in the directory where the build file is found to get a list of subprojects for this Gradle build. These subprojects are used to create the analysis targets for this Gradle build.
+
+If there are no subprojects, an analysis target is created that analyzes the root project. Otherwise, a set of analysis targets is created: one for each Gradle subproject.
+
+## Tactics
+
+### Tactic selection
+
+This strategy selects tactics by trying them in preference order and uses the results of the first tactic to succeed.
+
+The order of tactics for this strategy is:
+
+1. Gradle build plugin
+2. Parsing `gradle :dependencies` (not yet implemented)
+
+### Gradle build plugin
+
+|                    |                                                           |
+| ------------------ | --------------------------------------------------------- |
+| :heavy_check_mark: | This tactic reports dependencies for all subprojects.     |
+| :heavy_check_mark: | This tactic provides a graph for subproject dependencies. |
+| :warning:          | This tactic requires dynamic analysis.                    |
+
+This tactic runs a Gradle [init script](https://docs.gradle.org/current/userguide/init_scripts.html) to output the dependencies in each Gradle subproject. Mechanically, this tactic:
+
+1. Unpacks our init script to a temporary directory.
+2. Invokes the init script with `gradle jsonDeps -Ipath/to/init.gradle`.
+3. Parses the JSON output of the init script.
+
+This init script is implemented [here](https://github.com/fossas/spectrometer/blob/master/scripts/jsondeps.gradle) and bundled into the CLI during compilation.
+
+The script works by iterating through configurations, resolving their dependencies, and then serializing those dependencies into JSON.
+
+### Parsing `gradle :dependencies`
+
+|           |                                        |
+| --------- | -------------------------------------- |
+| :x:       | This tactic is not yet implemented.    |
+| :warning: | This tactic requires dynamic analysis. |
+
+This not-yet-implemented tactic will execute `gradle $SUBPROJECT:dependencies` for each analysis target, and parse the tool's output.
+
+## Debugging an integration
+
+### Determining whether Gradle targets are detected
+
+To determine whether the CLI is properly detecting your Gradle project, run `fossa list-targets`. The output of this command is a list of analysis targets, in the format `type@path`.
+
+<!-- TODO: is there a guide for `fossa list-targets` I can reference here? -->
+
+For each of your Gradle subprojects, you should see a `gradle@PATH_TO_ROOT_PROJECT:SUBPROJECT` target in the list of analysis targets.
+
+If you _don't_ see this, one of two things is likely happening:
+
+1. Your Gradle project does not have a `build.gradle` file. This is an unsupported configuration.
+2. `gradle projects` is failing to execute. Make sure that a Gradle wrapper is accessible (see [Running Gradle](#running-gradle)), and make sure `gradle projects` runs successfully.
+
+### Manually checking Gradle dependency results
+
+To manually verify the correctness of the CLI's results, run `gradle :dependencies` in your root project, and `gradle $SUBPROJECT:dependencies` for each subproject.
+
+The CLI should produce a graph of dependencies that's a union of the dependencies of each subproject.
+
+If your CLI run uploads versions that differ from the output of `gradle $SUBPROJECT:dependencies`, check to make sure that the subproject dependency's version is the actual version resolved across the entire Gradle build. Different Gradle subprojects may select different dependency versions when resolved independently, but will select a single resolved version when the build is resolved as a whole.
+
+If you'd like to make a bug report about incorrect dependencies, make sure to include the list of incorrect dependencies, as well as the commands you ran to obtain that list.
+
+### Debugging the "Gradle build plugin" tactic
+
+The Gradle build plugin is a Gradle [init script](https://docs.gradle.org/current/userguide/init_scripts.html) implemented [here](../../scripts/jsondeps.gradle).
+
+If this tactic doesn't appear to be working (e.g. is giving you incorrect dependencies or is missing dependencies), you can run the init script directly using:
+
+```
+gradle -I$PATH_TO_SCRIPT jsonDeps
+```
+
+For example, with the script extracted to `/tmp/jsondeps.gradle`, you should run (from within the Gradle build script's working directory):
+
+```
+gradle -I/tmp/jsondeps.gradle jsonDeps
+```
+
+Providing this output with a bug report will help us debug issues with the analysis.
+
+## Manually specifying Gradle dependencies
+
+If the CLI doesn't natively integrate with your build tool (e.g. if you have a homegrown tool), and your build tool uses Gradle dependencies, you can still manually add Gradle dependencies to an uploaded build. This feature is generally known as [manual dependencies](../userguide.md#manually-specifying-dependencies).
+
+Gradle in particular actually uploads _Maven_ dependencies, since most Gradle builds use Gradle's Maven interoperability to get dependencies from Maven repositories.
+
+An example configuration file looks like:
+
+```yaml
+# fossa-deps.yml
+referenced-dependencies:
+- type: maven
+  name: javax.xml.bind:jaxb-api
+  version: 1.0.0
+```
+
+Notice that the `name` field follows Maven conventions: `groupId:artifactId`.
+
+For more details, see the [manual dependencies](../userguide.md#manually-specifying-dependencies) documentation.
diff --git a/docs/userguide.md b/docs/userguide.md
@@ -1,23 +1,49 @@
-
 # Spectrometer User Guide
 
-1. [Quick Start](#quick-start)
-2. [Supported Languages](#supported-languages)
-3. Commands
-    - [`fossa analyze`](#fossa-analyze)
-        - [Specifying FOSSA project details](#specifying-fossa-project-details)
-        - [Printing results without uploading to FOSSA](#printing-results-without-uploading-to-fossa)
-        - [Running in a specific directory](#running-in-a-specific-directory)
-        - [Scanning archive contents](#scanning-archive-contents)
-        - [Manually specifying dependencies](#manually-specifying-dependencies)
-    - [`fossa test`](#fossa-test)
-        - [Specifying a timeout](#specifying-a-timeout)
-        - [Print issues as json](#print-issues-as-json)
-4. [Common FOSSA Project Flags](#common-fossa-project-flags)
-5. [Frequently-Asked Questions](#frequently-asked-questions)
-    - [`fossa analyze`: Why wasn't my project found?](#fossa-analyze-why-wasnt-my-project-found)
-    - [When are you adding support for (some buildtool/language)](#when-are-you-adding-support-for-some-buildtoollanguage)
-    - [What are these experimental monorepo flags?](#what-are-these-experimental-monorepo-flags)
+<!-- omit in toc -->
+## Table of contents
+
+- [Quick Start](#quick-start)
+  - [Configure your API key](#configure-your-api-key)
+  - [Run Analysis](#run-analysis)
+  - [Check for FOSSA scan results](#check-for-fossa-scan-results)
+- [Supported Languages](#supported-languages)
+  - [clojure](#clojure)
+  - [erlang](#erlang)
+  - [golang](#golang)
+  - [haskell](#haskell)
+  - [java](#java)
+  - [javascript/typescript](#javascripttypescript)
+  - [.NET](#net)
+  - [objective-c](#objective-c)
+  - [python](#python)
+  - [ruby](#ruby)
+  - [rust](#rust)
+  - [scala](#scala)
+  - [swift](#swift)
+- [`fossa analyze`](#fossa-analyze)
+  - [Specifying FOSSA project details](#specifying-fossa-project-details)
+  - [Printing results without uploading to FOSSA](#printing-results-without-uploading-to-fossa)
+  - [Printing project metadata](#printing-project-metadata)
+  - [Running in a specific directory](#running-in-a-specific-directory)
+  - [Scanning archive contents](#scanning-archive-contents)
+- [Manually specifying dependencies](#manually-specifying-dependencies)
+  - [Custom dependencies](#custom-dependencies)
+  - [Remote dependencies](#remote-dependencies)
+  - [Errors in the `fossa-deps` file](#errors-in-the-fossa-deps-file)
+  - [License scanning local dependencies](#license-scanning-local-dependencies)
+- [`fossa test`](#fossa-test)
+  - [Specifying a timeout](#specifying-a-timeout)
+  - [Print issues as JSON](#print-issues-as-json)
+- [`fossa report`](#fossa-report)
+  - [Report types](#report-types)
+  - [Specifying a report timeout](#specifying-a-report-timeout)
+  - [Print report as JSON](#print-report-as-json)
+- [Common FOSSA Project Flags](#common-fossa-project-flags)
+- [Frequently-Asked Questions](#frequently-asked-questions)
+  - [`fossa analyze`: Why wasn't my project found?](#fossa-analyze-why-wasnt-my-project-found)
+  - [When are you adding support for (some buildtool/language)?](#when-are-you-adding-support-for-some-buildtoollanguage)
+  - [What are these experimental monorepo flags?](#what-are-these-experimental-monorepo-flags)
 
 ## Quick Start
 
@@ -236,7 +262,7 @@ custom-dependencies:
 - name: foo
   version: 1.2.3
   license: "MIT or Apache-2.0"
-# You can also provide a description and/or homepage. These values populate metadata fields in reports in the FOSSA web UI. 
+# You can also provide a description and/or homepage. These values populate metadata fields in reports in the FOSSA web UI.
 - name: foo-wrapper
   version: 1.2.3
   license: MIT