Adds swift package manager manifest support (one of many)

Changelog.md

@@ -1,5 +1,8 @@
 # Spectrometer Changelog
 
+## v2.16.0
+- Swift: Supports dependencies analysis for dependencies managed by Swift Package Manager. ([#354](https://github.com/fossas/spectrometer/pull/354))
+
 ## v2.15.24
 - Leiningen: Executes `lein --version` before performing any analysis, to ensure Leiningen has performed its install tasks (done on its first invocation). ([#379](https://github.com/fossas/spectrometer/pull/379))
 

docs/quickreference/swiftpm.md

@@ -0,0 +1,12 @@
+# Quick reference: Swift Package Manager
+
+## Requirements
+
+**Minimum**
+
+- `Package.swift` file present in your project
+
+## Project discovery
+
+Directories containing `Package.swift` files are considered
+projects managed by swift package manager.

docs/strategies.md

@@ -76,7 +76,7 @@ The CLI supports the following strategies:
 - [golang](strategies/golang.md) (gomodules, dep, glide)
 - [gradle](strategies/gradle.md)
 - [haskell](strategies/haskell.md) (cabal, stack)
-- [iOS](strategies/ios.md) (Carthage, Cocoapods)
+- [iOS](strategies/ios.md) (Carthage, Cocoapods, Swift package manager)
 - [maven](strategies/maven.md)
 - [nodejs](strategies/nodejs.md) (yarn, npmcli)
 - [python](strategies/python.md) (conda, pipenv, setuptools)

docs/strategies/ios/swift.md

@@ -0,0 +1,168 @@
+# Swift Package Manager
+
+## Project Discovery
+
+Find all files named: `Package.swift` or find Xcode's project file named: `project.pbxproj`. 
+We will not scan `.build` directory if the `Package.swift` or Xcode project file is discovered. 
+
+# Swift Analysis
+
+| Strategy                                                                 | Direct Deps        | Deep Deps          | Edges | Classifies Test Dependencies |
+| ------------------------------------------------------------------------ | ------------------ | ------------------ | ----- | ---------------------------- |
+| Parse dependencies from `Package.swift`                                  | :white_check_mark: | :x:                | :x:   | :x:                          |
+| Parse dependencies from `Package.swift` and `Package.resolved`           | :white_check_mark: | :white_check_mark: | :x:   | :x:                          |
+| Parse dependencies from Xcode's `project.pbxproj`                        | :white_check_mark: | :x:                | :x:   | :x:                          |
+| Parse dependencies from Xcode's `project.pbxproj` and `Package.resolved` | :white_check_mark: | :white_check_mark: | :x:   | :x:                          |
+
+- Manifest file: `Package.swift`, must begin with `// swift-tools-version:` string, followed by version number specifier. 
+- We follow swift package manager's convention and presume properties of the package are defined in a single nested initializer statement and are not modified after initialization.
+- Valid Xcode project for swift, is defined by the discovery of `project.pbxproj` file in ASCII plist format with at least one `XCRemoteSwiftPackageReference` object in its content.
+
+## Limitations
+
+- Path dependencies are ignored in the analysis (e.g. `package(path: "./../local-pkg")`)
+- If the Xcode project dependencies are sourced via a local path, they will be ignored in the analysis.
+- Only Xcode project files in ASCII plist format with UTF-8 encoding are supported.
+
+## Example
+
+Create Package.swift file in the directory. Add dependencies, targets, products, and source code. Example Package.swift file is shown below. By convention, the properties of a Package are defined in a single nested initializer statement, and not modified after initialization.
+
+```swift
+// swift-tools-version:5.4.0
+import PackageDescription
+
+let package = Package(
+    name: "Example",
+    defaultLocalization: "en",
+    products: [],
+    dependencies: [
+        .package(name: "grpc-swift", url: "https://github.com/grpc/grpc-swift.git", from: "1.0.0"),
+    ]
+)
+```
+
+We can update and resolve dependencies by performing `swift package update`. Executing this will create Package.resolved in the directory. An example file is shown below:
+
+```json
+{
+  "object": {
+    "pins": [
+      {
+        "package": "grpc-swift",
+        "repositoryURL": "https://github.com/grpc/grpc-swift.git",
+        "state": {
+          "branch": null,
+          "revision": "14e1ea3350892a864386517c037e11fb68baf818",
+          "version": "1.3.0"
+        }
+      },
+      {
+        "package": "swift-log",
+        "repositoryURL": "https://github.com/apple/swift-log.git",
+        "state": {
+          "branch": null,
+          "revision": "5d66f7ba25daf4f94100e7022febf3c75e37a6c7",
+          "version": "1.4.2"
+        }
+      }
+    ]
+  },
+  "version": 1
+}
+
+```
+Note: Only a few pins are shown above for brevity.
+
+### `Package.swift` and `Package.resolved`
+
+When the analysis is performed (e.g. `fossa analyze -o`), we will identify the following as direct dependencies:
+
+- https://github.com/grpc/grpc-swift.git@1.3.0
+
+If `Package.resolved` is discovered, the following deep dependencies will be identified, however, we will not identify the edges in the dependency graph:
+
+- https://github.com/apple/swift-log.git@1.4.2
+
+If `Package.resolved` is not discovered, only direct dependencies will be reported. 
+
+### Xcode Project and `Package.resolved`
+
+For Xcode project using swift package manager to manage swift package dependencies, Xcode project file named `project.pbxproj` will be analyzed. In the Xcode project file, `XCRemoteSwiftPackageReference` objects will be used to identify swift packages that are direct dependencies. For the analysis, at least one such reference must exist in the file. If no such references are found, we will not consider the Xcode project in the swift analysis.
+
+Excerpt from example `project.pbxproj`:
+
+```
+// !$*UTF8*$!
+{
+  archiveVersion = 1;
+  classes = {
+  };
+  objectVersion = 52;
+  objects = {
+
+    ...
+
+    170A463726ECEDEF002DDFB8 /* XCRemoteSwiftPackageReference "example-package-deckofplayingcards" */ = {
+      isa = XCRemoteSwiftPackageReference;
+      repositoryURL = "https://github.com/apple/example-package-deckofplayingcards";
+      requirement = {
+        branch = main;
+        kind = branch;
+      };
+    };
+
+    ...
+  };
+  rootObject = 17874CD926C46B8500D16CA8 /* Project object */;
+} 
+```
+
+If the `Package.resolved` is discovered, deep dependencies will be identified. If not, only direct dependencies listed in xcode project file will be identified. In either case, no edges among dependencies will be reported.
+
+## F.A.Q
+
+### How do I *only perform analysis* for swift package dependencies?
+
+You can explicitly specify the analysis target in `.fossa.yml` file.
+
+The example below will exclude all analysis targets except swift. 
+
+```yaml
+# .fossa.yml 
+
+version: 3
+targets:
+  only:
+    - type: swift
+```
+
+### Swift packages sourced from local directories are not discovered in the analysis. Is there a workaround?
+
+This is a current limitation. For swift package manager analysis, we only support non-path dependencies at the moment. 
+To include local dependencies, you can use `fossa-deps.yml` file to upload the local package for license scanning and analysis.
+
+```yaml
+# in fossa-deps.yml
+
+vendored-dependencies:
+- name: MyLocalPackage
+  path: /Jenkins/App/Resources/MyLocalPackage # path can be either a file or a folder.
+  version: 3.4.16 # revision will be set to the MD5 hash of the file path if left unspecified.
+```
+
+Note: License scanning currently operates by uploading the files at the specified path to a secure S3 bucket. All files that do not contain licenses are then removed after 2 weeks.
+Refer to [User guide](../../userguide.md) for more details. 
+
+### When performing `fossa list-targets`, Xcode project using swift packages are not getting discovered.
+
+For swift, we consider the Xcode project to be a valid Xcode project, if and only if it meets the following requirements:
+- Xcode project file named: `project.pbxproj` exists in the directory.
+- Xcode project file must be in ASCII plist format with UTF-8 encoding.
+- Xcode project file has at least one object, with isa of `XCRemoteSwiftPackageReference`. 
+
+## References
+
+- [Swift Package Manager](https://github.com/apple/swift-package-manager)
+- [Package.swift, must begin with version specifier](https://github.com/apple/swift-package-manager/blob/main/Documentation/PackageDescription.md#about-the-swift-tools-version)
+- [Package.swift, must be defined in a single nested statement, and should not be modified after initialization](https://github.com/apple/swift-package-manager/blob/main/Documentation/PackageDescription.md#package)

docs/userguide.md

@@ -153,6 +153,7 @@ fossa analyze --help
 
 - [carthage](quickreference/carthage.md)
 - [cocoapods](quickreference/cocoapods.md)
+- [swift package manager](quickreference/swiftpm.md)
 
 ## `fossa analyze`
 
@@ -259,6 +260,7 @@ Supported dependency types:
 - `pub` - Dart dependencies found at [pub.dev](https://www.pub.dev/).
 - `pypi` - Python dependencies that are typically found at [Pypi.org](https://pypi.org/).
 - `cocoapods` - Swift and Objective-C dependencies found at [Cocoapods.org](https://cocoapods.org/).
+- `swift` - Swift package manager dependencies. Specified as the full git repository `https://github.com/fossas/spectrometer`.
 - `url` - The URL type allows you to specify only the download location of an archive (e.g.: `.zip`, .`tar.gz`, etc.) in the `name` field and the FOSSA backend will attempt to download and scan it. Example for a github source dependency `https://github.com/fossas/spectrometer/archive/refs/tags/v2.7.2.tar.gz`. The `version` field will be silently ignored for `url` type dependencies.
 
 ### Custom dependencies

spectrometer.cabal

@@ -99,6 +99,7 @@ common deps
     , prettyprinter                >=1.6      && <1.8
     , prettyprinter-ansi-terminal  ^>=1.1.1
     , random                       ^>=1.2.0
+    , raw-strings-qq               ^>=1.1
     , req                          ^>=3.9.1
     , retry                        ^>=0.9.0.0
     , semver                       ^>=0.4.0.1
@@ -295,6 +296,11 @@ library
     Strategy.Ruby.BundleShow
     Strategy.Ruby.GemfileLock
     Strategy.Scala
+    Strategy.SwiftPM
+    Strategy.Swift.PackageResolved
+    Strategy.Swift.PackageSwift
+    Strategy.Swift.Xcode.Pbxproj
+    Strategy.Swift.Xcode.PbxprojParser
     Strategy.Yarn
     Strategy.Yarn.V1.YarnLock
     Strategy.Yarn.V2.Lockfile
@@ -391,6 +397,10 @@ test-suite unit-tests
     RPM.SpecFileSpec
     Ruby.BundleShowSpec
     Ruby.GemfileLockSpec
+    Swift.PackageResolvedSpec
+    Swift.PackageSwiftSpec
+    Swift.Xcode.PbxprojSpec
+    Swift.Xcode.PbxprojParserSpec
     Yarn.V2.LockfileSpec
     Yarn.V2.ResolversSpec
     Yarn.YarnLockV1Spec

src/App/Fossa/Analyze.hs

@@ -98,6 +98,7 @@ import Strategy.Python.Setuptools qualified as Setuptools
 import Strategy.RPM qualified as RPM
 import Strategy.Rebar3 qualified as Rebar3
 import Strategy.Scala qualified as Scala
+import Strategy.SwiftPM qualified as SwiftPM
 import Strategy.Yarn qualified as Yarn
 import System.Exit (die)
 import Types (DiscoveredProject (..), FoundTargets)
@@ -222,6 +223,7 @@ discoverFuncs =
   , Scala.discover
   , Setuptools.discover
   , Stack.discover
+  , SwiftPM.discover
   , Yarn.discover
   ]
 

src/App/Fossa/Analyze/Project.hs

@@ -22,7 +22,7 @@ mkResult basedir project dependencyResults =
         -- their dependencies would be filtered out. The real fix to this is to
         -- have a separate designation for "reachable" vs "direct" on nodes in a
         -- Graphing, where direct deps are inherently reachable.
-        if null (Graphing.directList graph)
+        if null (Graphing.directList graph) || shouldKeepUnreachableDeps (projectType project)
           then graph
           else Graphing.pruneUnreachable graph
     , projectResultGraphBreadth = dependencyGraphBreadth dependencyResults
@@ -39,3 +39,7 @@ data ProjectResult = ProjectResult
   , projectResultGraphBreadth :: GraphBreadth
   , projectResultManifestFiles :: [SomeBase File]
   }
+
+shouldKeepUnreachableDeps :: Text -> Bool
+shouldKeepUnreachableDeps "swift" = True
+shouldKeepUnreachableDeps _ = False

src/App/Fossa/ManualDeps.hs

@@ -253,6 +253,7 @@ depTypeFromText text = case text of
   "pypi" -> Just PipType
   "cocoapods" -> Just PodType
   "url" -> Just URLType
+  "swift" -> Just SwiftType
   _ -> Nothing -- unsupported dep, need to respond with an error and skip this dependency
   -- rpm is an unsupported type. This is because we currently have 2 RPM fetchers
   -- and we should wait for a need to determine which one to use for manually

src/DepTypes.hs

@@ -95,6 +95,8 @@ data DepType
     HackageType
   | -- | A Carthage dependency -- effectively a "git" dependency. Name is repo path and version is tag/branch/hash
     CarthageType
+  | -- | A Swift Package Dependency -- effectively a "git" dependency. Name is repo path and version is tag/branch/hash
+    SwiftType
   deriving (Eq, Ord, Show, Generic)
 
 data VerConstraint

src/Srclib/Converter.hs

@@ -121,6 +121,7 @@ depTypeToFetcher = \case
   URLType -> "url"
   UserType -> "user"
   PubType -> "pub"
+  SwiftType -> "swift"
 
 -- | GooglesourceType and SubprojectType are not supported with this function, since they're ambiguous.
 fetcherToDepType :: Text -> Maybe DepType

src/Strategy/Swift/PackageResolved.hs

@@ -0,0 +1,78 @@
+module Strategy.Swift.PackageResolved (
+  SwiftPackageResolvedFile (..),
+  SwiftResolvedPackage (..),
+  resolvedDependenciesOf,
+) where
+
+import Data.Aeson (
+  FromJSON (parseJSON),
+  Object,
+  withObject,
+  (.:),
+  (.:?),
+ )
+import Data.Aeson.Types (Parser)
+import Data.Foldable (asum)
+import Data.Text (Text)
+import DepTypes (DepType (GitType), Dependency (..), VerConstraint (CEq))
+
+data SwiftPackageResolvedFile = SwiftPackageResolvedFile
+  { version :: Integer
+  , pinnedPackages :: [SwiftResolvedPackage]
+  }
+  deriving (Show, Eq, Ord)
+
+data SwiftResolvedPackage = SwiftResolvedPackage
+  { package :: Text
+  , repositoryURL :: Text
+  , repositoryBranch :: Maybe Text
+  , repositoryRevision :: Maybe Text
+  , repositoryVersion :: Maybe Text
+  }
+  deriving (Show, Eq, Ord)
+
+instance FromJSON SwiftPackageResolvedFile where
+  parseJSON = withObject "Package.resolved content" $ \obj ->
+    SwiftPackageResolvedFile <$> obj .: "version"
+      <*> (obj .: "object" |> "pins")
+
+(|>) :: FromJSON a => Parser Object -> Text -> Parser a
+(|>) parser key = do
+  obj <- parser
+  obj .: key
+
+(|?>) :: FromJSON a => Parser (Maybe Object) -> Text -> Parser (Maybe a)
+(|?>) parser key = do
+  obj <- parser
+  case obj of
+    Nothing -> pure Nothing
+    Just o -> o .:? key
+
+instance FromJSON SwiftResolvedPackage where
+  parseJSON = withObject "Package.resolved pinned object" $ \obj ->
+    SwiftResolvedPackage <$> obj .: "package"
+      <*> obj .: "repositoryURL"
+      <*> (obj .:? "state" |?> "branch")
+      <*> (obj .:? "state" |?> "revision")
+      <*> (obj .:? "state" |?> "version")
+
+-- Note, Package.resolved does not include path dependencies.
+resolvedDependenciesOf :: SwiftPackageResolvedFile -> [Dependency]
+resolvedDependenciesOf resolvedContent = map toDependency $ pinnedPackages resolvedContent
+  where
+    toDependency :: SwiftResolvedPackage -> Dependency
+    toDependency pkg =
+      Dependency
+        { dependencyType = GitType
+        , dependencyName = repositoryURL pkg
+        , dependencyVersion =
+            CEq
+              <$> asum
+                [ repositoryRevision pkg
+                , repositoryVersion pkg
+                , repositoryBranch pkg
+                ]
+        , dependencyLocations = []
+        , dependencyEnvironments = []
+        , dependencyTags = mempty
+        }