Skip to content
This repository has been archived by the owner on Apr 1, 2022. It is now read-only.

Commit

Permalink
Swift PM: Analyzes Package.resolved file (second of many) (#356)
Browse files Browse the repository at this point in the history
  • Loading branch information
meghfossa authored Sep 24, 2021
1 parent 6ad6418 commit 13a09fb
Show file tree
Hide file tree
Showing 14 changed files with 1,279 additions and 81 deletions.
155 changes: 125 additions & 30 deletions docs/strategies/ios/swift.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,60 +2,131 @@

## Project Discovery

Find all swift manifest files, named: `Package.swift`
Find all files named: `Package.swift` or find Xcode's project file named: `project.pbxproj`.
We will not scan `.build` directory if the `Package.swift` or Xcode project file is discovered.

# Swift Package Analysis
# Swift Analysis

| Strategy | Direct Deps | Deep Deps | Edges | Classifies Test Dependencies |
| --------------------------------------------- | ------------------ | --------- | ----- | ---------------------------- |
| parse package dependencies in `Package.swift` | :white_check_mark: | :x: | :x: | :x: |
| Strategy | Direct Deps | Deep Deps | Edges | Classifies Test Dependencies |
| ------------------------------------------------------------------------ | ------------------ | ------------------ | ----- | ---------------------------- |
| Parse dependencies from `Package.swift` | :white_check_mark: | :x: | :x: | :x: |
| Parse dependencies from `Package.swift` and `Package.resolved` | :white_check_mark: | :white_check_mark: | :x: | :x: |
| Parse dependencies from Xcode's `project.pbxproj` | :white_check_mark: | :x: | :x: | :x: |
| Parse dependencies from Xcode's `project.pbxproj` and `Package.resolved` | :white_check_mark: | :white_check_mark: | :x: | :x: |

- Manifest file - `Package.swift`, must begin with `// swift-tools-version:` string, followed by version number specifier.
- We follow swift package manager's convention, and presume properties of package are defined in a single nested initializer statement, and are not modified after initialization.
- Manifest file: `Package.swift`, must begin with `// swift-tools-version:` string, followed by version number specifier.
- We follow swift package manager's convention and presume properties of the package are defined in a single nested initializer statement and are not modified after initialization.
- Valid Xcode project for swift, is defined by the discovery of `project.pbxproj` file in ASCII plist format with at least one `XCRemoteSwiftPackageReference` object in its content.

## Limitation
## Limitations

- Path dependencies are ignored in the analysis (e.g. `package(path: "./../local-pkg")`)
- If the Xcode project dependencies are sourced via a local path, they will be ignored in the analysis.
- Only Xcode project files in ASCII plist format with UTF-8 encoding are supported.

## Example
## Example

Create Package.swift file in the directory. Add dependencies, targets, products, and source code. Example Package.swift file is shown below. By convention, the properties of a Package are defined in a single nested initializer statement, and not modified after initialization.

```swift
// swift-tools-version:4.0
// swift-tools-version:5.4.0
import PackageDescription

let package = Package(
name: "DeckOfPlayingCards",
products: [
.library(name: "DeckOfPlayingCards", targets: ["DeckOfPlayingCards"]),
],
name: "Example",
defaultLocalization: "en",
products: [],
dependencies: [
.package(url: "https://github.com/apple/example-package-fisheryates.git", from: "2.0.0"),
.package(url: "https://github.com/apple/example-package-playingcard.git", from: "3.0.0"),
],
targets: [
.target(
name: "DeckOfPlayingCards",
dependencies: ["FisherYates", "PlayingCard"]),
.testTarget(
name: "DeckOfPlayingCardsTests",
dependencies: ["DeckOfPlayingCards"]),
.package(name: "grpc-swift", url: "https://github.com/grpc/grpc-swift.git", from: "1.0.0"),
]
)
```

When analysis is performed (e.g. `fossa analyze -o`), we will identify following as direct dependencies:
- https://github.com/apple/example-package-fisheryates.git
- https://github.com/apple/example-package-playingcard.git
We can update and resolve dependencies by performing `swift package update`. Executing this will create Package.resolved in the directory. An example file is shown below:

```json
{
"object": {
"pins": [
{
"package": "grpc-swift",
"repositoryURL": "https://github.com/grpc/grpc-swift.git",
"state": {
"branch": null,
"revision": "14e1ea3350892a864386517c037e11fb68baf818",
"version": "1.3.0"
}
},
{
"package": "swift-log",
"repositoryURL": "https://github.com/apple/swift-log.git",
"state": {
"branch": null,
"revision": "5d66f7ba25daf4f94100e7022febf3c75e37a6c7",
"version": "1.4.2"
}
}
]
},
"version": 1
}

```
Note: Only a few pins are shown above for brevity.

### `Package.swift` and `Package.resolved`

When the analysis is performed (e.g. `fossa analyze -o`), we will identify the following as direct dependencies:

- https://github.com/grpc/grpc-swift.git@1.3.0

If `Package.resolved` is discovered, the following deep dependencies will be identified, however, we will not identify the edges in the dependency graph:

- https://github.com/apple/swift-log.git@1.4.2

If `Package.resolved` is not discovered, only direct dependencies will be reported.

### Xcode Project and `Package.resolved`

For Xcode project using swift package manager to manage swift package dependencies, Xcode project file named `project.pbxproj` will be analyzed. In the Xcode project file, `XCRemoteSwiftPackageReference` objects will be used to identify swift packages that are direct dependencies. For the analysis, at least one such reference must exist in the file. If no such references are found, we will not consider the Xcode project in the swift analysis.

Excerpt from example `project.pbxproj`:

```
// !$*UTF8*$!
{
archiveVersion = 1;
classes = {
};
objectVersion = 52;
objects = {
...
170A463726ECEDEF002DDFB8 /* XCRemoteSwiftPackageReference "example-package-deckofplayingcards" */ = {
isa = XCRemoteSwiftPackageReference;
repositoryURL = "https://github.com/apple/example-package-deckofplayingcards";
requirement = {
branch = main;
kind = branch;
};
};
...
};
rootObject = 17874CD926C46B8500D16CA8 /* Project object */;
}
```

If the `Package.resolved` is discovered, deep dependencies will be identified. If not, only direct dependencies listed in xcode project file will be identified. In either case, no edges among dependencies will be reported.

## F.A.Q

### How do I *only perform analysis* for swift package dependencies?

You can explicitly specify analysis an target in `.fossa.yml` file.
You can explicitly specify the analysis target in `.fossa.yml` file.

Example below, will exclude all analysis targets except swift.
The example below will exclude all analysis targets except swift.

```yaml
# .fossa.yml
Expand All @@ -66,8 +137,32 @@ targets:
- type: swift
```
### Swift packages sourced from local directories are not discovered in the analysis. Is there a workaround?
This is a current limitation. For swift package manager analysis, we only support non-path dependencies at the moment.
To include local dependencies, you can use `fossa-deps.yml` file to upload the local package for license scanning and analysis.

```yaml
# in fossa-deps.yml
vendored-dependencies:
- name: MyLocalPackage
path: /Jenkins/App/Resources/MyLocalPackage # path can be either a file or a folder.
version: 3.4.16 # revision will be set to the MD5 hash of the file path if left unspecified.
```

Note: License scanning currently operates by uploading the files at the specified path to a secure S3 bucket. All files that do not contain licenses are then removed after 2 weeks.
Refer to [User guide](../../userguide.md) for more details.

### When performing `fossa list-targets`, Xcode project using swift packages are not getting discovered.

For swift, we consider the Xcode project to be a valid Xcode project, if and only if it meets the following requirements:
- Xcode project file named: `project.pbxproj` exists in the directory.
- Xcode project file must be in ASCII plist format with UTF-8 encoding.
- Xcode project file has at least one object, with isa of `XCRemoteSwiftPackageReference`.

## References

- [Swift Package Manager](https://github.com/apple/swift-package-manager)
- [Package.swift, must begin with version specifier](https://github.com/apple/swift-package-manager/blob/main/Documentation/PackageDescription.md#about-the-swift-tools-version)
- [Package.swift, must be defined in single nested statement, and should not be modified after initialization](https://github.com/apple/swift-package-manager/blob/main/Documentation/PackageDescription.md#package)
- [Package.swift, must be defined in a single nested statement, and should not be modified after initialization](https://github.com/apple/swift-package-manager/blob/main/Documentation/PackageDescription.md#package)
7 changes: 7 additions & 0 deletions spectrometer.cabal
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ common deps
, prettyprinter >=1.6 && <1.8
, prettyprinter-ansi-terminal ^>=1.1.1
, random ^>=1.2.0
, raw-strings-qq ^>=1.1
, req ^>=3.9.1
, retry ^>=0.9.0.0
, semver ^>=0.4.0.1
Expand Down Expand Up @@ -293,7 +294,10 @@ library
Strategy.Ruby.GemfileLock
Strategy.Scala
Strategy.SwiftPM
Strategy.Swift.PackageResolved
Strategy.Swift.PackageSwift
Strategy.Swift.Xcode.Pbxproj
Strategy.Swift.Xcode.PbxprojParser
Strategy.Yarn
Strategy.Yarn.V1.YarnLock
Strategy.Yarn.V2.Lockfile
Expand Down Expand Up @@ -387,7 +391,10 @@ test-suite unit-tests
RPM.SpecFileSpec
Ruby.BundleShowSpec
Ruby.GemfileLockSpec
Swift.PackageResolvedSpec
Swift.PackageSwiftSpec
Swift.Xcode.PbxprojSpec
Swift.Xcode.PbxprojParserSpec
Yarn.V2.LockfileSpec
Yarn.V2.ResolversSpec
Yarn.YarnLockV1Spec
Expand Down
6 changes: 5 additions & 1 deletion src/App/Fossa/Analyze/Project.hs
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ mkResult basedir project dependencyResults =
-- their dependencies would be filtered out. The real fix to this is to
-- have a separate designation for "reachable" vs "direct" on nodes in a
-- Graphing, where direct deps are inherently reachable.
if null (Graphing.directList graph)
if null (Graphing.directList graph) || shouldKeepUnreachableDeps (projectType project)
then graph
else Graphing.pruneUnreachable graph
, projectResultGraphBreadth = dependencyGraphBreadth dependencyResults
Expand All @@ -39,3 +39,7 @@ data ProjectResult = ProjectResult
, projectResultGraphBreadth :: GraphBreadth
, projectResultManifestFiles :: [SomeBase File]
}

shouldKeepUnreachableDeps :: Text -> Bool
shouldKeepUnreachableDeps "swift" = True
shouldKeepUnreachableDeps _ = False
78 changes: 78 additions & 0 deletions src/Strategy/Swift/PackageResolved.hs
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
module Strategy.Swift.PackageResolved (
SwiftPackageResolvedFile (..),
SwiftResolvedPackage (..),
resolvedDependenciesOf,
) where

import Data.Aeson (
FromJSON (parseJSON),
Object,
withObject,
(.:),
(.:?),
)
import Data.Aeson.Types (Parser)
import Data.Foldable (asum)
import Data.Text (Text)
import DepTypes (DepType (GitType), Dependency (..), VerConstraint (CEq))

data SwiftPackageResolvedFile = SwiftPackageResolvedFile
{ version :: Integer
, pinnedPackages :: [SwiftResolvedPackage]
}
deriving (Show, Eq, Ord)

data SwiftResolvedPackage = SwiftResolvedPackage
{ package :: Text
, repositoryURL :: Text
, repositoryBranch :: Maybe Text
, repositoryRevision :: Maybe Text
, repositoryVersion :: Maybe Text
}
deriving (Show, Eq, Ord)

instance FromJSON SwiftPackageResolvedFile where
parseJSON = withObject "Package.resolved content" $ \obj ->
SwiftPackageResolvedFile <$> obj .: "version"
<*> (obj .: "object" |> "pins")

(|>) :: FromJSON a => Parser Object -> Text -> Parser a
(|>) parser key = do
obj <- parser
obj .: key

(|?>) :: FromJSON a => Parser (Maybe Object) -> Text -> Parser (Maybe a)
(|?>) parser key = do
obj <- parser
case obj of
Nothing -> pure Nothing
Just o -> o .:? key

instance FromJSON SwiftResolvedPackage where
parseJSON = withObject "Package.resolved pinned object" $ \obj ->
SwiftResolvedPackage <$> obj .: "package"
<*> obj .: "repositoryURL"
<*> (obj .:? "state" |?> "branch")
<*> (obj .:? "state" |?> "revision")
<*> (obj .:? "state" |?> "version")

-- Note, Package.resolved does not include path dependencies.
resolvedDependenciesOf :: SwiftPackageResolvedFile -> [Dependency]
resolvedDependenciesOf resolvedContent = map toDependency $ pinnedPackages resolvedContent
where
toDependency :: SwiftResolvedPackage -> Dependency
toDependency pkg =
Dependency
{ dependencyType = GitType
, dependencyName = repositoryURL pkg
, dependencyVersion =
CEq
<$> asum
[ repositoryRevision pkg
, repositoryVersion pkg
, repositoryBranch pkg
]
, dependencyLocations = []
, dependencyEnvironments = []
, dependencyTags = mempty
}
Loading

0 comments on commit 13a09fb

Please sign in to comment.