Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Swift: make extractor compilable with Swift 6 #17699

Merged
merged 58 commits into from
Dec 13, 2024
Merged
Show file tree
Hide file tree
Changes from 56 commits
Commits
Show all changes
58 commits
Select commit Hold shift + click to select a range
524686c
Swift: make extractor compilable with Swift 6
Oct 8, 2024
d3fb254
Swift: try out new builds
Oct 30, 2024
91f683d
Merge branch 'main' into redsun82/swift-6
Oct 30, 2024
d325b8e
Swift: update `load.bzl` and resources
Oct 30, 2024
df8184e
Swift: fix `check-no-override` job
Oct 30, 2024
5e6228c
Swift: fix
Oct 30, 2024
0141702
Swift: use `ubuntu-latest`
Oct 30, 2024
3877eb8
Swift: tentatively fix type mangling
Oct 30, 2024
66e43c4
Swift: use ubuntu 22.04
Oct 30, 2024
7f3745c
Swift: fix `@isolated(any)` DB inconsistencies
redsun82 Oct 31, 2024
34b8b43
Swift: use `-typecheck` in QL tests
Oct 31, 2024
3aa7123
Swift: restrict `UnresolvedAstNodes` to known locations
redsun82 Oct 31, 2024
ce4273d
Revert "Swift: use `-typecheck` in QL tests"
redsun82 Oct 31, 2024
5a045be
Swift: turn off SIL verifications
redsun82 Oct 31, 2024
1f35607
Swift: temporarily accept test changes
Oct 31, 2024
6f0f739
Swift: Update dropFirst / dropLast / reversed models for Swift 6.
geoffw0 Nov 1, 2024
be12649
Swift: Update joined models for Swift 6.
geoffw0 Nov 1, 2024
954fbc4
Swift: Update prefix / suffix models for Swift 6.
geoffw0 Nov 1, 2024
f3ea75d
Swift: Further modelling updates / gap filling that doesn't seem to a…
geoffw0 Nov 1, 2024
24c4e87
Swift: Fix stray [].
geoffw0 Nov 1, 2024
87a08fe
Merge pull request #17891 from geoffw0/swift6models
redsun82 Nov 5, 2024
c875667
Swift: update to 6.0.2
Nov 6, 2024
afd4585
Merge branch 'main' into redsun82/swift-6
Nov 6, 2024
9e1e56f
Swift: remove obsolete bazel definitions
Nov 6, 2024
02a0021
Swift: tweak mangling of extensions
Nov 6, 2024
51f7129
Swift: accept integration test change
Nov 6, 2024
f0dc4f5
Swift: Convert the dataflow-taint-core-conversions test to labelled s…
geoffw0 Nov 14, 2024
570393f
Swift: Additional test cases.
geoffw0 Nov 14, 2024
c2c23c8
Swift: Address some modelling issues for Swift 6.
geoffw0 Nov 14, 2024
6aa43e0
Swift: Effect on other tests.
geoffw0 Nov 14, 2024
e589b1f
Swift: Fix query barriers.
geoffw0 Nov 14, 2024
8245e6c
Swift: Fix unwanted flows.
geoffw0 Nov 14, 2024
fa43207
Swift: Autoformat.
geoffw0 Nov 18, 2024
8bcc5f4
Merge pull request #17989 from geoffw0/swift6models2
redsun82 Nov 18, 2024
6130679
Swift: Label the now missing cases for CWE-020 and dataflow.
geoffw0 Nov 26, 2024
4585852
Swift: Add another test case.
geoffw0 Nov 26, 2024
d1915c7
Swift: Add a test revealing the issue in pure dataflow.
geoffw0 Nov 26, 2024
814218c
Swift: extract variables as children of `ForEachStmt`
Nov 28, 2024
1d43abf
Swift: Model Collection.makeIterator().
geoffw0 Nov 28, 2024
23ed48e
Swift: Add a couple more makeIterator() implementations to be safe.
geoffw0 Nov 28, 2024
14b70b8
Merge pull request #18143 from geoffw0/swift6models3
geoffw0 Nov 28, 2024
e08eac0
Swift: Fix for Int.description.
geoffw0 Nov 28, 2024
1fc112e
Swift: Fix for OptionSet.
geoffw0 Nov 28, 2024
e9deec7
Rust: Autoformat.
geoffw0 Nov 29, 2024
eeed2c2
Merge pull request #18154 from geoffw0/swift6models4
geoffw0 Nov 29, 2024
9da7e8a
Merge pull request #18142 from github/redsun82/swift-6-for-each-vars
redsun82 Dec 4, 2024
4ac602c
Merge branch 'main' into redsun82/swift-6
Dec 6, 2024
7ee3bf9
Swift: add update/downgrade scripts
Dec 6, 2024
bed6387
Swift: remove linux QL tests check
Dec 6, 2024
f81f30c
Swift: accept test changes
redsun82 Dec 6, 2024
a4f7981
Swift: accept test changes
redsun82 Dec 6, 2024
a82d37e
Swift: fix gracefully accepting missing `ForEachStmt.iteratorVar`
redsun82 Dec 6, 2024
263bd69
Swift: fix artifact update infrastructure
redsun82 Dec 6, 2024
926d65b
Swift: update LFS artifacts and remove override
redsun82 Dec 6, 2024
198417c
Swift: fix filename case for prebuilt artifacts
Dec 6, 2024
3bc822f
Swift: add change note for Swift 6 upgrade
Dec 9, 2024
5ca89ea
Merge branch 'main' into redsun82/swift-6
Dec 12, 2024
92ec7e8
Merge branch 'main' into redsun82/swift-6
redsun82 Dec 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -86,4 +86,5 @@
/misc/ripunzip/ripunzip-* filter=lfs diff=lfs merge=lfs -text

# swift prebuilt resources
/swift/third_party/resource-dir/*.zip filter=lfs diff=lfs merge=lfs -text
/swift/third_party/resources/*.zip filter=lfs diff=lfs merge=lfs -text
/swift/third_party/resources/*.tar.zst filter=lfs diff=lfs merge=lfs -text
16 changes: 8 additions & 8 deletions .github/workflows/swift.yml
Original file line number Diff line number Diff line change
Expand Up @@ -50,17 +50,10 @@ jobs:
- uses: ./swift/actions/build-and-test
build-and-test-linux:
if: github.repository_owner == 'github'
runs-on: ubuntu-latest-xl
runs-on: ubuntu-22.04
steps:
- uses: actions/checkout@v4
- uses: ./swift/actions/build-and-test
qltests-linux:
if: github.repository_owner == 'github'
needs: build-and-test-linux
runs-on: ubuntu-latest-xl
steps:
- uses: actions/checkout@v4
- uses: ./swift/actions/run-ql-tests
qltests-macos:
if: ${{ github.repository_owner == 'github' && github.event_name == 'pull_request' }}
needs: build-and-test-macos
Expand Down Expand Up @@ -109,3 +102,10 @@ jobs:
- uses: actions/checkout@v4
- uses: ./.github/actions/fetch-codeql
- uses: ./swift/actions/database-upgrade-scripts
check-no-override:
if : github.event_name == 'pull_request'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- shell: bash
run: bazel test //swift/... --test_tag_filters=override --test_output=errors
21 changes: 7 additions & 14 deletions MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ bazel_dep(name = "gazelle", version = "0.38.0")
bazel_dep(name = "rules_dotnet", version = "0.17.4")
bazel_dep(name = "googletest", version = "1.14.0.bcr.1")
bazel_dep(name = "rules_rust", version = "0.52.2")
bazel_dep(name = "zstd", version = "1.5.5.bcr.1")

bazel_dep(name = "buildifier_prebuilt", version = "6.4.0", dev_dependency = True)

Expand Down Expand Up @@ -94,10 +95,12 @@ use_repo(
swift_deps,
"binlog",
"picosha2",
"swift_prebuilt_darwin_x86_64",
"swift_prebuilt_linux",
"swift_toolchain_linux",
"swift_toolchain_macos",
"swift-prebuilt-linux",
"swift-prebuilt-linux-download-only",
"swift-prebuilt-macos",
"swift-prebuilt-macos-download-only",
"swift-resource-dir-linux",
"swift-resource-dir-macos",
)

node = use_extension("@rules_nodejs//nodejs:extensions.bzl", "node")
Expand Down Expand Up @@ -189,16 +192,6 @@ lfs_files(
executable = True,
)

lfs_files(
name = "swift-resource-dir-linux",
srcs = ["//swift/third_party/resource-dir:resource-dir-linux.zip"],
)

lfs_files(
name = "swift-resource-dir-macos",
srcs = ["//swift/third_party/resource-dir:resource-dir-macos.zip"],
)

register_toolchains(
"@nodejs_toolchains//:all",
)
6 changes: 6 additions & 0 deletions misc/bazel/lfs.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -77,6 +77,12 @@ def _download_lfs(repository_ctx):
)
repository_ctx.file("BUILD.bazel", build)

# this is for drop-in compatibility with `http_file`
repository_ctx.file(
"file/BUILD.bazel",
'alias(name = "file", actual = "//:%s", visibility = ["//visibility:public"])\n' % name,
)

lfs_archive = repository_rule(
doc = "Export the contents from an on-demand LFS archive. The corresponding path should be added to be ignored " +
"in `.lfsconfig`.",
Expand Down
29 changes: 1 addition & 28 deletions swift/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
load("@rules_pkg//pkg:mappings.bzl", "pkg_filegroup")
load("//:defs.bzl", "codeql_platform")
load(
"//misc/bazel:pkg.bzl",
"codeql_pack",
Expand Down Expand Up @@ -81,7 +80,7 @@ codeql_pack(
zips = select({
"@platforms//os:windows": {},
"//conditions:default": {
"//swift/third_party/resource-dir": "resource-dir/{CODEQL_PLATFORM}",
"//swift/third_party/resources:dir": "resource-dir/{CODEQL_PLATFORM}",
},
}),
)
Expand All @@ -90,29 +89,3 @@ alias(
name = "create-extractor-pack",
actual = ":swift-installer",
)

# TODO: following rules are for internal repo backward compatibility only
geoffw0 marked this conversation as resolved.
Show resolved Hide resolved
alias(
name = "extractor-pack-generic",
actual = "swift-generic",
visibility = ["//visibility:public"],
)

pkg_filegroup(
name = "resource-dir",
srcs = select({
"@platforms//os:linux": ["@swift_toolchain_linux//:resource-dir-files"],
"@platforms//os:macos": ["@swift_toolchain_macos//:resource-dir-files"],
"@platforms//os:windows": [],
}),
prefix = "resource-dir/" + codeql_platform,
)

pkg_filegroup(
name = "extractor-pack-arch",
srcs = [
":resource-dir",
":swift-arch",
],
visibility = ["//visibility:public"],
)
5 changes: 5 additions & 0 deletions swift/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -113,3 +113,8 @@ In particular for breakpoints to work you might need to setup the following remo
### Thread safety

The extractor is single-threaded, and there was no effort to make anything in it thread-safe.

### Updating the swift compiler version

This can only be done with access to the internal repository at the moment. Some (incomplete) instructions are
found [here](third_party/resources/updating.md).
2 changes: 1 addition & 1 deletion swift/actions/build-and-test/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ runs:
if: ${{ github.event_name == 'pull_request' }}
shell: bash
run: |
bazel test //swift/...
bazel test //swift/... --test_tag_filters=-override --test_output=errors
- name: Evict bazel cache
if: ${{ github.event_name != 'pull_request' }}
shell: bash
Expand Down
Loading
Loading