-
Notifications
You must be signed in to change notification settings - Fork 1.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce initial Swift support in fdbserver #10156
Conversation
[swift] add radar links for C++ interop issues found in getVersion br…
This removes the workaround for the FRT type layout issue, and gets us closer to making MasterData a full reference type
Migrate MasterData closer to full reference type
[Swift] fix computation of toAdd for getVersion Swift implementation
@jzhou77 fixed the formatting in those cmake files. |
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've looked through C++ changes. Mostly nits and trying to preserve the existing C++ behavior.
private: | ||
ACTOR static Future<LoadedTLSConfig> loadAsync(const TLSConfig* self); | ||
#endif | ||
ACTOR static Future<LoadedTLSConfig> loadAsync(const TLSConfig* self); // FIXME |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
can you be more specific of "FIXME" actions?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure why it's here. I can check, but I think probably something to do with Swift importing this module creating issue.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
4f53df0
to
b3e9a72
Compare
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
This comment was marked as outdated.
Doxense CI Report for Windows 10
|
This comment was marked as outdated.
This comment was marked as outdated.
Result of foundationdb-pr-clang-ide on Linux CentOS 7
|
Result of foundationdb-pr on Linux CentOS 7
|
Result of foundationdb-pr-cluster-tests on Linux CentOS 7
|
Result of foundationdb-pr-clang on Linux CentOS 7
|
Hello everyone,
This is a cumulative draft pull request of an effort we had ongoing for a while now that enables the use of Swift to implement (or port) FDB functionality. The pull request specifically extended the cmake build to support Swift "side by side" existing fdb modules (flow, fdbrpc, fdbclient, fdbserver), integrated the Swift concurrency runtime into sim2 and net2 runtimes and proofs that this works by porting masterserver logic to Swift.
Swift
Swift brings to the table its strong typesafety and compiler verified concurrency model along with async-await, tasks and actors (and even distributed actors, though not shown in this PR yet) all built into the language.
Swift Concurrency and Flow
Swift comes with a fully fledged actor model runtime as well as distributed actors (which we're not making use of yet though in this PR).
Of course, it is important to keep the existing runtime characteristics of Flow and the
sim2
andnet2
eventloops. So we've ensured that all Swift concurrency code executes on the configured Flow evenloop.Both simulation and production workloops are supported. In other words, creating a
Task {}
, child task or running Swiftactor
s (which execute on Tasks), including hitting suspension points (await
) all execute on foundationdb's "usual" scheduling infrastructure.You can learn about the basics of Swift concurrency here: https://docs.swift.org/swift-book/documentation/the-swift-programming-language/concurrency/ and we're more than happy to provide additional guidance, docs, or just chat about the capabilities.
Swift / C++ Interoperability
To enable all of this we're using Swift's new C++ interopability feature in order to directly call into existing Flow/C++ code (no shims, or bindings!), and can support interop both ways: Flow/C++ calling Swift, or Swift using existing C++ types and functions. Swift is even able to ref-count existing C++ types (e.g. see
MasterData
).You can learn more about the C++ interop here:
Status of the PR
The pull requested code is functional and we've been passing manually executed existing tests for a while now, but it is possible we missed some issues. For build instructions you can refer to the SWIFT_GUIDE.md. For setting up a developer environment including IDE with autocompletion across languages you can refer to the SWIFT_IDE_SETUP.md guide.
We'll continue working on more cleanups (e.g. re-enable some commented out code), improvements in quality of life and some more build adjustments in this pull request for the time being.
Our immediate next steps are that @ammolitor will prepare the necessary the integration testing jobs so we'll be able to verify this code does not regress accidentally. At the same time, we'll want to make sure building with Swift is not causing the current developer workflow any issues, so that's another focus area we'll look at.
Overall, the goal here is to allow for a non-disruptive introduction of the language, and allow development of new features in Swift. Right now there is a
FLOW_WITH_SWIFT
Knob that can be used to toggle between using Swift or C++/Flow implementations of masterserver (seemasterserver.swift
andmasterserver.actor.cpp
), so we can build a binary with "both versions" and gain confidence before turning on Swift implementations for good. TheWITH_SWIFT
build time option does not yet entirely enable/disable building Swift but the intent would be to do that in order to smooth out adoption.Continuous Integration
This PR right now is expected to fail, and we're working with @ammolitor to prepare a suitable CI configuration.
Not featured in this PR yet
Swift also has macros and distributed actors which we intend to use to simplify and streamline some of the boilerplate involved in crossing the languages today. Macros specifically we'll be able to use to get rid of the manual wrappers Flow future using code into async functions in Swift. Though these are not included in this initial effort yet.
FYI: @mpilman @sfc-gh-jslocum @sfc-gh-ljoswiak @vishesh @ammolitor @jleach4 @xis19
From the Swift team @zoecarver, @hyp (for Swift/C++ topics), @etcwilde (with most build expertise) and myself (mostly concurrency, however feel free to ping me with any questions) are all available to answer questions or debug issues.