Use the new #available(Android API, *) instead to look for backtrace()

[finagolfin]

Mads just added this compiler feature for Android in swiftlang/swift#84574

I tested this locally on an Android API 35 device in the Termux app, no problem, before transferring the Testing test runner to an API 28 device. It would not link there, since it was an executable and libc.so at API 28 didn't have backtrace(), so I couldn't run the tests that way.

I then ran patchelf --add-needed libandroid-execinfo.so swift-testingPackageTests.xctest to have the test runner use the backported libexecinfo from the Termux app, which got all the tests to run. Three backtrace tests correctly failed because the runtime #available checking disabled this call, showing that runtime version checking worked. 😄

[grynspan]

We need to continue to build with the Swift 6.2 toolchain on other platforms. Do we need to preserve that functionality for Android given there is no official Android 6.2 toolchain? (Also pinging @compnerd.)

If so, the right way forward is to hold off on merging this PR until Swift 6.3 is released, at which point we'll drop 6.2 support and can proceed.

[grynspan]

You can get rid of this #if.

[finagolfin]

I thought so too initially, but the alternative is Android with static linking, in which case it will hit the final #else clause below and error.

[grynspan]

That's not what we use SWT_NO_DYNAMIC_LINKING for. We use it to indicate if dlsym() or equivalent is available. (I named it bad.) So you can just remove it. All Android codepaths will go through here. Even statically-linked Android binaries should be able to use weakly-linked NDK symbols.

[finagolfin]

I see, I thought you meant remove just the #if, meaning adding an &&. I kept the check based on your adding it before and misreading this doc, will remove it.

Unsure how this will interact with static linking executables, given the dynamic linking issue I mentioned, but considering most will static link this into shared libraries eventually, may not matter.

[compnerd]

@grynspan's suggestion makes sense - testing requires dynamic linking (i.e. you cannot statically link the testing framework). We know that the platform supports dynamic symbol resolution and because this is within the testing framework, you will be dynamically linking. As a result, just the new handling for the lookup is needed.

[grynspan]

You can in fact statically link Swift Testing!

[finagolfin]

In any case, we agreed on removing this check, which he had added, not me.

[grynspan]

Yarp.

[grynspan]

It would not link there, since it was an executable and libc.so at API 28 didn't have backtrace(), so I couldn't run the tests that way.

Does this mean that compiling against API 33+ and then running on an older device will fail? If so, that's not what we want: it indicates that weak linking isn't working, which means you can't use if #available(Android) to conditionally enable new NDK API use.

[finagolfin]

Do we need to preserve that functionality for Android given there is no official Android 6.2 toolchain?

Not sure what you mean, is that because trunk in this repo is currently auto-merged to release/6.2? If so, we could also check #if compiler(>=6.3) to make sure this is not parsed by the 6.2 toolchain.

Does this mean that compiling against API 33+ and then running on an older device will fail? If so, that's not what we want: it indicates that weak linking isn't working, which means you can't use if #available(Android) to conditionally enable new NDK API use.

Yep, if it's an executable, which seem to be required on Android to resolve all symbols at startup. Since the SwiftPM-produced test runner executable for this repo, that I built against a target, ie minimum, of API 24, doesn't invoke these methods from the toolchain's libTesting.so, but keeps them in the test runner and invokes them from there, I believe that's why it failed.

I don't think it will be an issue in Android apks, which almost always ship with native code in shared libraries instead, but haven't looked into it in detail.

[stmontgomery]

If so, we could also check #if compiler(>=6.3) to make sure this is not parsed by the 6.2 toolchain.

That was going to be my suggestion as well.

[grynspan]

Not sure what you mean, is that because trunk in this repo is currently auto-merged to release/6.2? If so, we could also check #if compiler(>=6.3) to make sure this is not parsed by the 6.2 toolchain.

It is possible and supported to build Swift Testing as a package using the current released toolchain (6.2). So we need to keep that working.

Yep, if it's an executable, which seem to be required on Android to resolve all symbols at startup. Since the SwiftPM-produced test runner executable for this repo, that I built against a target, ie minimum, of API 24, doesn't invoke these methods from the toolchain's libTesting.so, but keeps them in the test runner and invokes them from there, I believe that's why it failed.

I would expect weak linking to NDK symbols to work even for executable targets. If it doesn't, then we can't use #available here and must continue using dynamic lookup indefinitely.

[finagolfin]

It is possible and supported to build Swift Testing as a package using the current released toolchain (6.2). So we need to keep that working.

Ah, I see, changing the dynamic linking check to #if compiler(>=6.3) instead okay then?

I would expect weak linking to NDK symbols to work even for executable targets. If it doesn't, then we can't use #available here and must continue using dynamic lookup indefinitely.

I don't know whether it should work differently for executables or not, was just presenting my hypothesis, as most won't use executables for testing their apps anyway. However, I'll look into it with Mads and let you know.

[grynspan]

I don't know whether it should work differently for executables or not, was just presenting my hypothesis, as most won't use executables for testing their apps anyway. However, I'll look into it with Mads and let you know.

Let's see if we can solve the weak-linking mystery and then revisit. If it turns out it's fine, or it's some narrow edge case we really don't need to care about, great. If there's a bug in how #available functions on Android, I'd prefer we resolve that bug before proceeding. Thanks!

[grynspan]

When you tried to run on API 28, had you set a minimum deployment target at compile time? Do we have a mechanism for doing so when building for Android? If it built against API 35 and you didn't set a minimum deployment target, I'd expect it to not bother weak-linking (same as on Darwin.)

[finagolfin]

Yep, target, ie minimum, was API 24, mentioned it above. Mads and I are looking into it, could be something specific to the toolchain I natively built in the Termux app on Android, will let you know in a day or two.

[grynspan]

D'oh, misread. Okay.

[finagolfin]

It was ninja edited in, so maybe you read the first version. 😉

[finagolfin]

We looked into the weak linking issue: it only shows up in the natively-built toolchain in the Termux app, not when cross-compiling. I'll look into the Termux issue later, but no reason to keep this waiting on that.

Rebased and made the changes discussed, ready to go ahead.

[grynspan]

Alright. We can merge once CI is passing.

[finagolfin]

Been discussing this pull with @marcprux and @madsodgaard: turns out this pull won't build with the next trunk Android SDK snapshot tag either, because this new #available(Android <API>, *) feature requires Android NDK 28 or later, but we currently build the Swift SDK for Android and use it with LTS NDK 27 only.

I just passed in NDK 28 on github to the official Docker scripts that build the SDK snapshots, and other than four compiler validation suite tests failing, everything worked, including locally cross-compiling this pull.

@compnerd, what do you think about switching the trunk 6.3 SDK snapshots over to NDK 28 now, both in Docker and in the Windows toolchain? We can still keep the current 6.2 release branch and the official 6.2 CI for Android on LTS NDK 27, thus only offering LTS NDK support for actual releases.

This will allow us to start preparing for the final 6.3 release in 4-5 months, so it will then use what will likely be the next LTS NDK 30 around that time.