another attempt at getting Swift to build; works but I think it rebuild antlr lib each time.

[parrt]

Heh @tonyarnold how about this one?

[parrt]

Runs stuff like:

RUNNING swift build -c release in /Users/parrt/antlr/code/antlr4/runtime/Swift
RUNNING swift build -c release -Xlinker -L/Users/parrt/antlr/code/antlr4/.build/release -Xlinker -lAntlr4Static in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-25-1669420733260
RUNNING swift build -c release -Xlinker -L/Users/parrt/antlr/code/antlr4/.build/release -Xlinker -lAntlr4Static in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-39-1669420733257
RUNNING swift build -c release -Xlinker -L/Users/parrt/antlr/code/antlr4/.build/release -Xlinker -lAntlr4Static in /var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-33-1669420733251
...

[parrt]

Maybe @nesevis or @edigaryev can comment. This is much simpler than #3969

[tonyarnold]

This was exactly what I was going to try next, so if this works 👍🏻 Great stuff, thank you for persisting with getting this working!

[tonyarnold]

Three hours on CI? 😬 I'm going to call time of death on this approach.

So the next step would be to build a distributable .xcframework from the Antlr Swift library, and use that as a binary dependency. That would be following a supported use of Swift Packages, so it should Just Work™ (nervous laughter).

It's the weekend here, so I'm not going to be able to look at this properly for a couple of days, but if you're happy to wait I will pull together the scripts to build an XCFramework from a Swift package, and we can try that.

[edigaryev]

I'm fine with any Swift-related change as long as one will be able to link against ANTLR 4 statically and use it from Swift without forking this repository.

Also, consider trying out the Cirrus CI for faster build times with Apple silicon.

[parrt]

one will be able to link against ANTLR 4 statically and use it from Swift without forking this repository.

I totally agree. that's what I'm hoping to do here: build the runtime lib ONCE then reuse/bind into a test example. It's has to be a simple matter of command-line args. I'm enclosing a sample generated test rig example.

example.zip

Currently the antlr test infra runs this equivalent:

$ rm -rf .build                       # I'm doing this manually to ensure fresh test
$ swift build -c release -Xlinker -L/Users/parrt/antlr/code/antlr4/.build/release -Xlinker -lAntlr4Static
'swiftrunner-forkjoinpool-1-worker-39-1669422098223': warning: found 1 file(s) which are unhandled; explicitly declare them as resources or exclude from the target
    /private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-39-1669422098223/example.zip

Building for production...
/Users/parrt/antlr/code/antlr4/runtime/Swift/Sources/Antlr4/UnbufferedCharStream.swift:336:9: warning: switch covers known cases, but 'Stream.Status' may have additional unknown values, possibly added in future versions
        switch stream.streamStatus {
        ^
/Users/parrt/antlr/code/antlr4/runtime/Swift/Sources/Antlr4/UnbufferedCharStream.swift:336:9: note: handle unknown values using "@unknown default"
        switch stream.streamStatus {
        ^
/private/var/folders/w1/_nr4stn13lq0rvjdkwh7q8cc0000gn/T/SwiftRunner-ForkJoinPool-1-worker-39-1669422098223/TParser.swift:132:44: warning: left side of nil coalescing operator '??' has non-optional type 'String', so the right side is never used
                        print((_localctx.r.toStringTree(self)) ?? "nil")
                              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^~~~~~~~
                                                               
[3/3] Linking Test
Build complete! (11.47s)

Unfortunately, this builds the runtime and dumps into current dir's .build rather than reusing the static lib. Must be a config or command line thing to fix easily. Hopefully you guys can use this as a base to more quickly figure it out. :)

[parrt]

btw, I added a bug about the github actions not installing some targets properly #3988

[parrt]

Hi @tonyarnold had a chance to take a peek at this? If you can make example.zip reuse the existing library we are 99% there!

[tonyarnold]

Apologies for the delay in replying - I've been travelling for work.

Okay, so the good news is that I have part of a workable solution. The bad news is that it is only part of a solution.

If you (temporarily) force the Antlr4 library in Package.swift to have a type: .static (we don't want to force this option on for production builds, but for testing it's fine):

// swift-tools-version:5.7

import PackageDescription

let package = Package(
    name: "Antlr4",
    products: [
        .library(name: "Antlr4", type: .static, targets: ["Antlr4"]),
    ],
    // …

Then execute the following commands to build an XCFramework (assuming that antlr4-checkout is the project sources, and ~/Downloads/example is the example zip you posted above:

$ cd antlr4-checkout
$ swift build --configuration release
…
$ swift build --configuration release --show-bin-path
/Users/tonyarnold/Developer/Repositories/antlr4/.build/arm64-apple-macosx/release
$ xcodebuild -create-xcframework -library .build/arm64-apple-macosx/release/libAntlr4.a -allow-internal-distribution -output Antlr4.xcframework
$ cp Antlr4.xcframework ~/Downloads/example/

Now, update the example Package.swift to add the XCFramework as a binary target, like so:

// swift-tools-version: 5.6

import PackageDescription

let package = Package(
    name: "Test",
    products: [
        .executable(name: "Test", targets: ["Test"]),
    ],
    targets: [
        .executableTarget(
            name: "Test",
            dependencies: [
                .target(name: "Antlr4")
            ],
            path: ".",
            exclude: [
                "TLexer.tokens", "T.interp", "T.g4", "T.tokens", "TLexer.interp", "Antlr4.xcframework"
            ]
        ),

        .binaryTarget(
            name: "Antlr4",
            path: "Antlr4.xcframework"
        )
    ]
)

You should now be able to open the Package.swift in Xcode 14.x, and it will build the binary correctly.

Unfortunately, swift build (which is what we should be running) still fails to build:

~/Downloads/example/main.swift:1:8: error: no such module 'Antlr4'
import Antlr4

[tonyarnold]

It looks like what I was proposing will work once this fix makes it into a released version of Swift and Xcode: swiftlang/swift-package-manager#5724

[parrt]

Well this is kind of disappointing that Swift can't reuse an existing library. Finding it hard to believe but I have no idea. I guess we can only turn off the swift tests until set a time is Swift can run the test in a couple of minutes.

[parrt]

I turned off the swift tests with 539ffaf

[tonyarnold]

Well this is kind of disappointing that Swift can't reuse an existing library.

It's not Swift, per se, it's Swift's package manager (SwiftPM). Swift itself supports what you're trying to do, it just gets extremely manual to get it to do it - we'd need to manually find and link all the object files ourselves (which is totally plausible, but outside of what I know how to do right now).

Once SwiftPM has had that PR merged, this should work without a lot of fuss.

[parrt]

ok. just surprising people have tolerated this lack of build reuse for years. seems odd. make xcode hides this?