Token creation fails on ubuntu but not macOS

[leisurehound]

Hi, I have a Swift command line app that writes to Firebase Firestore after generating a JWT for a service account and works fine on macOS, but as I'm porting it to Ubuntu to run it under systemctl I'm failing to successfully generate the token. Its always returning nil. I thought at first that it may be because the key had a trailing \n but removing that doesn't seem to help. Any assistance would be appreciated.

func constructJWT() throws -> String? {
        //TODO:  get the private kay location from an env variable.
        let myKeyPath: URL
        #if os(macOS)
            myKeyPath = URL.init(fileURLWithPath: "/Users/USERNAME/development/Firebase_RSA.privatekey")
        #endif
        #if os(Linux)
            myKeyPath = URL.init(fileURLWithPath: "/home/USERNAME/development//Firebase_RSA.privatekey")
        #endif
        var key: Data = try Data(contentsOf: myKeyPath, options: .alwaysMapped)
        var keyString = String(data: key, encoding: .utf8)
        print(keyString)
        if let str = keyString, str.last == "\n" {
            keyString = String(str.dropLast())
            if let keyData = keyString?.data(using: .utf8) {
                key =  keyData
            }
            print("Drop trailing return")
            print("Key:  ", keyString)
        }

        let header = Header([.typ:"JWT", .alg:"rsa256"])
        
        let issuedTime = Date()
        let timeToLive: TimeInterval = tokenDuration
        let expirationTime = issuedTime.addingTimeInterval(timeToLive)
        
        let claimsDict: [String:Any] = [ClaimKeys.iss.rawValue:"firebase-....gserviceaccount.com",
                                        "scope":"https://www.googleapis.com/auth/datastore",
                                        ClaimKeys.aud.rawValue:"https://www.googleapis.com/oauth2/v4/token",
                                        ClaimKeys.iat.rawValue:Int(issuedTime.timeIntervalSince1970),
                                        ClaimKeys.exp.rawValue:Int(expirationTime.timeIntervalSince1970)]
        
        let claims = Claims(claimsDict)
        
        var jwt = JWT(header: header, claims: claims)
        let signedJWT: String? = try jwt.sign(using: .rs256(key, .privateKey))
        print("Signed Token:  ", signedJWT)
        return signedJWT
    }

[leisurehound]

well, it did end up being a key formatting issue. Even after executing the steps in the other recently opened issues I was still having problems so thought it was different, but found another formatting error which when fixed resolved the issue. Would be nicer if the signing was a little more forgiving tho, like it is on macOS.

[Andrew-Lees11]

hello @leisurehound
I'm glad you were able to fix this issue. What was the formatting issue you were having? The behavior should be the same on macOS and Linux so I would like to look into what was causing the difference.

[leisurehound]

I had to do the formatting that were described (specifically remove the \n and the manually edit the files, add the header) but then I must have entered a non-printable character by mistake. once I found that (after doing the other edits) it worked. However, on macOS I simply had to download the Google service account private key and it worked. On Ubuntu I had to manually edit the google file. Would be great if it Just Worked(tm) on either platform (admittedly not expecting it to just work from pilot error entering hidden chars, but rather take file from google, load it and it works).

[leisurehound]

Interestingly, I'm now having the opposite problem. I'm in the process of rotating keys as I had to do some Heroku key debugging (the ENV didn't like spaces or my quoting the key, so I added %20 and %0A for all the spaces & newlines, then convert them back in swift after reading the ENV variable). With the new key this works great on Heroku.

But on macOS, with Google Cloud Console and gcloud command line returning a JSON file now, with the private key as a field in the JSON, when I pick that out and create a private key file, read it in I get a try failure on the last line below.

       #if os(macOS)
        var key: Data {
            let myKeyPath = URL.init(fileURLWithPath: "/Users/me/development/Firebase_RSA.privatekey")
            guard let keyData = try? Data(contentsOf: myKeyPath, options: .alwaysMapped) else {
                fatalError("Could not retrieve Firebase Privatekey from \(myKeyPath.absoluteString)")
            }
            return keyData
        }
       #endif 

        var jwt = JWT(claims: claims)
        let signer = JWTSigner.rs256(privateKey: key)
        let signedJWT: String? = try jwt.sign(using: signer)

While the #if os(linux) code (no included above) gets the same key from an ENV variable and works in Heroku, this key does not work on macOS. I've tried taking the private key from the downloaded JSON file and save it with TextMate, vi, and Xcode via a scheme env variable, thinking one my be inserting something I can't see, but alas same behavior on all.

I'm using Swift-JWT 3.4.0 if that matters.

Thoughts?

[Andrew-Lees11]

@leisurehound I have raised pull request #68 that should handle newlines a bit better.
Would you be able to test the keyGen branch and see if that solves you issue?

[leisurehound]

I get duplicate logging symbols when depending on that branch:

$ swift build
error: multiple targets named 'Logging' in: Console, swift-log

Here is how I'm referencing that branch.

    dependencies: [
        .package(url:"https://github.com/IBM-Swift/Swift-JWT.git", .branch("keyGen")),

$ swift package update
Updating https://github.com/vapor/vapor.git
Updating https://github.com/IBM-Swift/Swift-JWT.git
Updating https://github.com/IBM-Swift/BlueRSA.git
Updating https://github.com/IBM-Swift/BlueECC.git
Updating https://github.com/IBM-Swift/BlueCryptor.git
Updating https://github.com/IBM-Swift/KituraContracts.git
Updating https://github.com/IBM-Swift/LoggerAPI.git
Updating https://github.com/apple/swift-log.git
Updating https://github.com/vapor/service.git
Updating https://github.com/vapor/template-kit.git
Updating https://github.com/vapor/url-encoded-form.git
Updating https://github.com/vapor/crypto.git
Updating https://github.com/vapor/http.git
Updating https://github.com/vapor/console.git
Updating https://github.com/vapor/multipart.git
Updating https://github.com/vapor/database-kit.git
Updating https://github.com/vapor/routing.git
Updating https://github.com/vapor/core.git
Updating https://github.com/vapor/validation.git
Updating https://github.com/vapor/websocket.git
Updating https://github.com/apple/swift-nio.git
Updating https://github.com/apple/swift-nio-zlib-support.git
Updating https://github.com/apple/swift-nio-ssl-support.git
Updating https://github.com/apple/swift-nio-ssl.git
Completed resolution in 7.90s
Resolving https://github.com/IBM-Swift/Swift-JWT.git at keyGen

Tried to swift package clean in between, which did not resolve it. Sorry, am a bit new to SPM sorry if this is a basic question.

[Andrew-Lees11]

@leisurehound
This is caused by a name-space conflict between Vapor logging and Swift-Log. There is a forum post describing this issue.
The short term fix is to limit the LoggerAPI version so it doesn't depend on Swift-log:

.package(url: "https://github.com/IBM-Swift/LoggerAPI.git", .upToNextMinor(from: "1.8.0"))

[leisurehound]

Thanks for the tip, am past the building with duplicate symbols issue. Updating to use the keyGen branch I'm still seeing this error when trying to sign:

The operation couldn’t be completed. (CryptorRSA.CryptorRSA.Error error 1.)

[Andrew-Lees11]

I'm wondering if the error could be in getting the key from the file.
Could you check the key is correct? If you utf8 decode the data to a String, and remove the headers, you should be able to ASN1 decode the key here. If it successfully decodes could you let me know the structure of the key?
Otherwise could you try putting the PEM string key into the swift file directly, UTF8 encoding it to data and seeing if we can use it to sign the JWT?

[leisurehound]

Am not sure how to interpret the results of the ASN1 site.

SEQUENCE (3 elem)
  INTEGER 0
  SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.1.1 rsaEncryption (PKCS #1)
    NULL
  OCTET STRING (1 elem)
    SEQUENCE (9 elem)
      INTEGER 0
      INTEGER (2048 bit) 233874767721255060139214951444040486286214425089537775943792072501918…
      INTEGER 65537
      INTEGER (2047 bit) 103510559111993595593299493134959463283609802312390447060830261607803…
      INTEGER (1024 bit) 153831302701951280450579453179928247974659679221181895093701932627394…
      INTEGER (1024 bit) 152033275161420358018338804855116604514542524721639212160035053030695…
      INTEGER (1023 bit) 767079202938762507457609675438310442011669486999439146079646483400711…
      INTEGER (1021 bit) 145034413584570559886881335421851627542444705213801113023870355913133…
      INTEGER (1024 bit) 112505708108801475347934017254847497431988546498446736134447279840118…

Here's the code with getting the key data on macOS (ignore the force unwrap unwrap for now, it just a shortcut to get this to work):

    func constructJWT() throws -> String? {
        //TODO:  get the private kay location from an env variable.
       #if os(macOS)
        var key: Data {
            let keyString = """
MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQC5Q7ZwZErgnoD9
ZYrksFRBjmj4TvuRzn/35okDkAZQaarufQM4siivO7Z/WbJ+HBwK8hm2/AZZk+NK
N10rs9zmGMN2/Joz4hJ6xE93ogQ1pqNSbHZwjX6vX0RYt9mUQ/lep+T81JboXFbj
wPKITWmocHQelgbsGTqQlXe4wftLStnsgob08IWrSyUM+CfmyCw5Lh3XdZyzWT7G
CUKej0zWxyTNsMwD53EGFG/iF2XSp4YPzLN7EDPeAFiObo9+HxdzrkjaPqEzGz0Z
// lines deleted; there were 26 lines of key total
apJDLTz3nJ/AOXBpXMDuWF/M7IatoqjPpmqXynMg4KdNFvr3TnwhsMI36Gj6uf9z
/GTd0tj1fBQnZZE20EEN+tn5LTAS/bxpVPlkGzu0qA5OHID3A38f0ca+VTDwWRLY
c+vv1us6a4IRovw1v9ju0RZUKIr2TPSebg8FQVECgYEAoDakbNkqqKGTwvGo65g7
JedWM9EpxebaDKZCWiuQ/y5rbFC3D5tjYjjQk+jYgvWfygCXShXTJzXRtw/9aQZI
v9i6m4wjpNz8d9LhM3pKpueaeHc5Htwess1tDWgdGvEvsWBBlLcWUCBkaYgKbQbt
Wfoz04HaTyLisdBZywmB2qA=
"""
            print(keyString )
            let keyData = keyString.data(using: .utf8)
            return keyData!
        }

I get the same error I shared above.

I can share my claims too, if that is not a security problem or direct message me to share it privately.

Again, here is how I try to sign, this code is platform independent between the dev env on macOS and test/production on Heroku:

        var jwt = JWT(claims: claims)
        let signer = JWTSigner.rs256(privateKey: key)
        var signedJWT: String = ""
        do {
            signedJWT = try jwt.sign(using: signer)
        } catch let error {
            print(error.localizedDescription)
        }

When I get the key from an env variable on Heroku (where I've %20 the spaces and %0A the new lines in the env variable, and then return them back to spaces/new lines in the swift string) the key signs successfully on Heroku. Here again is the code I use on Linux:

        #elseif os(Linux)
        var key: Data {
            guard var keyString : String = ProcessInfo.processInfo.environment["FIREBASESERVICEKEY"] else {
                fatalError("FIREBASESERVICEKEY is not set in the environmeent, giving up")
            }
            keyString = keyString.replacingOccurrences(of: "%20", with: " ")
            keyString = keyString.replacingOccurrences(of: "%0A", with: "\n")
            return Data(keyString.utf8) 
        }
        #endif

When I get the key on macOS either from the string literal or from the file I get the sign error. If I put the % escaped key into the string literal on macOS and replace the %20/%0A with spaces and new lines I still get the error. Essentially, keys that work on Linux fail on macOS. Very odd.

[Andrew-Lees11]

Ok so I think the problem is that you have a PKCS8 formatted key. This means that when you ASN1 decode it as you have done above you have the following header:

SEQUENCE (3 elem)
  INTEGER 0
  SEQUENCE (2 elem)
    OBJECT IDENTIFIER 1.2.840.113549.1.1.1 rsaEncryption (PKCS #1)
    NULL

The linux implementation seems to be able to handle this but the mac one doesn't. I have raised a PR 49 against BlueRSA which should fix this problem.

You can test this by pointing at the branch:

.package(url: "https://github.com/IBM-Swift/BlueRSA.git", .branch("pkcs8")),

Otherwise you can try and convert your key from pkcs8 to pkcs1.
If you look at issue #34 They had the same problem.

Once you convert the key the PEM header should be:

-----BEGIN RSA PRIVATE KEY-----

[leisurehound]

Thanks Andrew. Depending on the BlueRSA branch resolved the problem. Appreciate your help. Looking forward to seeing the changes in the main line.

[ianpartridge]

Thanks for testing it out. I've taken this issue over from Andy and we hope to get the BlueRSA fix out soon. Thanks.

[leisurehound]

I see the fix in the PR for Blue-RSA is failing on CI. Is there an ETA on the merge so I can reset my dependencies to the mainline? It also appears the formatting change PR is also not merged.

[ianpartridge]

Thanks for the prod!

I fixed the BlueRSA CI problem, hopefully we can merge that soon.

I also merged #68 and released https://github.com/IBM-Swift/Swift-JWT/releases/tag/3.5.2 with the fix.

By the way, you can work around the problem by converting your key from PKCS8 to PKCS1 format - it should be as simple as running openssl rsa -in server.key -out server_new.key.