Skosmos UI loads in Arabic

[namedgraph]

Some users are reporting that the Skosmos UI loads in Arabic by default, without having chosen that language.

Will try to investigate and gather more information.

[namedgraph]

General

Request URL: https://localhost:9090/something/
Request Method: GET
Status Code: 302
Remote Address: 127.0.0.1:443
Referrer Policy: strict-origin-when-cross-origin

Response Headers

Alt-Svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
Cf-Cache-Status: DYNAMIC
Cf-Ray: 7cb53b388a5f23e9-LHR
Content-Type: text/html; charset=UTF-8
Date: Mon, 22 May 2023 12:58:33 GMT
Location: ar/
Server: cloudflare
Strict-Transport-Security: max-age=0; includeSubDomains
Vary: Accept-Language
X-Content-Type-Options: nosniff

Request Headers

:Authority: localhost
:Method: GET
:Path: /
:Scheme: https
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
Cookie: SKOSMOS_SEARCH_LANG=; CF_AppSession=n053caa872f67e200; CF_Authorization=eyJhbGciOiJSUzI1NiIsImtpZCI6Ijg4ZjdmNjlkODQwYzA4YzYxMTZhYjAzZGMyYjI2NDZmNWNkYTdjNzhiMDQ1NGNlZDU5YTE4Njc0OTUzM2NhYjAifQ.eyJhdWQiOlsiOWQ4N2UwM2Y0NDQ1Yzc5ZjBmYzA1MGQ2ZWY1MDcyMTMzOTQzY2U5ZjQwNDE1ZjA3ZTlkYjk0YWVmMmY4ZTE5OCJdLCJlbWFpbCI6InZhaXNoYWxpLnJhZ2h2YW5pQGJlYW1lcnkuY29tIiwiZXhwIjoxNjg0ODAyMjU0LCJpYXQiOjE2ODQ3NTkwNTQsIm5iZiI6MTY4NDc1OTA1NCwiaXNzIjoiaHR0cHM6Ly9iZWFtZXJ5LmNsb3VkZmxhcmVhY2Nlc3MuY29tIiwidHlwZSI6ImFwcCIsImlkZW50aXR5X25vbmNlIjoiS2V3Vm95bzVaUUp3RDZLTiIsInN1YiI6IjhmZWY3ZDM2LTkyNWEtNTU2Ny05ZGMzLWM2MzAzYTE4OTA1YSIsImNvdW50cnkiOiJHQiJ9.5qE4GPUqY907Un7xytzo3h6odSVHNmoUzgCpmu61YQ62JGnq5w-KpBWJOztNPcDEp1qGmDqACGkxbBT05uPpAGYBB3OGdmHmSKXIid1raXHw4gPOe0O7D3WOHcKNhwFzPwrj9k-zYKYWzkDHZ5MJ-tj1FmkMpfn5Zx6_-4933a5rNQkVrOaN18BKxZqWmCDuOW4mLlk2BSRqO7UCMvqXmD6K6D4DbNPlke9-Ng8QNllUftUTOwocLoylbugpxKoZgoGhmixMDdCbKgV6vBFdalUjLemRWnI1X6CEI6SG5ZM06AEQ6DnBw4nR30269Q_UE7UroNLlkIU8NfSqGH1wtA
Sec-Ch-Ua: "Google Chrome";v="113", "Chromium";v="113", "Not-A.Brand";v="24"
Sec-Ch-Ua-Mobile: ?0
Sec-Ch-Ua-Platform: "macOS"
Sec-Fetch-Dest: document
Sec-Fetch-Mode: navigate
Sec-Fetch-Site: none
Sec-Fetch-User: ?1
Upgrade-Insecure-Requests: 1
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/113.0.0.0 Safari/537.36

[osma]

What languages have been configured in config.ttl using skosmos:languages?

[namedgraph]

@osma we have the default value:

    skosmos:languages (
        [ rdfs:label "ar" ; rdf:value "ar_AE.utf8" ]
        [ rdfs:label "da" ; rdf:value "da_DK.utf8" ]
        [ rdfs:label "de" ; rdf:value "de_DE.utf8" ]
        [ rdfs:label "en" ; rdf:value "en_GB.utf8" ]
        [ rdfs:label "en_US" ; rdf:value "en_US.utf8" ]
        [ rdfs:label "es" ; rdf:value "es_ES.utf8" ]
        [ rdfs:label "fa" ; rdf:value "fa_IR.utf8" ]
        [ rdfs:label "fi" ; rdf:value "fi_FI.utf8" ]
        [ rdfs:label "fr" ; rdf:value "fr_FR.utf8" ]
        [ rdfs:label "it" ; rdf:value "it_IT.utf8" ]
        [ rdfs:label "nb" ; rdf:value "nb_NO.utf8" ]
        [ rdfs:label "nl" ; rdf:value "nl_NL.utf8" ]
        [ rdfs:label "nn" ; rdf:value "nn_NO.utf8" ]
        [ rdfs:label "pl" ; rdf:value "pl_PL.utf8" ]
        [ rdfs:label "pt" ; rdf:value "pt_PT.utf8" ]
        [ rdfs:label "pt_BR" ; rdf:value "pt_BR.utf8" ]
        [ rdfs:label "ru" ; rdf:value "ru_RU.utf8" ]
        [ rdfs:label "sv" ; rdf:value "sv_SE.utf8" ]
        [ rdfs:label "zh" ; rdf:value "zh_CN.utf8" ]
    ) ;

ar is suspiciously the 1st on the list ;) Is there some logic that makes the UI to default to the first value?

[osma]

This is the code that selects the language. The choice is informed by

1. SKOSMOS_LANGUAGE cookie, if it exists
2. (if inside vocabulary) default language of the vocabulary
3. Accept-Language header
4. If nothing above works, pick the first one from the language list

It looks like 4. happens, even though at least 3. should work in this case, as the request included an Accept-Language header which should have matched en_US and en.

But I think these response headers are suspicious:

Alt-Svc: h3=":443"; ma=86400, h3-29=":443"; ma=86400
Cf-Cache-Status: DYNAMIC
Cf-Ray: 7cb53b388a5f23e9-LHR
Server: cloudflare
X-Content-Type-Options: nosniff

These are not typical Skosmos/PHP/Apache response headers. Why is Server: cloudflare in there? Is this going through some frontend proxy? Is it possible that the response was cached earlier and the cache returns a stale response?

[namedgraph]

Yes there's a proxy in front.

We also have this in the config:

    skosmos:language "en";
    skosmos:defaultLanguage "en";

Assuming $acceptLanguage is detected correctly, could it be that this if condition fails?

Skosmos/controller/WebController.php

Line 106 in a6f1795

if (isset($bestLang) && in_array($bestLang, $langcodes)) {

If the skosmos:languages values are compared directly with Accept-Language values (without any replacement that I might have missed) it's not going to work, because they use different separators: en_US vs en-US.

[osma]

We also have this in the config:

skosmos:language "en";
skosmos:defaultLanguage "en";

These are vocabulary specific settings. They are not relevant in this case, since the request was for the front page, outside of any vocabulary.

If the skosmos:languages values are compared directly with Accept-Language values (without any replacement that I might have missed) it's not going to work, because they use different separators: en_US vs en-US.

That's a good point. I think you should use rdfs:label "en-US" in the config file.

[namedgraph]

Right.

But the lang codes are passed to the \Negotiation\LanguageNegotiator class which uses _ because that's the PHP convention?

I'm guessing some sort of mapping has to happen between the two lang code conventions.

[osma]

I don't think there exists a PHP convention for language tags. Mostly the en-US style (BCP 47) is used everywhere on the internet, but Linux locale names typically use underscores as a delimiter instead (though this may vary between distributions).

So the skosmos:languages setting is effectively this mapping: the rdfs:label part gives the BCP 47 language tag, while the rdf:value gives the corresponding locale (for looking up things like date and time formats).

[namedgraph]

I don't think there exists a PHP convention for language tags. Mostly the en-US style (BCP 47) is used everywhere on the internet, but Linux locale names typically use underscores as a delimiter instead (though this may vary between distributions).

PHP locale names is what I meant :)

So the skosmos:languages setting is effectively this mapping: the rdfs:label part gives the BCP 47 language tag, while the rdf:value gives the corresponding locale (for looking up things like date and time formats).

In this case the en_US and pt_BR language tags in dockerfiles/config/config-docker-compose.ttl are incorrect?
https://github.com/NatLibFi/Skosmos/blob/master/dockerfiles/config/config-docker-compose.ttl#L40

[osma]

In this case the en_US and pt_BR language tags in dockerfiles/config/config-docker-compose.ttl are incorrect?

You are probably right. I was looking at config.ttl.dist which is the default config for non-docker users. It does not have these language settings.

Can you verify that the problem is resolved if you change the underscores to hyphens in your config?

[namedgraph]

Will try :)