Upgrading from 11.0.3.2 to 12.0.7 fails - system wide character encoding problems

[walec51]

Web browser based upgrade - I'm on a hosted enviorment

Expected behavior

Clean upgrade to 12.0.7

Actual behaviour

Upgrade failed - a multitude of problems related to wrong character set encoding causing error both on the frontend and backend

Server configuration

Operating system: hosted environment, probably Debian

Database: 5.6 MariaDB

PHP version: 7.1

Nextcloud version: 12.0.7
Updated from an older Nextcloud/ownCloud or fresh install: 11.0.3.2

Nextcloud configuration:

Config report

<?php
$CONFIG = array (
  'instanceid' => 'masked.id',
  'passwordsalt' => 'masked.salt',
  'secret' => 'masked.secret',
  'trusted_domains' => 
  array (
    0 => 'masked.url',
  ),
  'datadirectory' => '/www/walczak_files/data',
  'overwrite.cli.url' => 'https://masked.url',
  'dbtype' => 'mysql',
  'version' => '12.0.7.1',
  'dbname' => 'walczak_files',
  'dbhost' => 'masked.url',
  'dbtableprefix' => 'oc_',
  'dbuser' => 'walczak_files',
  'dbpassword' => 'masked.password',
  'logtimezone' => 'UTC',
  'installed' => true,
  'theme' => '',
  'loglevel' => 1,
  'htaccess.RewriteBase' => '/',
  'maintenance' => false,
  'mail_from_address' => 'nextcloud',
  'mail_smtpmode' => 'php',
  'mail_domain' => 'masked.url',
  'updater.release.channel' => 'stable',
  'updater.secret' => 'masked.secret',
);

Are you using external storage, if yes which one: no

Are you using encryption: no

Are you using an external user-backend, if yes which one: no

Workaround

The discovered workaround probably says more about the problem then the above information.

After a couple of days of tinkering I noticed that in my hosting PHP configurations page that I have an option called mysql.connect_charset which is not set to any value. Its hard to say what it defaulted to on my hosting but after reverting my nextcloud files and database do the state before the upgrade, setting mysql.connect_charset to utf8 and running the upgrade again every thing worked fine.

This suggests that in version 11 this parameter was not read from the environment but hardcoded - at least its default value. I think this is the way it should be as SQL tables also assume that they will get utf8 content.

[MorrisJobke]

for DB stuff cc @nickvergessen @rullzer

[MorrisJobke]

ref https://docs.nextcloud.com/server/13/admin_manual/configuration_database/mysql_4byte_support.html

Should we still invest time into this or just see 11 as EoL and most instances are migrated to 12+ anyways.

[nickvergessen]

see 11 as EoL

This.