Mail is not displayed, JS running ready #560

foxbow · 2017-10-05T07:51:34Z

Steps to reproduce

Upgrade from Nextcloud 11 to 12
Upgrade mail plugin to 0.7.3
Try to check mails

Expected behaviour

The list of folders is shown, the mail titles of the current folder are shown, the selected mail is shown

Actual behaviour

The list of folders is shown, the mail titles of the current folder are shown, the selected mail is not shown, just the roller while one core stays busy. This blocks the whole app on Firefox. On Chrome you can still change folders and on some the mails are displayed just fine. I was not able to find a true pattern here. It has apparently nothing to do with number of mails in the folder, mail formatting (html/text/mixed) or sender. The Chrome javascript console however claims some cache error.

General server configuration

Operating system: Linux J6868 2.6.32-5-xen-686 #1 SMP Sun Jul 13 02:47:03 UTC 2014 i686

Web server: nginx/1.6.2 (fpm-fcgi)

Database: mysql 5.5.57

PHP version: 5.6.31-1~dotdeb+7.1

PHP-modules loaded

 - Core
 - date
 - ereg
 - libxml
 - openssl
 - pcre
 - zlib
 - bcmath
 - bz2
 - calendar
 - ctype
 - dba
 - dom
 - hash
 - fileinfo
 - filter
 - ftp
 - gettext
 - SPL
 - iconv
 - json
 - mbstring
 - session
 - posix
 - Reflection
 - standard
 - shmop
 - SimpleXML
 - soap
 - sockets
 - Phar
 - exif
 - sysvmsg
 - sysvsem
 - sysvshm
 - tokenizer
 - wddx
 - xml
 - xmlreader
 - xmlwriter
 - zip
 - cgi-fcgi
 - PDO
 - curl
 - gd
 - memcache
 - mysql
 - mysqli
 - pdo_mysql
 - pdo_sqlite
 - redis
 - sqlite3
 - xsl
 - mhash
 - Zend OPcache

Nextcloud configuration

Nextcloud version: 12.0.3 - 12.0.3.3

Updated from an older Nextcloud/ownCloud or fresh install: 11.0.x

Where did you install Nextcloud from: web-installer

Are you using external storage, if yes which one: files_external is disabled

Are you using encryption: no

Are you using an external user-backend, if yes which one: n/a

 - activity: 2.5.2
 - bruteforcesettings: 1.0.2
 - calendar: 1.5.5
 - comments: 1.2.0
 - contacts: 2.0.1
 - dav: 1.3.0
 - federatedfilesharing: 1.2.0
 - files: 1.7.2
 - files_pdfviewer: 1.1.1
 - files_sharing: 1.4.0
 - files_texteditor: 2.4.1
 - files_trashbin: 1.2.0
 - files_versions: 1.5.0
 - files_videoplayer: 1.1.0
 - firstrunwizard: 2.1
 - gallery: 17.0.0
 - issuetemplate: 0.2.2
 - logreader: 2.0.0
 - lookup_server_connector: 1.0.0
 - mail: 0.7.3
 - nextcloud_announcements: 1.1
 - notifications: 2.0.0
 - oauth2: 1.0.5
 - password_policy: 1.2.2
 - provisioning_api: 1.2.0
 - serverinfo: 1.2.0
 - sharebymail: 1.2.0
 - survey_client: 1.0.0
 - systemtags: 1.2.0
 - theming: 1.3.0
 - twofactor_backupcodes: 1.1.1
 - updatenotification: 1.2.0
 - workflowengine: 1.2.0

Disabled apps

 - admin_audit
 - audioplayer
 - encryption
 - federation
 - files_external
 - files_mv
 - user_external
 - user_ldap

Content of config/config.php

{
    "instanceid": "5112297a1ec88",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "datadirectory": "\/var\/www\/ocdata",
    "dbtype": "mysql",
    "version": "12.0.3.3",
    "installed": true,
    "loglevel": 1,
    "maintenance": false,
    "forcessl": true,
    "theme": "",
    "maxZipInputSize": 1073741824,
    "allowZipDownload": true,
    "trusted_domains": [
        "cloud.foxbow.de"
    ],
    "filelocking.enabled": "true",
    "mail_from_address": "cloud",
    "mail_smtpmode": "php",
    "mail_domain": "foxbow.de",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "dbname": "owncloud",
    "dbhost": "localhost",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "trashbin_retention_obligation": "auto",
    "updater.secret": "***REMOVED SENSITIVE VALUE***",
    "overwrite.cli.url": "https:\/\/cloud.foxbow.de"
}

Client configuration

Browser: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/61.0.3163.79 Chrome/61.0.3163.79 Safari/537.36

Operating system: Ubuntu 16.04

Logs

Nextcloud log (data/nextcloud.log)

``` Info mail Updating draft <627204655> in account <1> 2017-10-05T09:14:55+0200 Info mail Updating draft <627204654> in account <1> 2017-10-05T09:14:50+0200 Info mail Updating draft <627204653> in account <1> 2017-10-05T09:14:38+0200 Info mail Updating draft <627204652> in account <1> 2017-10-05T09:13:43+0200 Info mail Updating draft <627204651> in account <1> 2017-10-05T09:13:05+0200 Info mail Updating draft <627204650> in account <1> 2017-10-05T09:12:53+0200 Info mail Updating draft <627204649> in account <1> 2017-10-05T09:12:49+0200 Info mail Updating draft <627204648> in account <1> 2017-10-05T09:11:16+0200 Info mail Updating draft <627204647> in account <1> 2017-10-05T09:11:11+0200 Info mail Updating draft <627204646> in account <1> 2017-10-05T09:10:49+0200 Info mail Updating draft <627204645> in account <1> 2017-10-05T09:10:42+0200 Info mail Updating draft <627204644> in account <1> 2017-10-05T09:10:10+0200 Info mail Updating draft <627204643> in account <1> 2017-10-05T09:08:52+0200 Info mail Updating draft <627204642> in account <1> 2017-10-05T09:07:38+0200 Info mail Updating draft <627204641> in account <1> 2017-10-05T09:07:19+0200 Info mail Updating draft <627204640> in account <1> 2017-10-05T09:06:21+0200 Info mail Updating draft <627204639> in account <1> 2017-10-05T09:05:59+0200 Info mail Updating draft <627204638> in account <1> 2017-10-05T09:05:57+0200 ```

Browser log

``` JQMIGRATE: Migrate is installed, version 1.4.0 init.js:25 Starting Mail … cache.js:47 initializing cache… js.storage.js:152 Uncaught (in promise) RangeError: Invalid string length at JSON.stringify () at Object._set (js.storage.js:152) at Object._callMethod (js.storage.js:438) at Object.set (js.storage.js:461) at Object.addMessage (cache.js:108) at messagecontroller.js:87 at DevTools failed to parse SourceMap: https://cloud.foxbow.de/core/vendor/purify.min.js.map ```

Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

The text was updated successfully, but these errors were encountered:

foxbow · 2017-10-06T07:21:26Z

When debugging the javascript in Chrome, I can see where it fails. In js.storage.js an array is being stored as an item in the "messages" set. Apparently this is the basic mail info with the headers, like title, cc, reply-to etc. This is converted via JSON.stringify and that one throws an uncaught exception: Uncaught (in promise) RangeError: Invalid string length

Unfortunately I have no clue which string that may be and where the actual source is to add a try/catch block around it for further investigation. Also the callstack is not very informative either. So if anyone could either tell me where to find the sources in question, the point in the mail, where this is triggered or even a 'better' JS debugger I would offer to investigate further - especially as it seems that I'm the only one being able to reproduce this so far.

foxbow · 2017-10-06T08:13:17Z

I found a pattern! Messages that are displayed properly have a low message ID, about 0 to 300. The failing messages have HUGE IDs, like 627173617 or 627182862 which kind of feels like an overflow issue. Fun thing is that these apparently worked just fine in Nextcloud 11.

ChristophWurst · 2017-10-06T09:36:30Z

I found a pattern! Messages that are displayed properly have a low message ID, about 0 to 300. The failing messages have HUGE IDs, like 627173617 or 627182862 which kind of feels like an overflow issue.

Wow, good finding.

Fun thing is that these apparently worked just fine in Nextcloud 11.

Well, maybe your UIDs were just lower back then 😉

christophKi · 2017-10-22T18:54:55Z

I don't know if it depends on the same issue but the behaviour is similar.

Steps to reproduce

Upgrade from Nextcloud 11 to 12
Upgrade mail plugin to 0.7.3
Try to check mails
Mailbox from IMAP: imap.de.aol.com with 177 mails

Expected behaviour

The selected mail is shown

Actual behaviour

The list of folders is shown, the mail titles of the current folder are shown, the selected mail is not shown, just the roller while one core stays busy.
The Opera javascript console claims one error.

But i can show mails from the mailboxes from the local dovecot IMAP mailboxes.

General server configuration

Operating system: Linux J6868 2.6.32-5-xen-686 #1 SMP Sun Jul 13 02:47:03 UTC 2014 i686
Kernel: 3.10.0-693.2.2.el7.x86_64
CentOS Linux release 7.4.1708 (Core)

Web server: Apache/2.4.6 (CentOS)

Database: mysqld 10.1.28-MariaDB

PHP version: PHP 7.1.10 (cli)

Nextcloud configuration

Nextcloud version: 12.0.3 - 12.0.3.3

Updated from an older Nextcloud/ownCloud or fresh install: 11.0.x

Where did you install Nextcloud from: tar

Are you using encryption: no

Client configuration

Browser: Opera 48.0.2685.39 & Firefox 56.0

Logs

Nextcloud log

Error	PHP	Narrowing occurred during type inference. Please file a bug report on bugs.php.net at /var/www/html/nextcloud/lib/composer/composer/ClassLoader.php#444	2017-10-22T20:00:41+0200
Error	PHP	Narrowing occurred during type inference. Please file a bug report on bugs.php.net at /var/www/html/nextcloud/lib/composer/composer/ClassLoader.php#444	2017-10-22T19:55:46+0200
Error	PHP	Narrowing occurred during type inference. Please file a bug report on bugs.php.net at /var/www/html/nextcloud/lib/composer/composer/ClassLoader.php#444	2017-10-22T18:57:55+0200
Info	updater	\OC\Updater::resetLogLevel: Reset log level to Warning(2)	2017-10-22T18:54:45+0200
Info	updater	\OC\Updater::maintenanceActive: Maintenance mode is kept active

Browser log

ChristophWurst · 2017-10-23T06:19:02Z

js.storage.js:152 Uncaught (in promise) DOMException: Failed to execute 'setItem' on 'Storage': Setting the value of 'messages' exceeded the quota.

I guess that could be the issue. We don't catch that.

foxbow · 2017-10-23T07:02:52Z

@christophKi If you have the chance, check the MessageId. In Firefox you would open the web console with F12 or CTRL-SHIFT-k and watch the list of resources getting loaded, not sure how to see that on Opera.

If the browser gets stuck on a resource like: https:///index.php/apps/mail/accounts/2/folders/AB1CD2EF/messages/123456789 then it's probably the same issue. Most interesting is the last number, which is the message ID. If that is ridiculously large you're seeing the same issue - or at least the same thing triggers your issue.

By the way, Opera shows the error at the same place as Chrome: js.storage.js:152 Uncaught (in promise) RangeError: Invalid string length at JSON.stringify () at Object._set (js.storage.js:152) [...] it's just packing the information differently, so it's most likely the same issue.

christophKi · 2017-10-23T12:30:33Z

Yes :) it's the same. Thx

This is from the Firefox log with a large message ID where it gets stuck:
index.php/apps/mail/accounts/1/folders/SU5CT1g%3D/messages/32627360

From an other mailbox where i can show the messages, with shorter ID:
index.php/apps/mail/accounts/4/folders/SU5CT1g=/messages/1329

Best regards

Ah, as much as I know, opera used the same engine as chrome.

foxbow · 2017-10-28T16:16:11Z

I believe that the message ID is broken, or at least the way it is interpreted. Stepping through the code in question I noticed that the app generates an array with MessageID entries to store the messages for the folder. When trying to add the message to the cache, the localStorage gets exhausted by trying to add an empty array with over 600.000.000 indices. Looking into the data, the mail body is there though, so it's not like the server returned random data.

I have not yet found the place where the MessageID comes from. It also does not seem to correlate with the number of messages in the folder. What I saw so far: 106 Messages in Folder - last message ID 95, 295 - 280, 192 - 627182270, 408 - 627198350

foxbow · 2017-10-29T15:12:48Z

I found a workaround. Since the Message ID is just used as a hash token for the cache and the numbers are consecutive, I decided to just truncate them down to 1024. This lets me read my messages again. However this will mess up the cache on folders with more than 1024 messages - which is why I'm reluctant to call it a real fix.

The proper way would be to align the MessageID with the actual number of messages in the folder. On the other hand I feel like this is 'good enough' as I'm not sure how many people will actually have folders with more than 1024 messages.

getMessagePath: function(account, folder, messageId) {
return [this.getFolderPath(account, folder), (messageId%1024).toString()].join('.');
}

@christophKi I'm not sure how much control you have over your installation, otherwise I would send you the build.js to replace your local version so you can at least give it a spin. That should be enough to have my patch - it's also built against the current release (0.7.3), not the head as that doesn't work for me either.

You could also just edit mail/js/build/build.js, find the getMessagePath() function and edit it there. Should be line 8576. Tested that by incrementing the modulo to 2028

foxbow · 2018-01-13T13:56:11Z

Still happens with 0.7.7... modulo workaround still works.

foxbow · 2018-01-26T08:20:23Z

One thought on this: to have a clean solution I believe it would make most sense to get the lowest MessageID of the current folder and take that as base offset value. However that would mean that getMessagePath() needs an easy way to retrieve that number. Also I'm not sure if 'getting the lowest MessageID' is a feasible task at all. Right now I'm using a Modulo of '10240' as a workaround which has not yet shown any issues and has no impact on performance.

Otherwise I'm curious if me and @christophKi really the only ones seeing this problem.

christophKi · 2018-01-28T14:41:29Z

Jup, today I updated to nextcloud 12.0.5 and mail 0.7.9. Sill the same problem.

greetings

foxbow changed the title ~~Mail is not displayed, JS runnign ready~~ Mail is not displayed, JS running ready Oct 5, 2017

ChristophWurst added bug 1. to develop labels Oct 5, 2017

ChristophWurst added the help wanted label Jan 13, 2018

ChristophWurst added 3. to review and removed 1. to develop help wanted labels Feb 1, 2018

ChristophWurst added this to the 0.7.10 milestone Feb 1, 2018

ChristophWurst self-assigned this Feb 1, 2018

ChristophWurst mentioned this issue Feb 1, 2018

Remove session storage cache #752

Merged

ChristophWurst closed this as completed in #752 Feb 1, 2018

foxbow mentioned this issue Apr 16, 2020

Cannot load mail anymore since upgrade to 1.3.0 #2909

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mail is not displayed, JS running ready #560

Mail is not displayed, JS running ready #560

foxbow commented Oct 5, 2017 •

edited by jospoortvliet

Loading

foxbow commented Oct 6, 2017

foxbow commented Oct 6, 2017

ChristophWurst commented Oct 6, 2017

christophKi commented Oct 22, 2017 •

edited

Loading

ChristophWurst commented Oct 23, 2017

foxbow commented Oct 23, 2017 •

edited

Loading

christophKi commented Oct 23, 2017 •

edited

Loading

foxbow commented Oct 28, 2017

foxbow commented Oct 29, 2017 •

edited

Loading

foxbow commented Jan 13, 2018 •

edited

Loading

foxbow commented Jan 26, 2018 •

edited

Loading

christophKi commented Jan 28, 2018

Mail is not displayed, JS running ready #560

Mail is not displayed, JS running ready #560

Comments

foxbow commented Oct 5, 2017 • edited by jospoortvliet Loading

Steps to reproduce

Expected behaviour

Actual behaviour

General server configuration

Nextcloud configuration

Client configuration

Logs

foxbow commented Oct 6, 2017

foxbow commented Oct 6, 2017

ChristophWurst commented Oct 6, 2017

christophKi commented Oct 22, 2017 • edited Loading

Steps to reproduce

Expected behaviour

Actual behaviour

General server configuration

Nextcloud configuration

Client configuration

Logs

ChristophWurst commented Oct 23, 2017

foxbow commented Oct 23, 2017 • edited Loading

christophKi commented Oct 23, 2017 • edited Loading

foxbow commented Oct 28, 2017

foxbow commented Oct 29, 2017 • edited Loading

foxbow commented Jan 13, 2018 • edited Loading

foxbow commented Jan 26, 2018 • edited Loading

christophKi commented Jan 28, 2018

foxbow commented Oct 5, 2017 •

edited by jospoortvliet

Loading

christophKi commented Oct 22, 2017 •

edited

Loading

foxbow commented Oct 23, 2017 •

edited

Loading

christophKi commented Oct 23, 2017 •

edited

Loading

foxbow commented Oct 29, 2017 •

edited

Loading

foxbow commented Jan 13, 2018 •

edited

Loading

foxbow commented Jan 26, 2018 •

edited

Loading