External links blocked by modsecurity #790

Patta · 2018-02-19T15:53:05Z

Steps to reproduce

Set web application firewall mode (ModSecurity) to "On"
Set rule set to "Atomic Basic ModSecurity"
Set predefined set of values in configuration to "Fast"
Try to open a link in an HTML email

Expected behaviour

The redirectpage should be opened

Actual behaviour

A internal errorpage is shown

Mail app

Mail app version: 0.7.10

Mailserver or service: same server (dovecot imap)

Number of accounts: 1

Server configuration

Operating system: Linux 3.16.0-042stab125.3 #1 SMP Wed Sep 27 19:27:11 MSK 2017 x86_64

Web server: Apache (fpm-fcgi)

Database: mysql 10.0.32

PHP version: 7.2.2
Modules loaded: Core, date, libxml, openssl, pcre, zlib, bz2, calendar, ctype, hash, filter, ftp, gettext, gmp, SPL, iconv, Reflection, session, standard, SimpleXML, sockets, mbstring, tokenizer, xml, cgi-fcgi, mysqlnd, bcmath, curl, dba, dom, enchant, fileinfo, gd, imagick, imap, intl, json, ldap, exif, mysqli, odbc, PDO, pdo_mysql, PDO_ODBC, pdo_pgsql, pdo_sqlite, pgsql, Phar, posix, pspell, redis, soap, sqlite3, sysvmsg, sysvsem, sysvshm, tidy, xmlreader, xmlrpc, xmlwriter, xsl, zip, Zend OPcache

Version: 13.0.0 - 13.0.0.14

Updated from an older version or fresh install: updated from owncloud last year to maybe version 10 of nextcloud (not sure anymore)

Where did you install Nextcloud from: nextcloud.com (Nextcloud Server)

List of activated apps:

Enabled:

bruteforcesettings: 1.0.3
calendar: 1.6.0
comments: 1.3.0
contacts: 2.1.0
dav: 1.4.6
encryption: 2.0.0
federatedfilesharing: 1.3.1
files: 1.8.0
files_pdfviewer: 1.2.0
files_sharing: 1.5.0
files_texteditor: 2.5.1
files_trashbin: 1.3.0
files_versions: 1.6.0
files_videoplayer: 1.2.0
firstrunwizard: 2.2.1
gallery: 18.0.0
issuetemplate: 0.3.0
logreader: 2.0.0
lookup_server_connector: 1.1.0
mail: 0.7.10
news: 12.0.1
nextcloud_announcements: 1.2.0
notes: 2.3.2
notifications: 2.1.2
oauth2: 1.1.0
ownbackup: 17.5.0
password_policy: 1.3.0
provisioning_api: 1.3.0
serverinfo: 1.3.0
sharebymail: 1.3.0
survey_client: 1.1.0
systemtags: 1.3.0
tasks: 0.9.6
theming: 1.4.1
twofactor_backupcodes: 1.2.3
twofactor_totp: 1.4.1
updatenotification: 1.3.0
workflowengine: 1.3.0

Disabled:

activity
admin_audit
deck
end_to_end_encryption
federation
files_external
files_markdown
user_external
user_ldap

The content of config/config.php:

{
    "instanceid": "***REMOVED SENSITIVE VALUE***",
    "passwordsalt": "***REMOVED SENSITIVE VALUE***",
    "secret": "***REMOVED SENSITIVE VALUE***",
    "trusted_domains": [
        "***REMOVED SENSITIVE VALUE***"
    ],
    "datadirectory": "***REMOVED SENSITIVE VALUE***",
    "overwrite.cli.url": "https:\/\/***REMOVED SENSITIVE VALUE***",
    "dbtype": "mysql",
    "version": "13.0.0.14",
    "dbname": "***REMOVED SENSITIVE VALUE***",
    "dbhost": "***REMOVED SENSITIVE VALUE***",
    "dbtableprefix": "oc_",
    "dbuser": "***REMOVED SENSITIVE VALUE***",
    "dbpassword": "***REMOVED SENSITIVE VALUE***",
    "logtimezone": "UTC",
    "installed": true,
    "mail_from_address": "***REMOVED SENSITIVE VALUE***",
    "mail_smtpmode": "php",
    "mail_domain": "***REMOVED SENSITIVE VALUE***",
    "loglevel": 4,
    "appstore.experimental.enabled": true,
    "maintenance": false,
    "theme": "",
    "mysql.utf8mb4": true
}

Are you using external storage, if yes which one: no

Are you using encryption: yes

Are you using an external user-backend, if yes which one: no

Client configuration

Browser: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.167 Safari/537.36

Operating system: Windows 10 Pro 1709 (Build 16299.248)

Logs

Web server error log

[Tue Feb 20 09:23:41.426162 2018] [:error] [pid 31900] [client ***REMOVED SENSITIVE VALUE***] ModSecurity: Access denied with code 403 (phase 2). Match of "beginsWith %{request_headers.host}" against "TX:1" required. [file "/etc/apache2/modsecurity.d/rules/tortix/modsec/50_plesk_basic_asl_rules.conf"] [line "179"] [id "340162"] [rev "294"] [msg "Protected by Atomicorp.com Basic Non-Realtime WAF Rules: URL detected as argument, possible RFI attempt detected"] [data "%TX:1,TX:1"] [severity "CRITICAL"] [hostname "***REMOVED SENSITIVE VALUE***"] [uri "/index.php/apps/mail/redirect"] [unique_id "WovbDVkWZHUAAHycASoAAAAD"]

Server log (data/nextcloud.log)

Nexcloud doesn't log errors in to the file in debug mode (i don't no why). Here is the message from the errorpage, after clicked the link.

Interner Serverfehler
Der Server konnte die Anfrage nicht fertig stellen.

Sollte dies erneut auftreten, sende bitte die nachfolgenden technischen Einzelheiten an Deinen Server-Administrator.

Weitere Details können im Server-Protokoll gefunden werden.

Technische Details
Entfernte Adresse: ***REMOVED SENSITIVE VALUE***
Anfragekennung: WoveDFkWZHUAAHyksiIAAAAE
Typ: Exception
Code: 1
Nachricht: URL is not valid.
Datei: /var/www/vhosts/***REMOVED SENSITIVE VALUE***/nextcloud/apps/mail/lib/Controller/ProxyController.php
Zeile: 87

Trace
#0 [internal function]: OCA\Mail\Controller\ProxyController->redirect(NULL)
#1 /var/www/vhosts/***REMOVED SENSITIVE VALUE***/nextcloud/lib/private/AppFramework/Http/Dispatcher.php(161): call_user_func_array(Array, Array)
#2 /var/www/vhosts/***REMOVED SENSITIVE VALUE***/nextcloud/lib/private/AppFramework/Http/Dispatcher.php(91): OC\AppFramework\Http\Dispatcher->executeController(Object(OCA\Mail\Controller\ProxyController), 'redirect')
#3 /var/www/vhosts/***REMOVED SENSITIVE VALUE***/nextcloud/lib/private/AppFramework/App.php(115): OC\AppFramework\Http\Dispatcher->dispatch(Object(OCA\Mail\Controller\ProxyController), 'redirect')
#4 /var/www/vhosts/***REMOVED SENSITIVE VALUE***/nextcloud/lib/private/AppFramework/Routing/RouteActionHandler.php(47): OC\AppFramework\App::main('OCA\\Mail\\Contro...', 'redirect', Object(OC\AppFramework\DependencyInjection\DIContainer), Array)
#5 [internal function]: OC\AppFramework\Routing\RouteActionHandler->__invoke(Array)
#6 /var/www/vhosts/***REMOVED SENSITIVE VALUE***/nextcloud/lib/private/Route/Router.php(297): call_user_func(Object(OC\AppFramework\Routing\RouteActionHandler), Array)
#7 /var/www/vhosts/***REMOVED SENSITIVE VALUE***/nextcloud/lib/base.php(998): OC\Route\Router->match('/apps/mail/redi...')
#8 /var/www/vhosts/***REMOVED SENSITIVE VALUE***/nextcloud/index.php(37): OC::handleRequest()
#9 {main}

Browser log

javascript console log

jquery-migrate.min.js:2 JQMIGRATE: Migrate is installed, version 1.4.0
shareconfigmodel.js:24 Uncaught ReferenceError: oc_appconfig is not defined
    at shareconfigmodel.js:24
    at shareconfigmodel.js:80
js.js:202 Uncaught TypeError: Cannot read property 'substring' of undefined
    at Object.filePath (js.js:202)
    at viewer.js:15
DevTools failed to parse SourceMap: https://***REMOVED SENSITIVE VALUE***/core/vendor/blueimp-md5/js/md5.min.js.map
DevTools failed to parse SourceMap: https://***REMOVED SENSITIVE VALUE***/core/vendor/DOMPurify/dist/purify.min.js.map

The text was updated successfully, but these errors were encountered:

ChristophWurst · 2018-02-19T16:00:25Z

Hi!

Thank you for your report. It looks like your report is missing some important sections of your issue template. Please complete it so that we get a better understanding of your setup and the problem to be able to fix the issue. It's okay to omit certain section where it's obvious that they are irrelevant, but please don't simply ignore almost the full template.

Thank you.

Patta · 2018-02-20T09:03:21Z

@ChristophWurst Sorry, issue is updated.

ChristophWurst · 2018-02-20T09:11:39Z

Nachricht: URL is not valid.

Which means this line is triggered

mail/lib/Controller/ProxyController.php

Line 87 in b0569a3

throw new Exception('URL is not valid.', 1);

Could you please share the URL (you can remove the domain) of the page that shows the error? I'd be interested in the redirection URL and if it specifies a protocol.

Patta · 2018-02-20T09:17:37Z

It's for example the "view it on GitHub" link in github mails like https://REMOVED/index.php/apps/mail/redirect?src=https%3A%2F%2Fgithub.com%2Fnextcloud%2Fmail%2Fissues%2F790 but also on all other links.

ChristophWurst · 2018-02-20T09:40:45Z

I'm not familiar with modsecurity. Does it rewrite/change the URL?

The requested URL looks good.

ChristophWurst · 2018-02-20T09:44:25Z

For debugging purposes it would help to know the value of $src in the method head

mail/lib/Controller/ProxyController.php

Line 81 in b0569a3

public function redirect($src) {

. Maybe you can find that out with a error_log statement and checking the php error logs.

Patta · 2018-02-20T09:48:49Z

I'm also not familiar with modsecurity. I only can provide the description from the plesk UI.

Mode: On

Each incoming HTTP request and the related response are checked against a set of rules. If the check succeeds, the HTTP request is passed to web site content. If the check fails, the event is logged, a notification is sent, and the HTTP response is provided with an error code.

Rule set: Atomic Basic ModSecurity

A starter version of the Atomic ModSecurity rules. Provides basic web application firewall functionality. Updated on a monthly basis.

Configuration: Fast

The HTTP request URI and parts of headers will be analyzed.

"Maybe you can find that out with a error_log statement and checking the php error logs."
I will look at this soon.

Patta · 2018-02-20T10:40:23Z

It seems that $src is empty. I adopt line 87 to:
throw new Exception("URL is not valid. Value of \$src = $src", 1);

The error message is:

Typ: Exception
Code: 1
Nachricht: URL is not valid. Value of $src =
Datei: /var/www/vhosts/REMOVED/nextcloud/apps/mail/lib/Controller/ProxyController.php
Zeile: 87

Now it is clear that modsecurity rule ID 340162 removes the value from $src. I testet it without that rule and the src is not empty.

ChristophWurst · 2018-02-20T11:20:31Z

Okay, great that you could verify that. Does that mean this issue is resolved? I don't think this app can do anything about this if an apache module mangles the request.

Patta · 2018-02-20T11:26:52Z

Maybe a solution/info in the documentation would be great.

ChristophWurst · 2018-02-20T11:31:58Z

Maybe a solution/info in the documentation would be great.

Please file a ticket here: https://github.com/nextcloud/documentation to discuss that. Thank you!

ChristophWurst added the needs info label Feb 19, 2018

ChristophWurst added bug 0. to triage labels Feb 20, 2018

ChristophWurst closed this as completed Feb 20, 2018

ChristophWurst added invalid and removed 0. to triage bug needs info labels Feb 20, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

External links blocked by modsecurity #790

External links blocked by modsecurity #790

Patta commented Feb 19, 2018 •

edited

Loading

ChristophWurst commented Feb 19, 2018

Patta commented Feb 20, 2018 •

edited

Loading

ChristophWurst commented Feb 20, 2018

Patta commented Feb 20, 2018

ChristophWurst commented Feb 20, 2018

ChristophWurst commented Feb 20, 2018

Patta commented Feb 20, 2018

Patta commented Feb 20, 2018 •

edited

Loading

ChristophWurst commented Feb 20, 2018

Patta commented Feb 20, 2018

ChristophWurst commented Feb 20, 2018

External links blocked by modsecurity #790

External links blocked by modsecurity #790

Comments

Patta commented Feb 19, 2018 • edited Loading

Steps to reproduce

Expected behaviour

Actual behaviour

Mail app

Server configuration

Client configuration

Logs

Web server error log

Server log (data/nextcloud.log)

Browser log

ChristophWurst commented Feb 19, 2018

Patta commented Feb 20, 2018 • edited Loading

ChristophWurst commented Feb 20, 2018

Patta commented Feb 20, 2018

ChristophWurst commented Feb 20, 2018

ChristophWurst commented Feb 20, 2018

Patta commented Feb 20, 2018

Patta commented Feb 20, 2018 • edited Loading

ChristophWurst commented Feb 20, 2018

Patta commented Feb 20, 2018

ChristophWurst commented Feb 20, 2018

Patta commented Feb 19, 2018 •

edited

Loading

Patta commented Feb 20, 2018 •

edited

Loading

Patta commented Feb 20, 2018 •

edited

Loading