Skip to content

Commit

Permalink
Add support for incoming emails (#22056)
Browse files Browse the repository at this point in the history
closes #13585
fixes #9067
fixes #2386
ref #6226
ref #6219
fixes #745

This PR adds support to process incoming emails to perform actions.
Currently I added handling of replies and unsubscribing from
issues/pulls. In contrast to #13585 the IMAP IDLE command is used
instead of polling which results (in my opinion 😉) in cleaner code.

Procedure:
- When sending an issue/pull reply email, a token is generated which is
present in the Reply-To and References header.
- IMAP IDLE waits until a new email arrives
- The token tells which action should be performed

A possible signature and/or reply gets stripped from the content.

I added a new service to the drone pipeline to test the receiving of
incoming mails. If we keep this in, we may test our outgoing emails too
in future.

Co-authored-by: silverwind <me@silverwind.io>
Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com>
  • Loading branch information
3 people authored Jan 14, 2023
1 parent 20e3ffd commit fc037b4
Show file tree
Hide file tree
Showing 26 changed files with 1,525 additions and 39 deletions.
4 changes: 4 additions & 0 deletions .drone.yml
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,10 @@ services:
MINIO_ACCESS_KEY: 123456
MINIO_SECRET_KEY: 12345678

- name: smtpimap
image: tabascoterrier/docker-imap-devel:latest
pull: always

steps:
- name: fetch-tags
image: docker:git
Expand Down
25 changes: 25 additions & 0 deletions assets/go-licenses.json

Large diffs are not rendered by default.

41 changes: 41 additions & 0 deletions custom/conf/app.example.ini
Original file line number Diff line number Diff line change
Expand Up @@ -1664,6 +1664,47 @@ ROUTER = console
;; convert \r\n to \n for Sendmail
;SENDMAIL_CONVERT_CRLF = true

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;[email.incoming]
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;
;; Enable handling of incoming emails.
;ENABLED = false
;;
;; The email address including the %{token} placeholder that will be replaced per user/action.
;; Example: incoming+%{token}@example.com
;; The placeholder must appear in the user part of the address (before the @).
;REPLY_TO_ADDRESS =
;;
;; IMAP server host
;HOST =
;;
;; IMAP server port
;PORT =
;;
;; Username of the receiving account
;USERNAME =
;;
;; Password of the receiving account
;PASSWORD =
;;
;; Whether the IMAP server uses TLS.
;USE_TLS = false
;;
;; If set to true, completely ignores server certificate validation errors. This option is unsafe.
;SKIP_TLS_VERIFY = true
;;
;; The mailbox name where incoming mail will end up.
;MAILBOX = INBOX
;;
;; Whether handled messages should be deleted from the mailbox.
;DELETE_HANDLED_MESSAGE = true
;;
;; Maximum size of a message to handle. Bigger messages are ignored. Set to 0 to allow every size.
;MAXIMUM_MESSAGE_SIZE = 10485760

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
;[cache]
Expand Down
14 changes: 14 additions & 0 deletions docs/content/doc/advanced/config-cheat-sheet.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -750,6 +750,20 @@ and
- `SEND_BUFFER_LEN`: **100**: Buffer length of mailing queue. **DEPRECATED** use `LENGTH` in `[queue.mailer]`
- `SEND_AS_PLAIN_TEXT`: **false**: Send mails only in plain text, without HTML alternative.

## Incoming Email (`email.incoming`)

- `ENABLED`: **false**: Enable handling of incoming emails.
- `REPLY_TO_ADDRESS`: **\<empty\>**: The email address including the `%{token}` placeholder that will be replaced per user/action. Example: `incoming+%{token}@example.com`. The placeholder must appear in the user part of the address (before the `@`).
- `HOST`: **\<empty\>**: IMAP server host.
- `PORT`: **\<empty\>**: IMAP server port.
- `USERNAME`: **\<empty\>**: Username of the receiving account.
- `PASSWORD`: **\<empty\>**: Password of the receiving account.
- `USE_TLS`: **false**: Whether the IMAP server uses TLS.
- `SKIP_TLS_VERIFY`: **false**: If set to `true`, completely ignores server certificate validation errors. This option is unsafe.
- `MAILBOX`: **INBOX**: The mailbox name where incoming mail will end up.
- `DELETE_HANDLED_MESSAGE`: **true**: Whether handled messages should be deleted from the mailbox.
- `MAXIMUM_MESSAGE_SIZE`: **10485760**: Maximum size of a message to handle. Bigger messages are ignored. Set to 0 to allow every size.

## Cache (`cache`)

- `ENABLED`: **true**: Enable the cache.
Expand Down
2 changes: 1 addition & 1 deletion docs/content/doc/features/comparison.en-us.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ _Symbols used in table:_
| Issue search ||||||||
| Global issue search | [/](https://github.com/go-gitea/gitea/issues/2434) |||||||
| Issue dependency ||||||||
| Create issue via email | [](https://github.com/go-gitea/gitea/issues/6226) ||| ||||
| Create issue via email | [](https://github.com/go-gitea/gitea/issues/6226) ||| ||||
| Service Desk | [](https://github.com/go-gitea/gitea/issues/6219) |||||||

## Pull/Merge requests
Expand Down
2 changes: 1 addition & 1 deletion docs/content/doc/features/comparison.zh-cn.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ _表格中的符号含义:_
| 工单搜索 ||||||||
| 工单全局搜索 | [](https://github.com/go-gitea/gitea/issues/2434) |||||||
| 工单依赖关系 ||||||||
| 通过 Email 创建工单 | [](https://github.com/go-gitea/gitea/issues/6226) | [](https://github.com/gogs/gogs/issues/2602) || ||||
| 通过 Email 创建工单 | [](https://github.com/go-gitea/gitea/issues/6226) | [](https://github.com/gogs/gogs/issues/2602) || ||||
| 服务台 | [](https://github.com/go-gitea/gitea/issues/6219) ||| [](https://gitlab.com/groups/gitlab-org/-/epics/3103) ||||

#### Pull/Merge requests
Expand Down
2 changes: 1 addition & 1 deletion docs/content/doc/features/comparison.zh-tw.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ menu:
| 問題搜尋 ||||||||
| 全域問題搜尋 | [](https://github.com/go-gitea/gitea/issues/2434) |||||||
| 問題相依 ||||||||
| 從電子郵件建立問題 | [](https://github.com/go-gitea/gitea/issues/6226) | [](https://github.com/gogs/gogs/issues/2602) || ||||
| 從電子郵件建立問題 | [](https://github.com/go-gitea/gitea/issues/6226) | [](https://github.com/gogs/gogs/issues/2602) || ||||
| 服務台 | [](https://github.com/go-gitea/gitea/issues/6219) ||| [](https://gitlab.com/groups/gitlab-org/-/epics/3103) ||||

## 拉取/合併請求
Expand Down
47 changes: 47 additions & 0 deletions docs/content/doc/usage/incoming-email.en-us.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
---
date: "2022-12-01T00:00:00+00:00"
title: "Incoming Email"
slug: "incoming-email"
draft: false
toc: false
menu:
sidebar:
parent: "usage"
name: "Incoming Email"
weight: 13
identifier: "incoming-email"
---

# Incoming Email

Gitea supports the execution of several actions through incoming mails. This page describes how to set this up.

**Table of Contents**

{{< toc >}}

## Requirements

Handling incoming email messages requires an IMAP-enabled email account.
The recommended strategy is to use [email sub-addressing](https://en.wikipedia.org/wiki/Email_address#Sub-addressing) but a catch-all mailbox does work too.
The receiving email address contains a user/action specific token which tells Gitea which action should be performed.
This token is expected in the `To` and `Delivered-To` header fields.

Gitea tries to detect automatic responses to skip and the email server should be configured to reduce the incoming noise too (spam, newsletter).

## Configuration

To activate the handling of incoming email messages you have to configure the `email.incoming` section in the configuration file.

The `REPLY_TO_ADDRESS` contains the address an email client will respond to.
This address needs to contain the `%{token}` placeholder which will be replaced with a token describing the user/action.
This placeholder must only appear once in the address and must be in the user part of the address (before the `@`).

An example using email sub-addressing may look like this: `incoming+%{token}@example.com`

If a catch-all mailbox is used, the placeholder may be used anywhere in the user part of the address: `incoming+%{token}@example.com`, `incoming_%{token}@example.com`, `%{token}@example.com`

## Security

Be careful when choosing the domain used for receiving incoming email.
It's recommended receiving incoming email on a subdomain, such as `incoming.example.com` to prevent potential security problems with other services running on `example.com`.
7 changes: 6 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -20,11 +20,13 @@ require (
github.com/buildkite/terminal-to-html/v3 v3.7.0
github.com/caddyserver/certmagic v0.17.2
github.com/chi-middleware/proxy v1.1.1
github.com/denisenkom/go-mssqldb v0.12.3
github.com/denisenkom/go-mssqldb v0.12.2
github.com/dimiro1/reply v0.0.0-20200315094148-d0136a4c9e21
github.com/djherbis/buffer v1.2.0
github.com/djherbis/nio/v3 v3.0.1
github.com/dustin/go-humanize v1.0.0
github.com/editorconfig/editorconfig-core-go/v2 v2.5.1
github.com/emersion/go-imap v1.2.1
github.com/emirpasic/gods v1.18.1
github.com/ethantkoenig/rupture v1.0.1
github.com/felixge/fgprof v0.9.3
Expand Down Expand Up @@ -58,6 +60,7 @@ require (
github.com/hashicorp/golang-lru v0.6.0
github.com/huandu/xstrings v1.4.0
github.com/jaytaylor/html2text v0.0.0-20211105163654-bc68cce691ba
github.com/jhillyerd/enmime v0.10.1
github.com/json-iterator/go v1.1.12
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51
github.com/keybase/go-crypto v0.0.0-20200123153347-de78d2cb44f4
Expand Down Expand Up @@ -145,6 +148,7 @@ require (
github.com/blevesearch/zapx/v15 v15.3.8 // indirect
github.com/boombuler/barcode v1.0.1 // indirect
github.com/bradfitz/gomemcache v0.0.0-20190913173617-a41fca850d0b // indirect
github.com/cention-sany/utf7 v0.0.0-20170124080048-26cad61bd60a // indirect
github.com/cespare/xxhash/v2 v2.1.2 // indirect
github.com/cloudflare/circl v1.2.0 // indirect
github.com/couchbase/go-couchbase v0.0.0-20210224140812-5740cd35f448 // indirect
Expand All @@ -155,6 +159,7 @@ require (
github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f // indirect
github.com/dlclark/regexp2 v1.7.0 // indirect
github.com/dsnet/compress v0.0.2-0.20210315054119-f66993602bf5 // indirect
github.com/emersion/go-sasl v0.0.0-20200509203442-7bfe0ed36a21 // indirect
github.com/felixge/httpsnoop v1.0.3 // indirect
github.com/fxamacker/cbor/v2 v2.4.0 // indirect
github.com/go-ap/errors v0.0.0-20221205040414-01c1adfc98ea // indirect
Expand Down
Loading

0 comments on commit fc037b4

Please sign in to comment.