Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[NEW][Apps] Add support to the "encoding" option in http requests from Apps #19002

Merged
merged 3 commits into from
Sep 28, 2020

Conversation

shiqimei
Copy link
Contributor

@shiqimei shiqimei commented Sep 23, 2020

This PR allows apps to specify encoding option while performing an HTTP request.

In this way, apps can download binary data (Buffer) from an URL instead of utf8 string only.

Proposed changes

Forward encoding option into npmRequestOptions.

Issue(s)

https://open.rocket.chat/group/t9xgtnadgqwf3xkzh?msg=x5EsHiqtzv7rJj3qR (Private)

How to test or reproduce

import { IAppAccessors, IHttp, ILogger, IModify, IPersistence, IRead } from '@rocket.chat/apps-engine/definition/accessors';
import { App } from '@rocket.chat/apps-engine/definition/App';
import { ILivechatRoom } from '@rocket.chat/apps-engine/definition/livechat';
import { IMessage, IPostMessageSent } from '@rocket.chat/apps-engine/definition/messages';
import { IAppInfo } from '@rocket.chat/apps-engine/definition/metadata';
import { RoomType } from '@rocket.chat/apps-engine/definition/rooms';

export class FileUploadApp extends App implements IPostMessageSent {
    constructor(info: IAppInfo, logger: ILogger, accessors: IAppAccessors) {
        super(info, logger, accessors);
    }

    public async executePostMessageSent(message: IMessage, read: IRead, http: IHttp, persistence: IPersistence, modify: IModify): Promise<void> {
        try {
            if (!message.text || message.room.type !== RoomType.LIVE_CHAT) {
                return;
            }
            const agent = (message.room as ILivechatRoom).servedBy;
            const { content } = await http.get('https://ss1.bdstatic.com/70cFvXSh_Q1YnxGkpoWK1HF6hhy/it/u=3640940258,545464235&fm=26&gp=0.jpg', {
                encoding: null,
            });

            for (const filename of Array(1).fill(Math.random())) {
                await modify.getCreator().getUploadCreator().uploadBuffer(content as any as Buffer, {
                    filename: `${ filename } sent by ${ agent && agent.name }.png`,
                    room: message.room,
                    user: agent,
                });
            }
        } catch (err) {
            console.error(err);
        }
    }
}

fileupload_0.0.1.zip

Screenshots

image

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • Improvement (non-breaking change which improves a current function)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Hotfix (a major bugfix that has to be merged asap)
  • Documentation Update (if none of the other choices apply)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works (if applicable)
  • I have added necessary documentation (if applicable)
  • Any dependent changes have been merged and published in downstream modules

Changelog

Further comments

@shiqimei
Copy link
Contributor Author

shiqimei commented Sep 23, 2020

Depends on: RocketChat/Rocket.Chat.Apps-engine#321

@d-gubert d-gubert changed the title [Apps] [NEW] Allow to specify encoding option [NEW][Apps] Add support to the "encoding" option in http requests from Apps Sep 28, 2020
@d-gubert d-gubert merged commit 77f2b51 into develop Sep 28, 2020
@d-gubert d-gubert deleted the apps/add-encoding-option branch September 28, 2020 23:08
@sampaiodiego sampaiodiego mentioned this pull request Sep 29, 2020
ear-dev pushed a commit to WideChat/Rocket.Chat that referenced this pull request Oct 5, 2020
* Bump version to 3.7.0-develop

* Do not use deprecated express method (RocketChat#18686)

* Replace assets copy on postinstall with symlinks (RocketChat#18707)

* Update Meteor to 1.11 (RocketChat#18754)

* Update Meteor to 1.11

* Update Node version

* Fix livechat view creation

* LingoHub Update 🚀 (RocketChat#18761)

Manual push by LingoHub User: Diego Sampaio.
Project: Rocket.Chat

Made with ❤️ by https://lingohub.com

Co-authored-by: Diego Sampaio <chinello@gmail.com>

* Fix french translations (RocketChat#18746)

Replace [Nom du site] by [Site_Name]...

* Refactor: Omnichannel Realtime Monitoring (RocketChat#18666)

Co-authored-by: Guilherme Gazzo <guilherme@gazzo.xyz>

* [FIX] Create Custom OAuth services from environment variables (RocketChat#17377)

* Fix saveRoomSettings method complexity (RocketChat#18840)

* [FIX] e.sendToBottomIfNecessaryDebounced is not a function (RocketChat#18834)

* Log WebDav upload errors (RocketChat#18849)

* [FIX] File upload (Avatars, Emoji, Sounds) (RocketChat#18841)

* [FIX] IE11 support livechat widget  (RocketChat#18850)

* [FIX] Can't change password (RocketChat#18836)

* [FIX] Admin user blank page (RocketChat#18851)

* [FIX][ENTERPRISE] Omnichannel service status switching to unavailable (RocketChat#18835)

* [FIX] User can't invite or join other Omnichannel rooms (RocketChat#18852)

* [FIX] Omnichannel Current Chats open status filter not working (RocketChat#18795)

* [FIX] Jitsi call start updating subscriptions (RocketChat#18837)

* [FIX] Showing alerts during setup wizard (RocketChat#18862)

* Remember users' 2FA right after registration

* Change register server to false by default

* [FIX] Omnichannel Current Chats open status filter not working (RocketChat#18795)

* [FIX][ENTERPRISE] Omnichannel service status switching to unavailable (RocketChat#18835)

* [FIX] File upload (Avatars, Emoji, Sounds) (RocketChat#18841)

* [FIX] IE11 support livechat widget  (RocketChat#18850)

* [FIX] Admin user blank page (RocketChat#18851)

* [FIX] User can't invite or join other Omnichannel rooms (RocketChat#18852)

* [FIX] Showing alerts during setup wizard (RocketChat#18862)

* Remember users' 2FA right after registration

* Change register server to false by default

* Bump version to 3.6.1

* LingoHub based on develop (RocketChat#18828)

* LingoHub Update 🚀

Manual push by LingoHub User: Diego Sampaio.
Project: Rocket.Chat

Made with ❤️ by https://lingohub.com

* LingoHub Update 🚀

Manual push by LingoHub User: Diego Sampaio.
Project: Rocket.Chat

Made with ❤️ by https://lingohub.com

Co-authored-by: Diego Sampaio <chinello@gmail.com>

* [FIX] invite-all-from and invite-all-to commands don't work with multibyte room names (RocketChat#18919)

* [FIX] If there is `ufs` somewhere in url the request to api always returns 404 (RocketChat#18874)

* chore(packages): add fixed version of ufs

* test(e2e): test usernames with 'ufs'

* test(ufs-router): remove expect for header

* test: turn test callback synchronous

* test: fix async to use done

* chore(packages/meteor-jalik-ufs): add ufs package

* Revert "test: fix async to use done"

This reverts commit 6276e0b.

* Revert "test: turn test callback synchronous"

This reverts commit 2af11bb.

* Revert "test(ufs-router): remove expect for header"

This reverts commit 2c4eeb0.

* [FIX] "Save to WebDav" not working (RocketChat#18883)

* [FIX] Read receipts showing blank names and not marking messages as read (RocketChat#18918)

* [FIX] Non-upload requests being passed to UFS proxy middleware (RocketChat#18931)

* fix(ufs-proxy): correct routing pattern

* chore(ufs-proxy): add deprecation console warning

* [FIX] Version update check cron job (RocketChat#18916)

* [FIX] Ignore User action from user card (RocketChat#18866)

* [FIX] Custom fields required if minLength set and no text typed (RocketChat#18838)

* [FIX] Dutch: add translations for missing variables (RocketChat#18814)

* [FIX] French: Add missing __online__ var (RocketChat#18813)

* [FIX] Deactivate users that are the last owner of a room using REST API (RocketChat#18864)

* test: add e2e tests for REST API user deactivation

* fix(app): read confirmRelinquish from HTTP request

* chore(app): remove unnecessary console.log

* [FIX] Show custom fields of invalid type (RocketChat#18794)

* [FIX] Create Custom OAuth services from environment variables (RocketChat#17377)

* [FIX] Read receipts showing blank names and not marking messages as read (RocketChat#18918)

* [FIX] Version update check cron job (RocketChat#18916)

* [FIX] Ignore User action from user card (RocketChat#18866)

* [FIX] Deactivate users that are the last owner of a room using REST API (RocketChat#18864)

* test: add e2e tests for REST API user deactivation

* fix(app): read confirmRelinquish from HTTP request

* chore(app): remove unnecessary console.log

* [FIX] Show custom fields of invalid type (RocketChat#18794)

* Bump version to 3.6.2

* [FIX] Stop adding push messages to queue if push is disabled (RocketChat#18830)

* [NEW] "Room avatar changed" system messages (RocketChat#18839)

* [FIX] PDF not rendering (RocketChat#18956)

* [FIX] Spurious expert role in startup data (RocketChat#18667)

* [FIX] Open room after guest registration (RocketChat#18755)

* [FIX] Block user action (RocketChat#18950)

* Refactor: Omnichannel Analytics (RocketChat#18766)

* [FIX] Deactivated users show as offline (RocketChat#18767)

* [FIX] Reaction buttons not behaving properly (RocketChat#18832)

* Refactor: Message Audit page & Audit logs (RocketChat#18808)

* [FIX] "Download my data" popup showing HTML code. (RocketChat#18947)

* [IMPROVE] Move jump to message outside menu (RocketChat#18928)

* [FIX] Users not being able to activate/deactivate E2E in DMs (RocketChat#18943)

* Refactor: Admin permissions page (RocketChat#18932)

Co-authored-by: Guilherme Gazzo <guilhermegazzo@gmail.com>

* [IMPROVEMENT] Add "Allow_Save_Media_to_Gallery" setting to mobile (RocketChat#18875)

* Bump lodash.merge from 4.6.1 to 4.6.2 (RocketChat#18800)

* [FIX] Errors in LDAP avatar sync preventing login (RocketChat#18948)

* [NEW][Apps] Add support for new livechat guest's and room's events (RocketChat#18946)

* Add support for new livechat's guest and room events

* Update trigger calls

* Update Apps-Engine version

Co-authored-by: Douglas Gubert <douglas.gubert@gmail.com>

* [NEW][Apps] Add a Livechat API - setCustomFields (RocketChat#18912)

* Map livechatData to customFields

* Add livechatData field to apps converter

Co-authored-by: Thassio Victor <tvmcarvalho@gmail.com>
Co-authored-by: Douglas Gubert <douglas.gubert@gmail.com>

* [NEW] UploadFS respects $TMPDIR environment variable (RocketChat#17012)

* [IMPROVE] Stop re-sending push notifications rejected by the gateway (RocketChat#18608)

Co-authored-by: Diego Sampaio <chinello@gmail.com>

* [FIX] User Info: Email and name/username display (RocketChat#18976)

* Update Meteor to 1.11.1 (RocketChat#18959)

* [FIX] API call users.setStatus does not trigger status update of clients (RocketChat#18961)

* LingoHub Update 🚀 (RocketChat#18973)

Manual push by LingoHub User: Gabriel Engel.
Project: Rocket.Chat

Made with ❤️ by https://lingohub.com

Co-authored-by: Gabriel Engel <gabriel.engel@gmail.com>

* Bump marked from 0.6.3 to 0.7.0 (RocketChat#18801)

Bumps [marked](https://github.com/markedjs/marked) from 0.6.3 to 0.7.0.
- [Release notes](https://github.com/markedjs/marked/releases)
- [Commits](markedjs/marked@v0.6.3...v0.7.0)

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump Livechat widget (RocketChat#18977)

* [NEW] Option to require settings on wizard UI via ENV variables (RocketChat#18974)

Co-authored-by: Martin <martin.schoeler@rocket.chat>

* [NEW] Retention policy precision defined by a cron job expression (RocketChat#18975)

Co-authored-by: Diego Sampaio <chinello@gmail.com>

* Set some queries to prefer the secondary database (RocketChat#18887)

* Check i18n file for missing variables (RocketChat#18762)

* [FIX] Scrollbar mention ticks always rendering as white (RocketChat#18979)

* [FIX] Purged threads still show as unread (RocketChat#18944)

Co-authored-by: Diego Sampaio <chinello@gmail.com>
Co-authored-by: Rodrigo Nascimento <rodrigoknascimento@gmail.com>

* [NEW][Apps] Add a new upload API (RocketChat#18955)

Co-authored-by: Douglas Gubert <douglas.gubert@gmail.com>

* [NEW] Send E2E encrypted messages’ content on push notifications (RocketChat#18882)

* New: Use database change streams when available (RocketChat#18892)

Co-authored-by: Diego Sampaio <chinello@gmail.com>

* Bump version to 3.7.0-rc.0

* Regression: Handle MongoDB authentication issues (RocketChat#18993)

* [FIX] LDAP avatar upload (RocketChat#18994)

* Bump version to 3.7.0-rc.1

* [FIX] Federation issues (RocketChat#18978)

* fixed message sending, you should not filter the domains, specially filter by only the local origin/source

* Fixing callback registration

* Increased the rate limiting

* Obbey to settings properties (RocketChat#19020)

* [FIX] PDF not rendering (RocketChat#18956)

* [FIX] Errors in LDAP avatar sync preventing login (RocketChat#18948)

* [FIX] LDAP avatar upload (RocketChat#18994)

* [FIX] Federation issues (RocketChat#18978)

* fixed message sending, you should not filter the domains, specially filter by only the local origin/source

* Fixing callback registration

* Increased the rate limiting

* Obbey to settings properties (RocketChat#19020)

* Bump version to 3.7.0-rc.2

* Bump version to 3.6.3

* Create VIP Sponsors.md

* Regression: Fix login screen reactivity of external login providers (RocketChat#19033)

* [NEW][Apps] Add support to the "encoding" option in http requests from Apps (RocketChat#19002)

* Allow specify the encoding of the response data

* Update Apps-Engine version

Co-authored-by: Douglas Gubert <douglas.gubert@gmail.com>
Co-authored-by: Douglas Gubert <d-gubert@users.noreply.github.com>

* Regression: File upload via apps not working in some scenarios (RocketChat#18995)

* [FIX] [Apps] Fix app user doesn't has permission to upload files

* Fix errorClass [Error]: Forbidden [forbidden]

In `app/file-upload/lib/FileUploadBase.js`, we configured `UploadFS.config.defaultStorePermissions`, which validates `insert(userId, doc)` etc. But the parameter userId cann't be always obtained by this validation method correctly (sometimes it's undefined). Meteor use the Meteor.userId() as its fallback option. So we can wrap the original call with `Meteor.runAsUser` to solve issue.

* Add a new validator into canAccessRoom

canAccessRoom (`app/authorization/server/functions/canAccessRoom.js`) is an essential validator for Rocket.Chat to check whether some user has permissions to access some room. In this PR, we added a new validator that allows app users to access any room on a Rocket.Chat server even if it is not a member of the room.

*  An attempt to fix Meteor code must always run within a Fiber Error

Original Error: "Error: Meteor code must always run within a Fiber. Try wrapping callbacks that you pass to non-Meteor libraries with Meteor.bindEnvironment."

* Add support for uploading files by a livecaht visitor

* Support upload files with livechat visitors

* Reduce an unnecessary DB query - Users.findOneById

* Move the "bypass" out of canAccessRoom

Adding a bypass inside canAccessRoom can potentially allow apps to do stuff we're not prepared (yet)

* Update Apps-Engine version

* Some refactoring

* Fix a rateada

Co-authored-by: Douglas Gubert <douglas.gubert@gmail.com>

* [NEW] Apps-Engine v1.18.0 (RocketChat#19047)

* Regression: Elements select & multiSelect not rendered correctly in the App Settings (RocketChat#19005)

Co-authored-by: Douglas Gubert <d-gubert@users.noreply.github.com>

* Bump version to 3.7.0-rc.3

* Bump version to 3.7.0-rc.4

* Bump version to 3.7.0

* Update package-lock.json

* Fix import in Omnichannel Filter

* Remove duplicate function

Co-authored-by: Diego Sampaio <chinello@gmail.com>
Co-authored-by: Tasso Evangelista <tasso.evangelista@rocket.chat>
Co-authored-by: Gabriel Engel <gabriel.engel@gmail.com>
Co-authored-by: lsignac <lsignac@users.noreply.github.com>
Co-authored-by: gabriellsh <40830821+gabriellsh@users.noreply.github.com>
Co-authored-by: Guilherme Gazzo <guilherme@gazzo.xyz>
Co-authored-by: Maarten <me@maartendewaard.nl>
Co-authored-by: pierre-lehnen-rc <55164754+pierre-lehnen-rc@users.noreply.github.com>
Co-authored-by: Guilherme Gazzo <guilhermegazzo@gmail.com>
Co-authored-by: Renato Becker <renato.augusto.becker@gmail.com>
Co-authored-by: Felipe Parreira <felipeparreirafb@gmail.com>
Co-authored-by: William Reiske <wreiske@mieweb.com>
Co-authored-by: Karting06 <Karting06@users.noreply.github.com>
Co-authored-by: Gilles Meyer <gilles@gmind.net>
Co-authored-by: Martin Schoeler <martin.schoeler@rocket.chat>
Co-authored-by: Diego Mello <diegolmello@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Thassio Victor <tvmcarvalho@gmail.com>
Co-authored-by: Douglas Gubert <douglas.gubert@gmail.com>
Co-authored-by: Shiqi Mei <shiqi.mei@rocket.chat>
Co-authored-by: Dustin Skoracki <d-sko@users.noreply.github.com>
Co-authored-by: lingohub[bot] <69908207+lingohub[bot]@users.noreply.github.com>
Co-authored-by: Rodrigo Nascimento <rodrigoknascimento@gmail.com>
Co-authored-by: Alan Sikora <alansikora@gmail.com>
Co-authored-by: Marcelo Schmidt <marcelo.schmidt@rocket.chat>
Co-authored-by: Douglas Gubert <d-gubert@users.noreply.github.com>
Co-authored-by: Shailesh Baldaniya <shaileshbaldaniya351@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants