#579 Add Slack rate-limit handling

[Helcaraxan]

Extended fix for #579 that covers all places where we could encounter Slack's rate-limiting.

[codeclimate]

Code Climate has analyzed commit 0d52090 and detected 0 issues on this pull request.

View more on Code Climate.

[42wim]

Thanks! Looks good!
Too bad of the repeated code with the handleRateLimit, but I can't come up with something to fix this besides using interface{} and reflect. (and that's something to avoid as much as possible :)

[TheHolyRoger]

I don't know if the slack API has changed but i'm getting these errors in the console:

[2019-11-07T11:52:16Z] ERROR slack: Connection failed "slack rate limit exceeded, retry after 59s" &slack.RateLimitedError{RetryAfter:59000000000}
[2019-11-07T11:52:16Z] ERROR slack: Connection failed "slack rate limit exceeded, retry after 59s" &slack.RateLimitedError{RetryAfter:59000000000}
[2019-11-07T12:07:17Z] ERROR slack: Connection failed "slack rate limit exceeded, retry after 59s" &slack.RateLimitedError{RetryAfter:59000000000}
[2019-11-07T12:07:17Z] ERROR slack: Connection failed "slack rate limit exceeded, retry after 59s" &slack.RateLimitedError{RetryAfter:59000000000}
@Helcaraxan

[qaisjp]

I'm getting rate limit messages too

[42wim]

Related issue: https://github.com/nlopes/slack/issues/613
I'll rebase my fork with current slack master

[42wim]

In master now, would be nice if someone could test it.

[qaisjp]

Running in production. I'll let you know if I run into any issues.

[qaisjp]

Running on f43faf1

Debug output

[0093] DEBUG discord:      == Receiving event &discordgo.Message{ID:"653195438391820288", ChannelID:"590172620440469505", GuildID:"278474088903606273", Content:"test", Timestamp:"2019-12-08T11:25:28.297000+00:00", EditedTimestamp:"", MentionRoles:[]string{}, Tts:false, MentionEveryone:false, Author:(*discordgo.User)(0xc00062b300), Attachments:[]*discordgo.MessageAttachment{}, Embeds:[]*discordgo.MessageEmbed{}, Mentions:[]*discordgo.User{}, Reactions:[]*discordgo.MessageReactions(nil), Pinned:false, Type:0, WebhookID:"", Member:(*discordgo.Member)(0xc000567ab0), MentionChannels:[]*discordgo.Channel(nil), Activity:(*discordgo.MessageActivity)(nil), Application:(*discordgo.MessageApplication)(nil), MessageReference:(*discordgo.MessageReference)(nil), Flags:0}
[0093] DEBUG discord:      <= Sending message from qaisjp on discord.mta to gateway
[0093] DEBUG discord:      <= Message is config.Message{Text:"test", Channel:"test", Username:"qaisjp", UserID:"83386293446246400", Avatar:"https://cdn.discordapp.com/avatars/83386293446246400/6c4342d5d3556d4d6993232272eb9699.jpg", Account:"discord.mta", Event:"", Protocol:"", Gateway:"", ParentID:"", Timestamp:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}, ID:"653195438391820288", Extra:map[string][]interface {}(nil)}
[0093] DEBUG gateway:      => Sending config.Message{Text:"test", Channel:"test", Username:"qaisjp", UserID:"83386293446246400", Avatar:"https://cdn.discordapp.com/avatars/83386293446246400/6c4342d5d3556d4d6993232272eb9699.jpg", Account:"discord.mta", Event:"", Protocol:"", Gateway:"mta.test", ParentID:"", Timestamp:time.Time{wall:0xbf73554a15877254, ext:93167846295, loc:(*time.Location)(0x1ce7500)}, ID:"653195438391820288", Extra:map[string][]interface {}(nil)} from discord.mta (test) to slack.mta (test)
[0093] DEBUG slack:        => Receiving config.Message{Text:"test", Channel:"test", Username:"qaisjp", UserID:"83386293446246400", Avatar:"https://cdn.discordapp.com/avatars/83386293446246400/6c4342d5d3556d4d6993232272eb9699.jpg", Account:"discord.mta", Event:"", Protocol:"", Gateway:"mta.test", ParentID:"", Timestamp:time.Time{wall:0xbf73554a15877254, ext:93167846295, loc:(*time.Location)(0x1ce7500)}, ID:"", Extra:map[string][]interface {}(nil)}
[0093] DEBUG slack:        == Receiving event &slack.MessageEvent{Msg:slack.Msg{Type:"message", Channel:"C4Z7JEUC8", User:"", Text:"test", Timestamp:"1575804328.002500", ThreadTimestamp:"", IsStarred:false, PinnedTo:[]string(nil), Attachments:[]slack.Attachment(nil), Edited:(*slack.Edited)(nil), LastRead:"", Subscribed:false, UnreadCount:0, SubType:"bot_message", Hidden:false, DeletedTimestamp:"", EventTimestamp:"1575804328.002500", BotID:"BF5S2HSTG", Username:"qaisjp", Icons:(*slack.Icon)(0xc00059aa20), Inviter:"", Topic:"", Purpose:"", Name:"", OldName:"", Members:[]string(nil), ReplyCount:0, Replies:[]slack.Reply(nil), ParentUserId:"", Files:[]slack.File(nil), Upload:false, Comment:(*slack.Comment)(nil), ItemType:"", ReplyTo:0, Team:"T216B1U03", Reactions:[]slack.ItemReaction(nil), ResponseType:"", ReplaceOriginal:false, DeleteOriginal:false, Blocks:slack.Blocks{BlockSet:[]slack.Block(nil)}}, SubMessage:(*slack.Msg)(nil), PreviousMessage:(*slack.Msg)(nil)}
[0093] DEBUG gateway:      mID slack.mta: 1575804328.002500
[0093] DEBUG slack:        Found bot &slack.Bot{ID:"BF5S2HSTG", Name:"Chat Bridge", Deleted:false, UserID:"UF3N0SGJU", AppID:"AF3RWLXBK", Updated:1546356234, Icons:slack.Icons{Image36:"https://a.slack-edge.com/80588/img/plugins/app/bot_36.png", Image48:"https://a.slack-edge.com/80588/img/plugins/app/bot_48.png", Image72:"https://a.slack-edge.com/80588/img/plugins/app/service_72.png"}}
[0093] DEBUG slack:        <= Sending message from qaisjp on slack.mta to gateway
[0093] DEBUG slack:        GetUserInfo failed for BF5S2HSTG: user_not_found
[0093] DEBUG slack:        <= Message is &config.Message{Text:"test", Channel:"test", Username:"qaisjp", UserID:"BF5S2HSTG", Avatar:"", Account:"slack.mta", Event:"", Protocol:"slack", Gateway:"", ParentID:"", Timestamp:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}, ID:"1575804328.002500", Extra:map[string][]interface {}{}}
[0093] DEBUG gateway:      => Sending config.Message{Text:"test", Channel:"test", Username:"qaisjp", UserID:"BF5S2HSTG", Avatar:"", Account:"slack.mta", Event:"", Protocol:"slack", Gateway:"mta.test", ParentID:"", Timestamp:time.Time{wall:0xbf73554a286d2fa3, ext:93484892412, loc:(*time.Location)(0x1ce7500)}, ID:"1575804328.002500", Extra:map[string][]interface {}{}} from slack.mta (test) to discord.mta (test)
[0093] DEBUG discord:      => Receiving config.Message{Text:"test", Channel:"test", Username:"qaisjp", UserID:"BF5S2HSTG", Avatar:"", Account:"slack.mta", Event:"", Protocol:"slack", Gateway:"mta.test", ParentID:"", Timestamp:time.Time{wall:0xbf73554a286d2fa3, ext:93484892412, loc:(*time.Location)(0x1ce7500)}, ID:"", Extra:map[string][]interface {}{}}
[0093] DEBUG discord:      Broadcasting using Webhook
[0093] DEBUG discord:      Setting webhook channel to "test"
[0093] DEBUG discord:      Processing webhook sending for message config.Message{Text:"test", Channel:"test", Username:"qaisjp", UserID:"BF5S2HSTG", Avatar:"", Account:"slack.mta", Event:"", Protocol:"slack", Gateway:"mta.test", ParentID:"", Timestamp:time.Time{wall:0xbf73554a286d2fa3, ext:93484892412, loc:(*time.Location)(0x1ce7500)}, ID:"", Extra:map[string][]interface {}{}}
[0093] DEBUG gateway:      mID discord.mta: 653195441021648965

it should look like this

[0056] DEBUG discord:      == Receiving event &discordgo.Message{ID:"653196539988279296", ChannelID:"590172620440469505", GuildID:"278474088903606273", Content:"hello1234", Timestamp:"2019-12-08T11:29:50.938000+00:00", EditedTimestamp:"", MentionRoles:[]string{}, Tts:false, MentionEveryone:false, Author:(*discordgo.User)(0xc000331000), Attachments:[]*discordgo.MessageAttachment{}, Embeds:[]*discordgo.MessageEmbed{}, Mentions:[]*discordgo.User{}, Reactions:[]*discordgo.MessageReactions(nil), Pinned:false, Type:0, WebhookID:"", Member:(*discordgo.Member)(0xc000b5e700), MentionChannels:[]*discordgo.Channel(nil), Activity:(*discordgo.MessageActivity)(nil), Application:(*discordgo.MessageApplication)(nil), MessageReference:(*discordgo.MessageReference)(nil), Flags:0}
[0056] DEBUG discord:      <= Sending message from qaisjp on discord.mta to gateway
[0056] DEBUG discord:      <= Message is config.Message{Text:"hello1234", Channel:"test", Username:"qaisjp", UserID:"83386293446246400", Avatar:"https://cdn.discordapp.com/avatars/83386293446246400/6c4342d5d3556d4d6993232272eb9699.jpg", Account:"discord.mta", Event:"", Protocol:"", Gateway:"", ParentID:"", Timestamp:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}, ID:"653196539988279296", Extra:map[string][]interface {}(nil)}
[0056] DEBUG gateway:      => Sending config.Message{Text:"hello1234", Channel:"test", Username:"qaisjp", UserID:"83386293446246400", Avatar:"https://cdn.discordapp.com/avatars/83386293446246400/6c4342d5d3556d4d6993232272eb9699.jpg", Account:"discord.mta", Event:"", Protocol:"", Gateway:"mta.test", ParentID:"", Timestamp:time.Time{wall:0xbf73558bbb63132c, ext:56046693989, loc:(*time.Location)(0x1ce2500)}, ID:"653196539988279296", Extra:map[string][]interface {}(nil)} from discord.mta (test) to slack.mta (test)
[0056] DEBUG slack:        => Receiving config.Message{Text:"hello1234", Channel:"test", Username:"qaisjp", UserID:"83386293446246400", Avatar:"https://cdn.discordapp.com/avatars/83386293446246400/6c4342d5d3556d4d6993232272eb9699.jpg", Account:"discord.mta", Event:"", Protocol:"", Gateway:"mta.test", ParentID:"", Timestamp:time.Time{wall:0xbf73558bbb63132c, ext:56046693989, loc:(*time.Location)(0x1ce2500)}, ID:"", Extra:map[string][]interface {}(nil)}
[0056] DEBUG slack:        == Receiving event &slack.MessageEvent{Msg:slack.Msg{Type:"message", Channel:"C4Z7JEUC8", User:"", Text:"hello1234", Timestamp:"1575804591.002900", ThreadTimestamp:"", IsStarred:false, PinnedTo:[]string(nil), Attachments:[]slack.Attachment{slack.Attachment{Color:"", Fallback:"", CallbackID:"matterbridge_bnmdstnrrg6f6to9vtp0", ID:1, AuthorID:"", AuthorName:"", AuthorSubname:"", AuthorLink:"", AuthorIcon:"", Title:"", TitleLink:"", Pretext:"", Text:"", ImageURL:"", ThumbURL:"", Fields:[]slack.AttachmentField(nil), Actions:[]slack.AttachmentAction(nil), MarkdownIn:[]string(nil), Footer:"", FooterIcon:"", Ts:""}}, Edited:(*slack.Edited)(nil), LastRead:"", Subscribed:false, UnreadCount:0, SubType:"bot_message", Hidden:false, DeletedTimestamp:"", EventTimestamp:"1575804591.002900", BotID:"BF5S2HSTG", Username:"qaisjp", Icons:(*slack.Icon)(0xc0005bc8c0), Inviter:"", Topic:"", Purpose:"", Name:"", OldName:"", Members:[]string(nil), ReplyCount:0, Replies:[]slack.Reply(nil), ParentUserId:"", Files:[]slack.File(nil), Upload:false, Comment:(*slack.Comment)(nil), ItemType:"", ReplyTo:0, Team:"T216B1U03", Reactions:[]slack.ItemReaction(nil), ResponseType:"", ReplaceOriginal:false, DeleteOriginal:false, Blocks:slack.Blocks{BlockSet:[]slack.Block(nil)}}, SubMessage:(*slack.Msg)(nil), PreviousMessage:(*slack.Msg)(nil)}
[0056] DEBUG slack:        Skipped message: &slack.MessageEvent{Msg:slack.Msg{Type:"message", Channel:"C4Z7JEUC8", User:"", Text:"hello1234", Timestamp:"1575804591.002900", ThreadTimestamp:"", IsStarred:false, PinnedTo:[]string(nil), Attachments:[]slack.Attachment{slack.Attachment{Color:"", Fallback:"", CallbackID:"matterbridge_bnmdstnrrg6f6to9vtp0", ID:1, AuthorID:"", AuthorName:"", AuthorSubname:"", AuthorLink:"", AuthorIcon:"", Title:"", TitleLink:"", Pretext:"", Text:"", ImageURL:"", ThumbURL:"", Fields:[]slack.AttachmentField(nil), Actions:[]slack.AttachmentAction(nil), MarkdownIn:[]string(nil), Footer:"", FooterIcon:"", Ts:""}}, Edited:(*slack.Edited)(nil), LastRead:"", Subscribed:false, UnreadCount:0, SubType:"bot_message", Hidden:false, DeletedTimestamp:"", EventTimestamp:"1575804591.002900", BotID:"BF5S2HSTG", Username:"qaisjp", Icons:(*slack.Icon)(0xc0005bc8c0), Inviter:"", Topic:"", Purpose:"", Name:"", OldName:"", Members:[]string(nil), ReplyCount:0, Replies:[]slack.Reply(nil), ParentUserId:"", Files:[]slack.File(nil), Upload:false, Comment:(*slack.Comment)(nil), ItemType:"", ReplyTo:0, Team:"T216B1U03", Reactions:[]slack.ItemReaction(nil), ResponseType:"", ReplaceOriginal:false, DeleteOriginal:false, Blocks:slack.Blocks{BlockSet:[]slack.Block(nil)}}, SubMessage:(*slack.Msg)(nil), PreviousMessage:(*slack.Msg)(nil)}
[0056] DEBUG gateway:      mID slack.mta: 1575804591.002900

it is caused exactly by f43faf1, because if i remove that commit, the issue goes away

[docwilco]

I'm running 1.19.0 2bb2126 and having this issue:

[614125] ERROR slack:        Connection failed "slack rate limit exceeded, retry after 1m0s" &slack.RateLimitedError{RetryAfter:60000000000}

Multiple like these, and messages don't show up on Discord when people are talking a lot in Slack. And it's not that much even.

[qaisjp]

Thanks for sharing this. Please could you create a specific issue for that problem? :)