Microsoft deprecating Teams webhooks, new "Workflows" uses incompatible schema #3920
Comments
|
Thanks for reporting this. I'll look into it. |
|
Related to: #3503 |
|
I have built a crude workaround to make the new Flow system accept the "old" data format. Start by creating a new "Post to a channel when a webhook request is received" Flow, which can be created by clicking on the "Set up workflow" link in the warning message or in power automate directly., then do the following modifications:
Be warned: I have no idea how reliable this will work or what content might get lost. This was my first time ever working with power automate and as far as I can tell it is a piece of s*** with the documentation doing an impressive balancing act between not existing and being completely useless and incomplete. There seems to be some limitations regarding private channels, but I'm way to annoyed by this whole ordeal to figure it out at the moment. It works in private channels when the message gets sent as the User instead of the Flow Bot as long as the User is a member of the channel. |
|
Is there any workaround about this issue? i am using msteams_config in alertmanager. Any ideas how to handle it with Workflows? |
tl;dr I am working on a hacky solution atm. {{ define "teams.txt" }}{'foo':'bar'}{{ end }}Alert Manager config: Then use PowerAutomate to parse the text field.... # PowerFX lang? idk man https://learn.microsoft.com/en-us/power-platform/power-fx/overview
json(trigger().outputs['body']['text'])Because I can do loops and conditionals, I am thinking of doing my complex alert routing via PowerAutomate with a single webhook instead of creating a webhook per channel. |
|
I hate that this works. I feel gross. I stole a template from this comment: Defined the template as the text field (see previous comment for how to do that). # Powerfx function
variables("msgs")Step 4: Profit! |
thaks for sharing! i tried as u mentioned but failed in 3rd step( I am using alertmanager template in the comment you mentioned, and workflows configuration |
In case somebody else encounters the same error. Using this setup, I was confronted with the error I adapted the Works like a charm. Thanks a lot. |
are you using template in alertmanager side? and what integration are u using in alertmanager? is it msteams ? |
I'm using alertmanager v0.27.0 and the relevant "Config" output in the alertmanager status page is |
Great! that works for me also. Thank you! |
|
FYI for those of us who were using markdown in the content of the teams notifications, the adaptative cards do not support a lot of things, like tables (https://learn.microsoft.com/en-us/adaptive-cards/authoring-cards/text-features):
|
|
re: This issue occurred when the final step changed itself to a foreach action instead of a Post to teams: Below is what you want to confirm: View #3920 (comment) for more details on the var setup My AM config: Beginning of Go template: Also note: When I tried updating the variable a few times testing different scenarios, it didn't actually save my changes so I was forced to delete the step and remake it. My brutal Go template does work in this circumstance as a variable with multiple alerts (below is an example of a grouped alert)
Obviously not amazing to use that wicked template but may be a workaround for alerts with functionality attached until a better solution is found |
|
Also be advised all: Our team decided to try and go the Microsoft Graph route instead to avoid this madness. We were thinking of using Alertmanager to then ship to an intermediary data transformer to then POST via Graph to the appropriate channels. The only way for this to work is to have your application registration's permission access in Entrant set as a Delegate, not the application context. You cannot set it to the application context, otherwise when you try to do a POST you'll get an error like:
Why does this matter? It means that the application's permission scope, by acting as a delegate, requires your Microsoft Admin to granularly manage permissions. If you operate a relatively large environment...good luck. |
|
It looks like Teams has some incompatibilities across the product tiers, none of which are documented. I suspect this is something like the enterprise SKUs but did not want to spend time trying to see if they've hidden that information somewhere else on microsoft.com. The example in #3920 (comment) came close to working for me but it turned out there were a couple of key differences:
|
|
Does anyone have a solution for sending from Alertmanager without the |
could you maybe use https://github.com/prometheus-msteams/prometheus-msteams ? I dont know if the workarounds here work for that tool |
It did not work with the dynamic URLs as it seems like it's cutting out some parts of the URL. |
|
The following one works for me. 
outputs('Compose') 
alertmanager.yml in alertmanager.yml, don't use templates: |
|
Thanks @The-M1k3y , @acdha and everyone else who shared their solution with the community in this issue. I was finally able to get MS Teams alerts to work with the workflows feature :-). |
|
How can I prevent my title from being cut off like this towards the end and make it go to a new line instead? - 
|
|
@sherwinwater Change msteam.tmpl to the following or do not define msteam.tmpl, you should receive an alert |
@lzdohh Thank you so much. It works for me after I remove the msteam.tmpl. thanks. |
|
The deadline was actually extended. However, the URL of the webhooks must be updated again by December 31, 2024. If other system administrators are having difficulties locating the affected teams: This has helped us enormously, as Microsoft does not offer its own solution for reading the affected channels and teams. We have over 350 teams in our company, which we would otherwise have had to search through manually. I hope I could help someone here with this. Independently of my day job, I would like to create an alternative to the native “incoming webhooks”. Maybe someone will benefit from it. |
You just need to decode the webhook endpoint link, decode the url, you can use this online service https://amp.urldecoder.org/, after that a you need to paste clean url in the config, this helped me bypass the utl problem. Thank you everyone for sharing your solution, I’m happy because I’m understanding I’m not self with this problem. |
@6fears7 I tried this method but got below error |
|
@maneeshcdls Does it happen for every alert or only for some?
|
|
I could successfully send OpenShift Alerts to Microsoft Teams channels using the following:
Adaptive Card TemplateAlertManager ConfigurationExample Card
|
Adding the msteams width:full to Compose step allowed me to have enough space so the title won't cut off: {
"$schema": "http://adaptivecards.io/schemas/adaptive-card.json",
"version": "1.4",
"type": "AdaptiveCard",
"body": [
{
"type": "TextBlock",
"size": "Medium",
"weight": "Bolder",
"text": "@{triggerBody()?['title']}"
},
{
"type": "TextBlock",
"text": "@{triggerBody()?['text']}",
"wrap": true
}
],
"msteams": {
"width": "Full"
}
} |
|
Hi Everyone, |
|
Hi Bhargav, |
Thanks @SaurabhKoverhoop |
|
my way was like this Teams Workflow Setup
Alertmanager Configuration
ISTIO (optional)don't forget make domain for webhooks accessible (prod-11.southeastasia.logic.azure.com in this case) apiVersion: networking.istio.io/v1alpha3
kind: ServiceEntry
metadata:
name: teams-webhook
spec:
hosts:
- prod-11.southeastasia.logic.azure.com
ports:
- name: https
number: 443
protocol: HTTPS
location: MESH_EXTERNAL
resolution: DNS
exportTo:
- "." |
|
@yykudrin thanks for your input , do we miss something at step 18 ? it is yellin' this ain't a correct array if we insert "outputs('Compose')" |
|
@odcheck select the Outputs option under compose appearing in Dynamic content. |
Signed-off-by: Simon Schneider <github@simon-schneider.eu>
Signed-off-by: Simon Schneider <github@simon-schneider.eu>
Thank you for this. I use this config, in PC work fine but the message not appear on my ios msteam app. |
|
Hi All, Greetings! When are we expecting the schema changes to be available in the alert manager code so that we can avoid these manual workarounds? |
|
Hi everyone, I created a working workflow which uses the Alertmanager's webhook receiver instead of msteams receiver. I wanted to avoid the alerting getting broken when either Microsoft changes something or Prometheus patches the schema of the msteams receiver. I read multiple times that the webhook receiver is planned to stay fixed in the format of the payload. I also plan on using this workflow to connect Google Alerting to Microsoft Teams as well. And instead of a third-party middleware to transform Alertmanager's webhook payload, I used the functionality of Workflows directly. The resulting rendered post in the Teams channel: This is what the workflow looks like: I would have liked to have only one "append to array" in the ForEach segment which adds all the TextBlocks at in a single iteration, but for me the formatting of the JSON got a little mangled. If someone would like to share a hint on this, I would be grateful. Using this example JSON: Webhook receiver payloadThis workflow will work very well for me:
Schema
Value
Value
Value
Value
Value
Value
Value
Value
Value
Input |
|
Anyone looking at #4024 can use this to expedite the deployment of the new Microsoft Teams v2 receiver configuration with workflows (for migrating connectors to workflows). I tested it today. Note: Please use the release channel version only for testing, and wait for the official version before deploying in production. image:
tag: v0.28.0-rc.0
config:
global:
resolve_timeout: 5m
receivers:
- name: "prometheus-microsoftteams-alertmanager"
msteamsv2_configs:
- send_resolved: true
webhook_url: "your_webhook_url_goes_here"
title: "{{ range .Alerts }}[{{ .Labels.severity | toUpper }}] {{ .Labels.alertname }}: {{ .Annotations.summary }}\n{{ end }}"
text: |
{{ range .Alerts }}
**Alert Details**
- **Alert Name:** {{ .Labels.alertname }}
- **Status:** {{ .Status }}
- **Severity:** {{ .Labels.severity }}
- **Description:** {{ .Annotations.description }}
- **LABELS:**
{{ range $key, $value := .Labels }}
- {{ $key }}: {{ $value }}
{{ end }}
- **Starts At:** {{ .StartsAt }}
{{ if eq .Status "resolved" }}
- **Ends At:** {{ .EndsAt }}
{{ else }}
- **Ends At:** Alert is still firing
{{ end }}
{{ end }}
route:
group_by: ["alertname"]
group_interval: 30s
repeat_interval: 15m
group_wait: 30s
receiver: "prometheus-microsoftteams-alertmanager"
routes:
- receiver: prometheus-microsoftteams-alertmanager
match:
severity: critical |
Would be possible to export the workflow to a file and paste it here? |
|
Hi There,
I followed exactly |
Hi @philipsabri |
and others
Update: For a (seemingly functional) workaround see my comment #3920 (comment) below.
Original Issue:
While setting up a new alertmanager - Teams notification via Webhook I noticed a new warning that the current webhook interface will be deprecated.
From August 15th onwards it will not be possible to setup new webhooks.
On October 1st all existing webhooks will stop working.
Details here: https://devblogs.microsoft.com/microsoft365dev/retirement-of-office-365-connectors-within-microsoft-teams/
A few tests show, that the workflow trigger "Teams webhook" uses a different scheme for its data.
schema of new endpoint according to documentation:
minimal example of a working request body:
The text was updated successfully, but these errors were encountered: