Merge pull request #384 from dyte-io/feature/ai-481

docs: ai config and summary guide
dyte-io · Aug 9, 2024 · b473bf6 · b473bf6
2 parents eafa789 + bab8ff8
commit b473bf6
Show file tree

Hide file tree

Showing 5 changed files with 368 additions and 18 deletions.
diff --git a/docs/guides/capabilities/ai/index.mdx b/docs/guides/capabilities/ai/index.mdx
@@ -3,13 +3,17 @@
 Learn how you can harnesses the power of Artificial Intelligence in Dyte meetings to perform transcription, summarization, and so on.
 
 <CardSection id="AI">
-
   <Card
     title="Meeting Transcription"
     to="/guides/capabilities/ai/meeting-transcription"
   />
+  <Card
+    title="Meeting Summary"
+    to="/guides/capabilities/ai/meeting-summary"
+  />
 </CardSection>
 
+
 <head>
   <title>Meeting AI Guide</title>
   <meta name="description" content="Dive into Dyte's AI capabilities with our comprehensive guide. Explore how AI enhances your meeting experience with Dyte."/>

diff --git a/docs/guides/capabilities/ai/meeting-summary.mdx b/docs/guides/capabilities/ai/meeting-summary.mdx
@@ -0,0 +1,106 @@
+---
+sidebar_position: 1
+description: >-
+  Learn how to generate and utilize meeting summaries with Dyte's AI.
+  This guide will help you understand and implement summarization features effectively.
+---
+
+# Meeting Summary
+
+Dyte's meeting summary feature allows you to automatically generate concise summaries of your meeting based on the transcription data.
+This feature makes it easy to capture key points and action items, providing a concise overview of your discussions.
+
+:::info
+To generate a meeting summary, transcription must be enabled. The summarization feature relies on the transcription data to create accurate and meaningful summaries.
+:::
+## Enabling Meeting Summarization
+
+To enable automatic summarization post meeting, you can set the `summarize_on_end` flag when creating a meeting using the [REST API](/api#/operations/create_meeting).
+
+
+### Summarization Configuration Options
+
+You can tailor the summarization process using the following configuration options:
+
+#### Word Limit
+
+Define the word limit for the summary, ensuring it fits your needs. You can set a limit between 150 and 1000 words.
+
+```json lines
+"word_limit": 500
+```
+
+#### Text Format
+
+Choose the format for the summary text. Supported formats are:
+
+- `plain_text`
+- `markdown`
+
+```json lines
+"text_format": "markdown"
+```
+
+#### Summary Type
+
+Select the type of summary based on the nature of the meeting. Supported types are:
+
+- `general`
+- `team_meeting`
+- `sales_call`
+- `client_check_in`
+- `interview`
+- `daily_standup`
+- `one_on_one_meeting`
+- `lecture`
+- `code_review`
+
+```json lines
+"summary_type": "team_meeting"
+```
+### Example Configuration
+
+Here is an example of how to enable summarization in the meeting creation API call:
+
+```json
+{
+  "title": "Team Meeting",
+  "ai_config": {
+    "summarization": {
+      "word_limit": 500,
+      "text_format": "plain_text",
+      "summary_type": "team_meeting"
+    }
+  },
+  "summarize_on_end": true
+}
+```
+
+
+
+## Consuming Summaries
+
+There are two ways to consume the generated summaries:
+
+1. **Webhooks:** Receive the meeting summary via a webhook after the meeting ends.
+2. **API Call:** Fetch the meeting summary using the [REST API](/api#/operations/GetMeetingSummary).
+
+### Fetching Summary via Webhook
+
+To receive the meeting summary automatically once the meeting concludes, configure a webhook with the `meeting.summary` event enabled. This can be done either on our [Developer Portal](https://dev.dyte.io/webhooks) or using a [REST API](/api#/operations/addWebhook).
+
+You can see the webhook format [here](/guides/capabilities/webhooks/webhook-events#meetingsummary).
+
+### Fetching Summary via API Call
+
+You can use API to fetch the summary for a meeting at a later time using [REST API](/api#/operations/GetSessionSummary)
+Dyte stores the summary of a meeting for 7 days since the start of the meeting.
+
+## Triggering Summary manually
+
+If you need to generate a summary after the meeting has ended, you can trigger the summary using [REST API](/api#/operations/post-sessions-session_id-summary).
+
+<head>
+  <title>Meeting Summary Guide</title>
+  <meta name="description" content="Learn how to generate and utilize meeting summaries with Dyte's AI. This guide will help you understand and implement summarization features effectively."/>
+</head>
diff --git a/docs/guides/capabilities/ai/meeting-transcription.mdx b/docs/guides/capabilities/ai/meeting-transcription.mdx
@@ -14,9 +14,6 @@ The meeting transcription feature is currently in beta, which means it is still
 Dyte's meeting transcription allows you to transcribe your Dyte meetings in real-time, making it easy to capture important discussions and refer back to them later.
 This guide will walk you through how to use this feature effectively.
 
-:::info NOTE
-Dyte's AI meeting transcription currently only supports English.
-:::
 
 ## Control transcriptions for participants using presets
 
@@ -25,6 +22,64 @@ All participants with the `transcription_enabled` turned on in their preset will
 
 You can create a new preset on our [Developer Portal](https://dev.dyte.io/presets), or using our [REST API](/api#/operations/post-presets).
 
+## Modify Transcription Behavior with AI Config
+
+You can control transcription behavior through a configurable AI setup. When creating a meeting using the [REST API](/api#/operations/create_meeting), you can pass an AI configuration for transcriptions. This allows for greater control over the transcription process.
+
+
+### Supported Languages
+
+You can specify the language for transcription to ensure accurate and relevant results. The following languages are supported:
+
+
+- *English (United States):* `en-US`
+- *English (India):* `en-IN`
+- *German:* `de`
+- *Hindi:* `hi`
+- *Swedish:* `sv`
+- *Russian:* `ru`
+- *Polish:* `pl`
+- *Greek:* `el`
+- *French:* `fr`
+- *Dutch:* `nl`
+
+```json lines
+"language": "en-US"
+```
+### Keywords
+
+Keywords can be added to help the transcription engine accurately detect and transcribe specific terms, such as names, technical jargon, or other context-specific words. This is particularly useful in meetings where certain terms are frequently used and need to be recognized correctly.
+
+```json lines
+"keywords": ["Dyte", "Mary", "Sue"]
+```
+
+
+### Profanity Filter
+
+You can enable or disable the profanity filter based on your needs. This feature ensures that any offensive language is either included or excluded from the transcriptions, depending on your preference.
+```json lines
+"profanity_filter": false
+```
+
+By utilizing these new features, you can customize the transcription experience to better suit your needs.
+
+### Example Configuration
+Here is an example of how to pass the AI configuration in the [meeting creation API](/api#/operations/create_meeting):
+```json
+{
+  "title": "Meeting Transcriptions",
+  "ai_config": {
+    "transcription": {
+      "keywords": ["Dyte"],
+      "language": "en-US",
+      "profanity_filter": false
+    }
+  }
+}
+
+```
+
 ## Consuming transcripts
 
 There are 3 ways in which these transcripts can be consumed.
@@ -52,6 +107,21 @@ meeting.ai.on('transcript', (transcriptData) => {
 });
 ```
 
+As participants speak during the meeting, you'll receive partial transcripts, giving you real-time feedback even before they finish their sentences. The `isPartialTranscript` flag in the transcript data shows whether the transcript is partial or final.
+```json
+{
+  "id": "1a2b3c4d-5678-90ab-cdef-1234567890ab",
+  "name": "Alice",
+  "peerId": "4f5g6h7i-8j9k-0lmn-opqr-1234567890st",
+  "userId": "uvwxyz-1234-5678-90ab-cdefghijklmn",
+  "customParticipantId": "abc123xyz",
+  "transcript": "Hello?",
+  "isPartialTranscript": true,
+  "date": "Wed Aug 07 2024 10:15:30 GMT+0530 (India Standard Time)"
+}
+```
+In the example above, `isPartialTranscript` is true, indicating the transcript is still in progress. Once the participant finishes speaking, the final transcript will be sent with `isPartialTranscript` set to false. This helps you distinguish between ongoing speech and completed transcriptions, making the transcription process more dynamic and responsive.
+
 ### Consume transcript via a post-meeting webhook
 
 You can configure a webhook with the `meeting.transcript` event enabled to receive the meeting transcript after the meeting has ended.

diff --git a/docs/guides/capabilities/webhooks/webhook-events.mdx b/docs/guides/capabilities/webhooks/webhook-events.mdx
@@ -220,6 +220,32 @@ The payload for this event is like the following:
 }
 ```
 
+### `meeting.summary`
+
+This event is triggered when the summary of a meeting becomes available after the meeting ends.
+
+The payload for this event is structured as follows:
+```json
+{
+  "event": "meeting.summary",
+  "meeting": {
+    "id": "bbbba351-90c5-487f-8358-ce4885555134",
+    "sessionId": "01eab2e1-3cd4-4237-8cf3-c8c071ef87e9",
+    "title": "Meeting",
+    "status": "LIVE",
+    "createdAt": "2023-08-10T10:31:20.173Z",
+    "startedAt": "2023-08-10T10:31:20.173Z",
+    "endedAt": "2023-08-10T10:33:56.771Z",
+    "organizedBy": {
+          "id": "d6f046b8-b638-4cf7-8090-d9332447ae49",
+          "name": "Dyte"
+      }
+    },
+  "summaryDownloadUrl": "https://sample.com/summary.txt",
+  "summaryDownloadUrlExpiry": "2023-12-13 07:08:39.051621Z"
+}
+```
+
 ### `recording.statusUpdate`
 
 This event is received when the status of a recording changes. The possible