Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Python: fix for file limit and some cleanup #9855

Merged
merged 4 commits into from
Dec 3, 2024
Merged

Python: fix for file limit and some cleanup #9855

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
28676f3
fix for file limit and some cleanup
eavanvalkenburg Dec 2, 2024
bcb67f4
updated ollama links
eavanvalkenburg Dec 2, 2024
22ec2f2
Merge branch 'main' into azure_assistant_fix
crickman Dec 2, 2024
1e5739c
Merge branch 'main' into azure_assistant_fix
eavanvalkenburg Dec 3, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/decisions/0031-feature-branch-strategy.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ Cons:
| Windows Support | No | Yes |
| Linux Support | Yes | Yes |
| MacOS Support | Yes | Yes |
| Number of Models | [61](https://ollama.ai/library) +Any GGUF converted | [25](https://github.com/lmstudio-ai/model-catalog/tree/main/models) +Any GGUF Converted |
| Number of Models | [61](https://ollama.com/search) +Any GGUF converted | [25](https://github.com/lmstudio-ai/model-catalog/tree/main/models) +Any GGUF Converted |

| Model Support | Ollama | LM Studio |
| --------------- | ------ | --------- |
Expand Down
38 changes: 19 additions & 19 deletions dotnet/docs/MODELS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ In the core Semantic Kernel repo, we plan on supporting up to four deployment ty

- Dedicated API endpoints (e.g., OpenAI's APIs, Mistral.AI, and Google Gemini)
- Azure AI deployments via the [model catalog](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/model-catalog)
- Local deployments via [Ollama](https://ollama.ai/library)
- Local deployments via [Ollama](https://ollama.ai/)
- Hugging face deployment using the [Hugging Face inference API](https://huggingface.co/docs/api-inference/index)

To support these different deployment types, we will follow a similar pattern to the Azure OpenAI and OpenAI connectors. Each connector uses the same underlying model and abstractions, but the connector constructors may take different parameters. For example, the Azure OpenAI connector expects an Azure endpoint and key, whereas the OpenAI connector expects an OpenAI organization ID and API key.
Expand All @@ -23,25 +23,25 @@ Please note that not all of the model interfaces are defined yet. As part of con

### OpenAI

| Priority | Model | Status | Interface | Deployment type | GitHub issue | Developer | Reviewer |
| -------- | ----------------------- | ----------- | ------------------------------ | --------------- | ------------ | ----------- | -------- |
| P0 | GPT-3.5-turbo | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-3.5-turbo | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | GPT-4 | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-4 | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | GPT-4v | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-4v | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | text-embedding-ada-002 | Preview | `IEmbeddingGeneration` | OpenAI API | N/A | N/A | N/A |
| P0 | text-embedding-ada-002 | Preview | `IEmbeddingGeneration` | Azure AI | N/A | N/A | N/A |
| P0 | DALL·E 3 | Preview | `ITextToImage` | OpenAI API | N/A | N/A | N/A |
| P0 | DALL·E 3 | Preview | `ITextToImage` | Azure AI | N/A | N/A | N/A |
| P0 | Text-to-speech | Complete | `ITextToSpeech` | OpenAI API | TBD | dmytrostruk | TBD |
| P0 | Speech-to-text | Complete | `ISpeechRecognition` | OpenAI API | TBD | dmytrostruk | TBD |
| P1 | openai-whisper-large-v3 | Not started | `ISpeechRecognition` | Azure AI | TBD | TBD | TBD |
| P1 | openai-whisper-large-v3 | Not started | `ISpeechRecognition` | Hugging Face | TBD | TBD | TBD |
| Priority | Model | Status | Interface | Deployment type | GitHub issue | Developer | Reviewer |
| -------- | ----------------------- | ----------- | ------------------------------ | --------------- | ------------ | ------------ | ----------- |
| P0 | GPT-3.5-turbo | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-3.5-turbo | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | GPT-4 | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-4 | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | GPT-4v | Complete | `IChatCompletion` | OpenAI API | N/A | N/A | N/A |
| P0 | GPT-4v | Complete | `IChatCompletion` | Azure AI | N/A | N/A | N/A |
| P0 | text-embedding-ada-002 | Preview | `IEmbeddingGeneration` | OpenAI API | N/A | N/A | N/A |
| P0 | text-embedding-ada-002 | Preview | `IEmbeddingGeneration` | Azure AI | N/A | N/A | N/A |
| P0 | DALL·E 3 | Preview | `ITextToImage` | OpenAI API | N/A | N/A | N/A |
| P0 | DALL·E 3 | Preview | `ITextToImage` | Azure AI | N/A | N/A | N/A |
| P0 | Text-to-speech | Complete | `ITextToSpeech` | OpenAI API | TBD | dmytrostruk | TBD |
| P0 | Speech-to-text | Complete | `ISpeechRecognition` | OpenAI API | TBD | dmytrostruk | TBD |
| P1 | openai-whisper-large-v3 | Not started | `ISpeechRecognition` | Azure AI | TBD | TBD | TBD |
| P1 | openai-whisper-large-v3 | Not started | `ISpeechRecognition` | Hugging Face | TBD | TBD | TBD |
| P2 | Moderation | In Progress | `ITextClassification` | OpenAI API | #5062 | Krzysztof318 | MarkWallace |
| P2 | clip-vit-base-patch32 | Not started | `IZeroShotImageClassification` | Azure AI | TBD | TBD | TBD |
| P2 | clip-vit-base-patch32 | Not started | `IZeroShotImageClassification` | Hugging Face | TBD | TBD | TBD |
| P2 | clip-vit-base-patch32 | Not started | `IZeroShotImageClassification` | Azure AI | TBD | TBD | TBD |
| P2 | clip-vit-base-patch32 | Not started | `IZeroShotImageClassification` | Hugging Face | TBD | TBD | TBD |

### Microsoft

Expand Down
24 changes: 13 additions & 11 deletions python/semantic_kernel/agents/open_ai/assistant_content_generation.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,17 +85,19 @@ def get_message_contents(message: "ChatMessageContent") -> list[dict[str, Any]]:
"""
contents: list[dict[str, Any]] = []
for content in message.items:
if isinstance(content, TextContent):
contents.append({"type": "text", "text": content.text})
elif isinstance(content, ImageContent) and content.uri:
contents.append(content.to_dict())
elif isinstance(content, FileReferenceContent):
contents.append({
"type": "image_file",
"image_file": {"file_id": content.file_id},
})
elif isinstance(content, FunctionResultContent):
contents.append({"type": "text", "text": content.result})
match content:
case TextContent():
contents.append({"type": "text", "text": content.text})
case ImageContent():
if content.uri:
contents.append(content.to_dict())
case FileReferenceContent():
contents.append({
"type": "image_file",
"image_file": {"file_id": content.file_id},
})
case FunctionResultContent():
contents.append({"type": "text", "text": content.result})
return contents


Expand Down
12 changes: 6 additions & 6 deletions python/semantic_kernel/agents/open_ai/azure_assistant_agent.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -54,11 +54,11 @@ def __init__(
enable_code_interpreter: bool | None = None,
enable_file_search: bool | None = None,
enable_json_response: bool | None = None,
file_ids: list[str] | None = [],
file_ids: list[str] | None = None,
temperature: float | None = None,
top_p: float | None = None,
vector_store_id: str | None = None,
metadata: dict[str, Any] | None = {},
metadata: dict[str, Any] | None = None,
max_completion_tokens: int | None = None,
max_prompt_tokens: int | None = None,
parallel_tool_calls_enabled: bool | None = True,
Expand Down Expand Up @@ -150,11 +150,11 @@ def __init__(
"enable_code_interpreter": enable_code_interpreter,
"enable_file_search": enable_file_search,
"enable_json_response": enable_json_response,
"file_ids": file_ids,
"file_ids": file_ids or [],
"temperature": temperature,
"top_p": top_p,
"vector_store_id": vector_store_id,
"metadata": metadata,
"metadata": metadata or {},
"max_completion_tokens": max_completion_tokens,
"max_prompt_tokens": max_prompt_tokens,
"parallel_tool_calls_enabled": parallel_tool_calls_enabled,
Expand Down Expand Up @@ -199,7 +199,7 @@ async def create(
temperature: float | None = None,
top_p: float | None = None,
vector_store_id: str | None = None,
metadata: dict[str, Any] | None = {},
metadata: dict[str, Any] | None = None,
max_completion_tokens: int | None = None,
max_prompt_tokens: int | None = None,
parallel_tool_calls_enabled: bool | None = True,
Expand Down Expand Up @@ -268,7 +268,7 @@ async def create(
temperature=temperature,
top_p=top_p,
vector_store_id=vector_store_id,
metadata=metadata,
metadata=metadata or {},
max_completion_tokens=max_completion_tokens,
max_prompt_tokens=max_prompt_tokens,
parallel_tool_calls_enabled=parallel_tool_calls_enabled,
Expand Down
19 changes: 19 additions & 0 deletions python/semantic_kernel/agents/open_ai/function_action_result.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
# Copyright (c) Microsoft. All rights reserved.

import logging
from dataclasses import dataclass

from semantic_kernel.contents.chat_message_content import ChatMessageContent
from semantic_kernel.utils.experimental_decorator import experimental_class

logger: logging.Logger = logging.getLogger(__name__)


@experimental_class
@dataclass
class FunctionActionResult:
"""Function Action Result."""

function_call_content: ChatMessageContent | None
function_result_content: ChatMessageContent | None
tool_outputs: list[dict[str, str]] | None
12 changes: 6 additions & 6 deletions python/semantic_kernel/agents/open_ai/open_ai_assistant_agent.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,11 @@ def __init__(
enable_code_interpreter: bool | None = None,
enable_file_search: bool | None = None,
enable_json_response: bool | None = None,
code_interpreter_file_ids: list[str] | None = [],
code_interpreter_file_ids: list[str] | None = None,
temperature: float | None = None,
top_p: float | None = None,
vector_store_id: str | None = None,
metadata: dict[str, Any] | None = {},
metadata: dict[str, Any] | None = None,
max_completion_tokens: int | None = None,
max_prompt_tokens: int | None = None,
parallel_tool_calls_enabled: bool | None = True,
Expand Down Expand Up @@ -125,11 +125,11 @@ def __init__(
"enable_code_interpreter": enable_code_interpreter,
"enable_file_search": enable_file_search,
"enable_json_response": enable_json_response,
"code_interpreter_file_ids": code_interpreter_file_ids,
"code_interpreter_file_ids": code_interpreter_file_ids or [],
"temperature": temperature,
"top_p": top_p,
"vector_store_id": vector_store_id,
"metadata": metadata,
"metadata": metadata or {},
"max_completion_tokens": max_completion_tokens,
"max_prompt_tokens": max_prompt_tokens,
"parallel_tool_calls_enabled": parallel_tool_calls_enabled,
Expand Down Expand Up @@ -173,7 +173,7 @@ async def create(
temperature: float | None = None,
top_p: float | None = None,
vector_store_id: str | None = None,
metadata: dict[str, Any] | None = {},
metadata: dict[str, Any] | None = None,
max_completion_tokens: int | None = None,
max_prompt_tokens: int | None = None,
parallel_tool_calls_enabled: bool | None = True,
Expand Down Expand Up @@ -236,7 +236,7 @@ async def create(
temperature=temperature,
top_p=top_p,
vector_store_id=vector_store_id,
metadata=metadata,
metadata=metadata or {},
max_completion_tokens=max_completion_tokens,
max_prompt_tokens=max_prompt_tokens,
parallel_tool_calls_enabled=parallel_tool_calls_enabled,
Expand Down
Loading
Loading