Python: fix for file limit and some cleanup (#9855)

### Motivation and Context  We got a report stating that there was still a old limit on the number of files supplied to the Azure Assistant API. This PR fixes that and also does some further cleanup of the code. ### Description  ### Contribution Checklist  - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄 --------- Co-authored-by: Chris <[email protected]>
microsoft · Dec 3, 2024 · be96919 · be96919
1 parent 7770aab
commit be96919
Show file tree

Hide file tree

Showing 7 changed files with 118 additions and 94 deletions.
diff --git a/docs/decisions/0031-feature-branch-strategy.md b/docs/decisions/0031-feature-branch-strategy.md
@@ -96,7 +96,7 @@ Cons:
 | Windows Support       | No                                                  | Yes                                                                                     |
 | Linux Support         | Yes                                                 | Yes                                                                                     |
 | MacOS Support         | Yes                                                 | Yes                                                                                     |
-| Number of Models      | [61](https://ollama.ai/library) +Any GGUF converted | [25](https://github.com/lmstudio-ai/model-catalog/tree/main/models) +Any GGUF Converted |
+| Number of Models      | [61](https://ollama.com/search) +Any GGUF converted | [25](https://github.com/lmstudio-ai/model-catalog/tree/main/models) +Any GGUF Converted |
 
 | Model Support   | Ollama | LM Studio |
 | --------------- | ------ | --------- |

diff --git a/dotnet/docs/MODELS.md b/dotnet/docs/MODELS.md
@@ -8,7 +8,7 @@ In the core Semantic Kernel repo, we plan on supporting up to four deployment ty
 
 - Dedicated API endpoints (e.g., OpenAI's APIs, Mistral.AI, and Google Gemini)
 - Azure AI deployments via the [model catalog](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/model-catalog)
-- Local deployments via [Ollama](https://ollama.ai/library)
+- Local deployments via [Ollama](https://ollama.ai/)
 - Hugging face deployment using the [Hugging Face inference API](https://huggingface.co/docs/api-inference/index)
 
 To support these different deployment types, we will follow a similar pattern to the Azure OpenAI and OpenAI connectors. Each connector uses the same underlying model and abstractions, but the connector constructors may take different parameters. For example, the Azure OpenAI connector expects an Azure endpoint and key, whereas the OpenAI connector expects an OpenAI organization ID and API key.
@@ -23,25 +23,25 @@ Please note that not all of the model interfaces are defined yet. As part of con
 
 ### OpenAI
 
-| Priority | Model                   | Status      | Interface                      | Deployment type | GitHub issue | Developer   | Reviewer |
-| -------- | ----------------------- | ----------- | ------------------------------ | --------------- | ------------ | ----------- | -------- |
-| P0       | GPT-3.5-turbo           | Complete    | `IChatCompletion`              | OpenAI API      | N/A          | N/A         | N/A      |
-| P0       | GPT-3.5-turbo           | Complete    | `IChatCompletion`              | Azure AI        | N/A          | N/A         | N/A      |
-| P0       | GPT-4                   | Complete    | `IChatCompletion`              | OpenAI API      | N/A          | N/A         | N/A      |
-| P0       | GPT-4                   | Complete    | `IChatCompletion`              | Azure AI        | N/A          | N/A         | N/A      |
-| P0       | GPT-4v                  | Complete    | `IChatCompletion`              | OpenAI API      | N/A          | N/A         | N/A      |
-| P0       | GPT-4v                  | Complete    | `IChatCompletion`              | Azure AI        | N/A          | N/A         | N/A      |
-| P0       | text-embedding-ada-002  | Preview     | `IEmbeddingGeneration`         | OpenAI API      | N/A          | N/A         | N/A      |
-| P0       | text-embedding-ada-002  | Preview     | `IEmbeddingGeneration`         | Azure AI        | N/A          | N/A         | N/A      |
-| P0       | DALL·E 3                | Preview     | `ITextToImage`                 | OpenAI API      | N/A          | N/A         | N/A      |
-| P0       | DALL·E 3                | Preview     | `ITextToImage`                 | Azure AI        | N/A          | N/A         | N/A      |
-| P0       | Text-to-speech          | Complete    | `ITextToSpeech`                | OpenAI API      | TBD          | dmytrostruk | TBD      |
-| P0       | Speech-to-text          | Complete    | `ISpeechRecognition`           | OpenAI API      | TBD          | dmytrostruk | TBD      |
-| P1       | openai-whisper-large-v3 | Not started | `ISpeechRecognition`           | Azure AI        | TBD          | TBD         | TBD      |
-| P1       | openai-whisper-large-v3 | Not started | `ISpeechRecognition`           | Hugging Face    | TBD          | TBD         | TBD      |
+| Priority | Model                   | Status      | Interface                      | Deployment type | GitHub issue | Developer    | Reviewer    |
+| -------- | ----------------------- | ----------- | ------------------------------ | --------------- | ------------ | ------------ | ----------- |
+| P0       | GPT-3.5-turbo           | Complete    | `IChatCompletion`              | OpenAI API      | N/A          | N/A          | N/A         |
+| P0       | GPT-3.5-turbo           | Complete    | `IChatCompletion`              | Azure AI        | N/A          | N/A          | N/A         |
+| P0       | GPT-4                   | Complete    | `IChatCompletion`              | OpenAI API      | N/A          | N/A          | N/A         |
+| P0       | GPT-4                   | Complete    | `IChatCompletion`              | Azure AI        | N/A          | N/A          | N/A         |
+| P0       | GPT-4v                  | Complete    | `IChatCompletion`              | OpenAI API      | N/A          | N/A          | N/A         |
+| P0       | GPT-4v                  | Complete    | `IChatCompletion`              | Azure AI        | N/A          | N/A          | N/A         |
+| P0       | text-embedding-ada-002  | Preview     | `IEmbeddingGeneration`         | OpenAI API      | N/A          | N/A          | N/A         |
+| P0       | text-embedding-ada-002  | Preview     | `IEmbeddingGeneration`         | Azure AI        | N/A          | N/A          | N/A         |
+| P0       | DALL·E 3                | Preview     | `ITextToImage`                 | OpenAI API      | N/A          | N/A          | N/A         |
+| P0       | DALL·E 3                | Preview     | `ITextToImage`                 | Azure AI        | N/A          | N/A          | N/A         |
+| P0       | Text-to-speech          | Complete    | `ITextToSpeech`                | OpenAI API      | TBD          | dmytrostruk  | TBD         |
+| P0       | Speech-to-text          | Complete    | `ISpeechRecognition`           | OpenAI API      | TBD          | dmytrostruk  | TBD         |
+| P1       | openai-whisper-large-v3 | Not started | `ISpeechRecognition`           | Azure AI        | TBD          | TBD          | TBD         |
+| P1       | openai-whisper-large-v3 | Not started | `ISpeechRecognition`           | Hugging Face    | TBD          | TBD          | TBD         |
 | P2       | Moderation              | In Progress | `ITextClassification`          | OpenAI API      | #5062        | Krzysztof318 | MarkWallace |
-| P2       | clip-vit-base-patch32   | Not started | `IZeroShotImageClassification` | Azure AI        | TBD          | TBD         | TBD      |
-| P2       | clip-vit-base-patch32   | Not started | `IZeroShotImageClassification` | Hugging Face    | TBD          | TBD         | TBD      |
+| P2       | clip-vit-base-patch32   | Not started | `IZeroShotImageClassification` | Azure AI        | TBD          | TBD          | TBD         |
+| P2       | clip-vit-base-patch32   | Not started | `IZeroShotImageClassification` | Hugging Face    | TBD          | TBD          | TBD         |
 
 ### Microsoft
 

diff --git a/python/semantic_kernel/agents/open_ai/assistant_content_generation.py b/python/semantic_kernel/agents/open_ai/assistant_content_generation.py
@@ -85,17 +85,19 @@ def get_message_contents(message: "ChatMessageContent") -> list[dict[str, Any]]:
     """
     contents: list[dict[str, Any]] = []
     for content in message.items:
-        if isinstance(content, TextContent):
-            contents.append({"type": "text", "text": content.text})
-        elif isinstance(content, ImageContent) and content.uri:
-            contents.append(content.to_dict())
-        elif isinstance(content, FileReferenceContent):
-            contents.append({
-                "type": "image_file",
-                "image_file": {"file_id": content.file_id},
-            })
-        elif isinstance(content, FunctionResultContent):
-            contents.append({"type": "text", "text": content.result})
+        match content:
+            case TextContent():
+                contents.append({"type": "text", "text": content.text})
+            case ImageContent():
+                if content.uri:
+                    contents.append(content.to_dict())
+            case FileReferenceContent():
+                contents.append({
+                    "type": "image_file",
+                    "image_file": {"file_id": content.file_id},
+                })
+            case FunctionResultContent():
+                contents.append({"type": "text", "text": content.result})
     return contents
 
 

diff --git a/python/semantic_kernel/agents/open_ai/azure_assistant_agent.py b/python/semantic_kernel/agents/open_ai/azure_assistant_agent.py
@@ -54,11 +54,11 @@ def __init__(
         enable_code_interpreter: bool | None = None,
         enable_file_search: bool | None = None,
         enable_json_response: bool | None = None,
-        file_ids: list[str] | None = [],
+        file_ids: list[str] | None = None,
         temperature: float | None = None,
         top_p: float | None = None,
         vector_store_id: str | None = None,
-        metadata: dict[str, Any] | None = {},
+        metadata: dict[str, Any] | None = None,
         max_completion_tokens: int | None = None,
         max_prompt_tokens: int | None = None,
         parallel_tool_calls_enabled: bool | None = True,
@@ -150,11 +150,11 @@ def __init__(
             "enable_code_interpreter": enable_code_interpreter,
             "enable_file_search": enable_file_search,
             "enable_json_response": enable_json_response,
-            "file_ids": file_ids,
+            "file_ids": file_ids or [],
             "temperature": temperature,
             "top_p": top_p,
             "vector_store_id": vector_store_id,
-            "metadata": metadata,
+            "metadata": metadata or {},
             "max_completion_tokens": max_completion_tokens,
             "max_prompt_tokens": max_prompt_tokens,
             "parallel_tool_calls_enabled": parallel_tool_calls_enabled,
@@ -199,7 +199,7 @@ async def create(
         temperature: float | None = None,
         top_p: float | None = None,
         vector_store_id: str | None = None,
-        metadata: dict[str, Any] | None = {},
+        metadata: dict[str, Any] | None = None,
         max_completion_tokens: int | None = None,
         max_prompt_tokens: int | None = None,
         parallel_tool_calls_enabled: bool | None = True,
@@ -268,7 +268,7 @@ async def create(
             temperature=temperature,
             top_p=top_p,
             vector_store_id=vector_store_id,
-            metadata=metadata,
+            metadata=metadata or {},
             max_completion_tokens=max_completion_tokens,
             max_prompt_tokens=max_prompt_tokens,
             parallel_tool_calls_enabled=parallel_tool_calls_enabled,

diff --git a/python/semantic_kernel/agents/open_ai/function_action_result.py b/python/semantic_kernel/agents/open_ai/function_action_result.py
@@ -0,0 +1,19 @@
+# Copyright (c) Microsoft. All rights reserved.
+
+import logging
+from dataclasses import dataclass
+
+from semantic_kernel.contents.chat_message_content import ChatMessageContent
+from semantic_kernel.utils.experimental_decorator import experimental_class
+
+logger: logging.Logger = logging.getLogger(__name__)
+
+
+@experimental_class
+@dataclass
+class FunctionActionResult:
+    """Function Action Result."""
+
+    function_call_content: ChatMessageContent | None
+    function_result_content: ChatMessageContent | None
+    tool_outputs: list[dict[str, str]] | None
diff --git a/python/semantic_kernel/agents/open_ai/open_ai_assistant_agent.py b/python/semantic_kernel/agents/open_ai/open_ai_assistant_agent.py
@@ -50,11 +50,11 @@ def __init__(
         enable_code_interpreter: bool | None = None,
         enable_file_search: bool | None = None,
         enable_json_response: bool | None = None,
-        code_interpreter_file_ids: list[str] | None = [],
+        code_interpreter_file_ids: list[str] | None = None,
         temperature: float | None = None,
         top_p: float | None = None,
         vector_store_id: str | None = None,
-        metadata: dict[str, Any] | None = {},
+        metadata: dict[str, Any] | None = None,
         max_completion_tokens: int | None = None,
         max_prompt_tokens: int | None = None,
         parallel_tool_calls_enabled: bool | None = True,
@@ -125,11 +125,11 @@ def __init__(
             "enable_code_interpreter": enable_code_interpreter,
             "enable_file_search": enable_file_search,
             "enable_json_response": enable_json_response,
-            "code_interpreter_file_ids": code_interpreter_file_ids,
+            "code_interpreter_file_ids": code_interpreter_file_ids or [],
             "temperature": temperature,
             "top_p": top_p,
             "vector_store_id": vector_store_id,
-            "metadata": metadata,
+            "metadata": metadata or {},
             "max_completion_tokens": max_completion_tokens,
             "max_prompt_tokens": max_prompt_tokens,
             "parallel_tool_calls_enabled": parallel_tool_calls_enabled,
@@ -173,7 +173,7 @@ async def create(
         temperature: float | None = None,
         top_p: float | None = None,
         vector_store_id: str | None = None,
-        metadata: dict[str, Any] | None = {},
+        metadata: dict[str, Any] | None = None,
         max_completion_tokens: int | None = None,
         max_prompt_tokens: int | None = None,
         parallel_tool_calls_enabled: bool | None = True,
@@ -236,7 +236,7 @@ async def create(
             temperature=temperature,
             top_p=top_p,
             vector_store_id=vector_store_id,
-            metadata=metadata,
+            metadata=metadata or {},
             max_completion_tokens=max_completion_tokens,
             max_prompt_tokens=max_prompt_tokens,
             parallel_tool_calls_enabled=parallel_tool_calls_enabled,