Model API doesn't retrieve Azure OpenAI deployments

[mspronesti]

OpenAI version: 1.3.7
Python version: 3.10

I'm trying to access the deployments associated with a specific Azure OpenAI subscription as follows

from openai import AzureOpenAI

client = AzureOpenAI(
    api_version="2023-08-01-preview",
    azure_endpoint="https://xxx.openai.azure.com/",
    api_key="xxx",
)

models = client.models.list()
print(models)
for model in models:
    print(model.id)

However this returns a list of all the models available, not those I deployed (with their specific deployment name).

ada
babbage
curie
davinci
gpt-35-turbo-0301
gpt-35-turbo-0613
gpt-35-turbo-1106
gpt-4-1106-Preview
gpt-35-turbo-instruct-0914
gpt-35-turbo-16k-0613
gpt-4-0314
gpt-4-0613
gpt-4-32k-0314
gpt-4-32k-0613
text-davinci-003
text-similarity-ada-001
text-similarity-babbage-001
text-similarity-curie-001
text-similarity-davinci-001
text-search-ada-doc-001
text-search-ada-query-001
text-search-babbage-doc-001
text-search-babbage-query-001
text-search-curie-doc-001
text-search-curie-query-001
text-search-davinci-doc-001
text-search-davinci-query-001
code-search-ada-code-001
code-search-ada-text-001
code-search-babbage-code-001
code-search-babbage-text-001
text-embedding-ada-002
text-embedding-ada-002-2
gpt-35-turbo
gpt-35-turbo-instruct
gpt-35-turbo-16k
gpt-4
gpt-4-32k
text-embedding-ada-002

How can I achieve the desired outcome? Thanks in advance :)

[rattrayalex]

cc @pamelafox

[rattrayalex]

cc @kristapratico

[mspronesti]

Any updates on this?

[pamelafox]

Sorry for delay, was just making sure I had the right answer. I believe you need to use this SDK for listing deployments:
https://pypi.org/project/azure-mgmt-cognitiveservices/

You'd initialize with this:
https://learn.microsoft.com/en-us/python/api/azure-mgmt-cognitiveservices/azure.mgmt.cognitiveservices.cognitiveservicesmanagementclient?view=azure-python

And then looks like the method to list deployments:
https://learn.microsoft.com/en-us/python/api/azure-mgmt-cognitiveservices/azure.mgmt.cognitiveservices.operations.deploymentsoperations?view=azure-python#azure-mgmt-cognitiveservices-operations-deploymentsoperations-list

It might require more filtering to get OpenAI specific deployments. I haven't used this particular API myself.

Background: For Azure resources, we generally distinguish between the "control plane" and the "data plane", where the control plane does stuff like create new Azure resources and the data plane actually uses those resources. In this case, the control plane is taken care of by azure-mgmt-cognitiveservices and the data plane by the openai package.

I've passed feedback onto the team that this is not a clear developer experience.

[kristapratico]

Deployments APIs were deprecated in the v0.X version of the openai library and so they were not added for v1.X. For all code managing deployments, please use the azure-mgmt-cognitiveservices client library. This library has additional options for working with deployments, like setting a deployment name or specific model version, and is the place where new features for deployments will be added.

Here are some samples for interacting with deployments:

Create a deployment
Retrieve a deployment
List deployments
Delete a deployment

[dominikriedermbw]

Eventhough it offers a "workaround", it is not really ideal having to provide some Tenant ID and client secrets...

It would be better if we could simply retrieve this information only with the azure openai api key and the resource name.....

[jnu]

It makes sense to separate the control plane from the data plane, but I don't understand why the list of base models (and not model deployments) is returned in the data plane, as the base models are essentially useless to the data plane. For example, the only valid values for the model field in the data plane API are deployment names, not base models. So the behavior of models.list in AzureOpenAI is not symmetrical with OpenAI.

[achandmsft]

@rattrayalex / @mspronesti Could you please let us know what you plan to do with the models (/deployments) returned once listed using this SDK? As in what is your overarching use case/job to be done to list this? Fine tuning or something else?

[mspronesti]

Hi there, the easiest thing that comes to mind is the validation of the provided deployment name (e.g. when writing a class on top of AzureOpenAI). I'm not sure I understand why the model API does not have a symmetric behavior as it used to have before ('til version 2022-05-12, I think). It would be nice to have it back unless I'm missing any critical drawbacks :)

[potatoqualitee]

I'd also like it to make things easy for end-users who didnt specify a deployment to begin with. then i can offer them a list.

[jnu]

Isn't it the case that everywhere the model parameter is required in the SDK it actually requires a model deployment? Whether it's completions or fine-tuning. This makes it frustrating that models#list returns the base models, which AFAIK are never useful in this SDK (and only useful to the azure-mgmt-cognitiveservices library).