LLM Provider Settings

[Oceania2018]

Use a more flexible configuration data structure to support versatile LLM configuration. Azure now activates different Models and versions in different Regions. The Text and Chat versions are in different Endpoints, making it impossible to switch partially. We need a configuration structure that can automatically locate configurations based on Provider and Model. One structure I currently envision is:

{
    "LlmProviders": [{
	"Provider": "azure-openai",
	"Models": [{
	    "Name": "",
	    "ApiKey": "",
	    "Endpoint": "",
            "Type": "text/chat"
	}]
    }]
}

Through this configuration, we can also update the token / cost statistics of ITokenStatistics to apply this new configuration structure.

{
    "Name": "",
    "ApiKey": "",
    "Endpoint": "",
    "Type": "text/chat",
    "PromptCost": 0.0015,
    "CompletionCost": 0.002 
}