Ability to pass session context via runtime between agents

[ashishsingla87]

Hi,
The pieces I am trying to solve have this need to access specific things via session variables. For example, keys in global variable which I would like all of my agents to have access to. Or a pdf byte array which i can pass around. We can use s3 to save files and give ids for the file but I feel that is an inefficient way to solve the problem as if in case of hallucination, I would not have access to the variables which I can locate deterministically. If we have a thin layer of session variables as an added json on top of memory, that would be great to reduce these problems. Interested to hear thoughts of this community on this topic.

[ekzhu]

At the Core API, you can define your message protocol to contain the session context as part of the messages between agents. Are you thinking of the AgentChat layer?

[ashishsingla87]

At the Core API, you can define your message protocol to contain the session context as part of the messages between agents. Are you thinking of the AgentChat layer?

Do you have an example of this by any chance?

[ekzhu]

I don't have an example, but at the Core API level, it is application's responsibility to define the message protocols and all message types. You can take a look at the Core's documentation: https://microsoft.github.io/autogen/dev/user-guide/core-user-guide/index.html

[ashishsingla87]

So in a nutshell, when triggering a team.run method like roundrobinchat.run. I would like to pass a list of session variables which I would like my agent to have access to.

[ashishsingla87]

So for example I can pass my agent a deterministic value of a uuid can be an entry in my dynamodb. Some of the agents in my pipeline would not be llm agents but just chaining tools to provide something in a deterministic manner. This will help avoid the indeterministic nature of the solution by reducing reliability on llms where we don't need it.

[ashishsingla87]

So i have a roundrobin chat between 5 agents where one agent is just supposed to do document extraction for a given pdf and pass it's text along with the previous messages.

[ekzhu]

So you are creating your customized agents?

[ashishsingla87]

Currently testing it, will raise a PR based on how the testing goes. But the key question is if I can pass some data to agent on top of messages from previous message which is limited to session context.

[ekzhu]

Thanks. Perhaps a message attachment?