Improvement: Chat Feature Tracker

[rmusser01]

Issue is to track improvements to the chat Functionality

• chat history is summarized with each new message and only the summary + the last N messages are sent to the model instead of the whole chat
• Chat branching + swappign branches
• Prompt template presets
  - Jinja templates?
• Sampling parameter presets
• Streaming response handling
• Lorebook management
• Prioritization of inclusion of messages/DBs for context - level 1, level 2, level 3
• Depth anchors for lorebooks

[rmusser01]

https://abbey.us.ai/

SillyTavern/SillyTavern#2189

At present, we can choose the specific depth of lorebook entries, but we cannot rapidly change the depth of most lorebook entries. This becomes a problem when switching models, because some models are good at reading information at higher depths, while others require information at much lower depths.

For example, lets say that I have a page full of lorebook entries at various depths. I want to subtract 2 from the depth of all of my lorebook entries as I switch models. For some of those lorebook entries, I could just tether them to the author's note and change the depth of the author's note. However, my lorebook entries span a wide variety of depths. Important information is set to a lower depth, while more general information is set to a higher depth.

So, if I tether the author's note to 'critically important information', then I'll still need to manually change the depth of all lorebook entries that are not 'critically important information'. This requires going through each entry one by one, and can be a very time consuming process.
Describe the solution you'd like

I propose the creation of new depth categories. That way, instead of setting the depth of each individual entry, I could instead tether the depth of each lorebook entry to a global category, similar to how many use the author's note category.

A few category suggestions:

    Directive information
    Priority information
    Important Information
    Scenario Information
    General Information

Using these examples, I could set directive information to (D1), priority information to (D2), important information to (D5), scenario information to (D7), and general information to (D10).

Using this system I could rapidly change the depth of all relevant entries by changing the depth of a single global variable. Ex, if a model doesn't seem to be reading my 'general information' very well, then I could set the depth to (D8) instead of (D10). If my priority information seems to be stifling my model, I could change it to (D3) instead of (D2).

I am planning to tackle this feature soon. You will be able to choose "Depth Anchors" that you can define as many as you want and name how you want. Each anchor will have a name, an attached role and depth setting. And the WI can connect to an anchor.
If you have a few dozen WI connected to an anchor for example called "Directive Information" at depth 5 and then change that to depth 3, next generation all WI with that anchor will be inserted together at depth 3 instead of 5.

[rmusser01]

https://arxiv.org/abs/2411.07641