feat: context compaction (#3446)

## Compact feature:
1. Stops the model when the context window become too large
2. Add a user turn, asking for the model to summarize
3. Build a bridge that contains all the previous user message + the
summary. Rendered from a template
4. Start sampling again from a clean conversation with only that bridge
This commit is contained in:
jif-oai
2025-09-12 13:07:10 -07:00
committed by GitHub
parent d4848e558b
commit ea225df22e
14 changed files with 1243 additions and 326 deletions

View File

@@ -13,6 +13,7 @@ workspace = true
[dependencies]
anyhow = "1"
askama = "0.12"
async-channel = "2.3.1"
base64 = "0.22"
bytes = "1.10.1"