Correctly calculate remaining context size (#3190)

We had multiple issues with context size calculation:
1. `initial_prompt_tokens` calculation based on cache size is not
reliable, cache misses might set it to much higher value. For now
hardcoded to a safer constant.
2. Input context size for GPT-5 is 272k (that's where 33% came from).

Fixes.
This commit is contained in:
pakrym-oai
2025-09-04 16:34:14 -07:00
committed by GitHub
parent b795fbe244
commit 7df9e9c664
4 changed files with 12 additions and 32 deletions

View File

@@ -1382,7 +1382,7 @@ model_verbosity = "high"
let expected_gpt5_profile_config = Config {
model: "gpt-5".to_string(),
model_family: find_family_for_model("gpt-5").expect("known model slug"),
model_context_window: Some(400_000),
model_context_window: Some(272_000),
model_max_output_tokens: Some(128_000),
model_provider_id: "openai".to_string(),
model_provider: fixture.openai_provider.clone(),

View File

@@ -79,12 +79,12 @@ pub(crate) fn get_model_info(model_family: &ModelFamily) -> Option<ModelInfo> {
}),
"gpt-5" => Some(ModelInfo {
context_window: 400_000,
context_window: 272_000,
max_output_tokens: 128_000,
}),
_ if slug.starts_with("codex-") => Some(ModelInfo {
context_window: 400_000,
context_window: 272_000,
max_output_tokens: 128_000,
}),