Files
llmx/codex-rs/tui/src/bottom_pane/chat_composer_history.rs

264 lines
8.9 KiB
Rust
Raw Normal View History

feat: record messages from user in ~/.codex/history.jsonl (#939) This is a large change to support a "history" feature like you would expect in a shell like Bash. History events are recorded in `$CODEX_HOME/history.jsonl`. Because it is a JSONL file, it is straightforward to append new entries (as opposed to the TypeScript file that uses `$CODEX_HOME/history.json`, so to be valid JSON, each new entry entails rewriting the entire file). Because it is possible for there to be multiple instances of Codex CLI writing to `history.jsonl` at once, we use advisory file locking when working with `history.jsonl` in `codex-rs/core/src/message_history.rs`. Because we believe history is a sufficiently useful feature, we enable it by default. Though to provide some safety, we set the file permissions of `history.jsonl` to be `o600` so that other users on the system cannot read the user's history. We do not yet support a default list of `SENSITIVE_PATTERNS` as the TypeScript CLI does: https://github.com/openai/codex/blob/3fdf9df1335ac9501e3fb0e61715359145711e8b/codex-cli/src/utils/storage/command-history.ts#L10-L17 We are going to take a more conservative approach to this list in the Rust CLI. For example, while `/\b[A-Za-z0-9-_]{20,}\b/` might exclude sensitive information like API tokens, it would also exclude valuable information such as references to Git commits. As noted in the updated documentation, users can opt-out of history by adding the following to `config.toml`: ```toml [history] persistence = "none" ``` Because `history.jsonl` could, in theory, be quite large, we take a[n arguably overly pedantic] approach in reading history entries into memory. Specifically, we start by telling the client the current number of entries in the history file (`history_entry_count`) as well as the inode (`history_log_id`) of `history.jsonl` (see the new fields on `SessionConfiguredEvent`). The client is responsible for keeping new entries in memory to create a "local history," but if the user hits up enough times to go "past" the end of local history, then the client should use the new `GetHistoryEntryRequest` in the protocol to fetch older entries. Specifically, it should pass the `history_log_id` it was given originally and work backwards from `history_entry_count`. (It should really fetch history in batches rather than one-at-a-time, but that is something we can improve upon in subsequent PRs.) The motivation behind this crazy scheme is that it is designed to defend against: * The `history.jsonl` being truncated during the session such that the index into the history is no longer consistent with what had been read up to that point. We do not yet have logic to enforce a `max_bytes` for `history.jsonl`, but once we do, we will aspire to implement it in a way that should result in a new inode for the file on most systems. * New items from concurrent Codex CLI sessions amending to the history. Because, in absence of truncation, `history.jsonl` is an append-only log, so long as the client reads backwards from `history_entry_count`, it should always get a consistent view of history. (That said, it will not be able to read _new_ commands from concurrent sessions, but perhaps we will introduce a `/` command to reload latest history or something down the road.) Admittedly, my testing of this feature thus far has been fairly light. I expect we will find bugs and introduce enhancements/fixes going forward.
2025-05-15 16:26:23 -07:00
use std::collections::HashMap;
use tui_textarea::CursorMove;
use tui_textarea::TextArea;
use crate::app_event::AppEvent;
use crate::app_event_sender::AppEventSender;
use codex_core::protocol::Op;
/// State machine that manages shell-style history navigation (Up/Down) inside
/// the chat composer. This struct is intentionally decoupled from the
/// rendering widget so the logic remains isolated and easier to test.
pub(crate) struct ChatComposerHistory {
/// Identifier of the history log as reported by `SessionConfiguredEvent`.
history_log_id: Option<u64>,
/// Number of entries already present in the persistent cross-session
/// history file when the session started.
history_entry_count: usize,
/// Messages submitted by the user *during this UI session* (newest at END).
local_history: Vec<String>,
/// Cache of persistent history entries fetched on-demand.
fetched_history: HashMap<usize, String>,
/// Current cursor within the combined (persistent + local) history. `None`
/// indicates the user is *not* currently browsing history.
history_cursor: Option<isize>,
/// The text that was last inserted into the composer as a result of
/// history navigation. Used to decide if further Up/Down presses should be
/// treated as navigation versus normal cursor movement.
last_history_text: Option<String>,
}
impl ChatComposerHistory {
pub fn new() -> Self {
Self {
history_log_id: None,
history_entry_count: 0,
local_history: Vec::new(),
fetched_history: HashMap::new(),
history_cursor: None,
last_history_text: None,
}
}
/// Update metadata when a new session is configured.
pub fn set_metadata(&mut self, log_id: u64, entry_count: usize) {
self.history_log_id = Some(log_id);
self.history_entry_count = entry_count;
self.fetched_history.clear();
self.local_history.clear();
self.history_cursor = None;
self.last_history_text = None;
}
/// Record a message submitted by the user in the current session so it can
/// be recalled later.
pub fn record_local_submission(&mut self, text: &str) {
if !text.is_empty() {
self.local_history.push(text.to_string());
self.history_cursor = None;
self.last_history_text = None;
}
}
/// Should Up/Down key presses be interpreted as history navigation given
/// the current content and cursor position of `textarea`?
pub fn should_handle_navigation(&self, textarea: &TextArea) -> bool {
if self.history_entry_count == 0 && self.local_history.is_empty() {
return false;
}
let lines = textarea.lines();
if lines.len() == 1 && lines[0].is_empty() {
return true;
}
// Textarea is not empty only navigate when cursor is at start and
// text matches last recalled history entry so regular editing is not
// hijacked.
let (row, col) = textarea.cursor();
if row != 0 || col != 0 {
return false;
}
matches!(&self.last_history_text, Some(prev) if prev == &lines.join("\n"))
}
/// Handle <Up>. Returns true when the key was consumed and the caller
/// should request a redraw.
pub fn navigate_up(&mut self, textarea: &mut TextArea, app_event_tx: &AppEventSender) -> bool {
let total_entries = self.history_entry_count + self.local_history.len();
if total_entries == 0 {
return false;
}
let next_idx = match self.history_cursor {
None => (total_entries as isize) - 1,
Some(0) => return true, // already at oldest
Some(idx) => idx - 1,
};
self.history_cursor = Some(next_idx);
self.populate_history_at_index(next_idx as usize, textarea, app_event_tx);
true
}
/// Handle <Down>.
pub fn navigate_down(
&mut self,
textarea: &mut TextArea,
app_event_tx: &AppEventSender,
) -> bool {
let total_entries = self.history_entry_count + self.local_history.len();
if total_entries == 0 {
return false;
}
let next_idx_opt = match self.history_cursor {
None => return false, // not browsing
Some(idx) if (idx as usize) + 1 >= total_entries => None,
Some(idx) => Some(idx + 1),
};
match next_idx_opt {
Some(idx) => {
self.history_cursor = Some(idx);
self.populate_history_at_index(idx as usize, textarea, app_event_tx);
}
None => {
// Past newest clear and exit browsing mode.
self.history_cursor = None;
self.last_history_text = None;
self.replace_textarea_content(textarea, "");
}
}
true
}
/// Integrate a GetHistoryEntryResponse event.
pub fn on_entry_response(
&mut self,
log_id: u64,
offset: usize,
entry: Option<String>,
textarea: &mut TextArea,
) -> bool {
if self.history_log_id != Some(log_id) {
return false;
}
let Some(text) = entry else { return false };
self.fetched_history.insert(offset, text.clone());
if self.history_cursor == Some(offset as isize) {
self.replace_textarea_content(textarea, &text);
return true;
}
false
}
// ---------------------------------------------------------------------
// Internal helpers
// ---------------------------------------------------------------------
fn populate_history_at_index(
&mut self,
global_idx: usize,
textarea: &mut TextArea,
app_event_tx: &AppEventSender,
) {
if global_idx >= self.history_entry_count {
// Local entry.
if let Some(text) = self
.local_history
.get(global_idx - self.history_entry_count)
{
let t = text.clone();
self.replace_textarea_content(textarea, &t);
}
} else if let Some(text) = self.fetched_history.get(&global_idx) {
let t = text.clone();
self.replace_textarea_content(textarea, &t);
} else if let Some(log_id) = self.history_log_id {
let op = Op::GetHistoryEntryRequest {
offset: global_idx,
log_id,
};
app_event_tx.send(AppEvent::CodexOp(op));
}
}
fn replace_textarea_content(&mut self, textarea: &mut TextArea, text: &str) {
textarea.select_all();
textarea.cut();
let _ = textarea.insert_str(text);
textarea.move_cursor(CursorMove::Jump(0, 0));
self.last_history_text = Some(text.to_string());
}
}
#[cfg(test)]
mod tests {
#![expect(clippy::expect_used)]
use super::*;
use crate::app_event::AppEvent;
use codex_core::protocol::Op;
use std::sync::mpsc::channel;
#[test]
fn navigation_with_async_fetch() {
let (tx, rx) = channel::<AppEvent>();
let tx = AppEventSender::new(tx);
let mut history = ChatComposerHistory::new();
// Pretend there are 3 persistent entries.
history.set_metadata(1, 3);
let mut textarea = TextArea::default();
// First Up should request offset 2 (latest) and await async data.
assert!(history.should_handle_navigation(&textarea));
assert!(history.navigate_up(&mut textarea, &tx));
// Verify that an AppEvent::CodexOp with the correct GetHistoryEntryRequest was sent.
let event = rx.try_recv().expect("expected AppEvent to be sent");
let AppEvent::CodexOp(history_request1) = event else {
panic!("unexpected event variant");
};
assert_eq!(
Op::GetHistoryEntryRequest {
log_id: 1,
offset: 2
},
history_request1
);
assert_eq!(textarea.lines().join("\n"), ""); // still empty
// Inject the async response.
assert!(history.on_entry_response(1, 2, Some("latest".into()), &mut textarea));
assert_eq!(textarea.lines().join("\n"), "latest");
// Next Up should move to offset 1.
assert!(history.navigate_up(&mut textarea, &tx));
// Verify second CodexOp event for offset 1.
let event2 = rx.try_recv().expect("expected second event");
let AppEvent::CodexOp(history_request_2) = event2 else {
panic!("unexpected event variant");
};
assert_eq!(
Op::GetHistoryEntryRequest {
log_id: 1,
offset: 1
},
history_request_2
);
history.on_entry_response(1, 1, Some("older".into()), &mut textarea);
assert_eq!(textarea.lines().join("\n"), "older");
}
}