valknar/llmx
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: @mention files in codex (#701)

Browse Source 
Solves #700

## State of the World Before

Prior to this PR, when users wanted to share file contents with Codex,
they had two options:
- Manually copy and paste file contents into the chat
- Wait for the assistant to use the shell tool to view the file

The second approach required the assistant to:
1. Recognize the need to view a file
2. Execute a shell tool call
3. Wait for the tool call to complete
4. Process the file contents

This consumed extra tokens and reduced user control over which files
were shared with the model.

## State of the World After

With this PR, users can now:
- Reference files directly in their chat input using the `@path` syntax
- Have file contents automatically expanded into XML blocks before being
sent to the LLM

For example, users can type `@src/utils/config.js` in their message, and
the file contents will be included in context. Within the terminal chat
history, these file blocks will be collapsed back to `@path` format in
the UI for clean presentation.

Tag File suggestions:
<img width="857" alt="file-suggestions"
src="https://github.com/user-attachments/assets/397669dc-ad83-492d-b5f0-164fab2ff4ba"
/>

Tagging files in action:
<img width="858" alt="tagging-files"
src="https://github.com/user-attachments/assets/0de9d559-7b7f-4916-aeff-87ae9b16550a"
/>

Demo video of file tagging:
[![Demo video of file
tagging](https://img.youtube.com/vi/vL4LqtBnqt8/0.jpg)](https://www.youtube.com/watch?v=vL4LqtBnqt8)

## Implementation Details

This PR consists of 2 main components:

1. **File Tag Utilities**:
- New `file-tag-utils.ts` utility module that handles both expansion and
collapsing of file tags
- `expandFileTags()` identifies `@path` tokens and replaces them with
XML blocks containing file contents
- `collapseXmlBlocks()` reverses the process, converting XML blocks back
to `@path` format for UI display
- Tokens are only expanded if they point to valid files (directories are
ignored)
   - Expansion happens just before sending input to the model

2. **Terminal Chat Integration**:
- Leveraged the existing file system completion system for tabbing to
support the `@path` syntax
   - Added `updateFsSuggestions` helper to manage filesystem suggestions
- Added `replaceFileSystemSuggestion` to replace input with filesystem
suggestions
- Applied `collapseXmlBlocks` in the chat response rendering so that
tagged files are shown as simple `@path` tags

The PR also includes test coverage for both the UI and the file tag
utilities.

## Next Steps

Some ideas I'd like to implement if this feature gets merged:

- Line selection: `@path[50:80]` to grab specific sections of files
- Method selection: `@path#methodName` to grab just one function/class
- Visual improvements: highlight file tags in the UI to make them more
noticeable
This commit is contained in:
moppywhip
2025-04-30 19:19:55 -04:00
committed by GitHub
parent bd82101859
commit bc4e6db749
10 changed files with 771 additions and 62 deletions
Download Patch File Download Diff File Expand all files Collapse all files

6
codex-cli/src/components/chat/multiline-editor.tsx
View File

@@ -137,6 +137,9 @@ export interface MultilineTextEditorProps {
// Called when the internal text buffer updates.
readonly onChange?: (text: string) => void;
// Optional initial cursor position (character offset)
readonly initialCursorOffset?: number;
}
// Expose a minimal imperative API so parent components (e.g. TerminalChatInput)
@@ -169,6 +172,7 @@ const MultilineTextEditorInner = (
onSubmit,
focus = true,
onChange,
initialCursorOffset,
}: MultilineTextEditorProps,
ref: React.Ref<MultilineTextEditorHandle | null>,
): React.ReactElement => {
@@ -176,7 +180,7 @@ const MultilineTextEditorInner = (
// Editor State
// ---------------------------------------------------------------------------
const buffer = useRef(new TextBuffer(initialText));
const buffer = useRef(new TextBuffer(initialText, initialCursorOffset));
const [version, setVersion] = useState(0);
// Keep track of the current terminal size so that the editor grows/shrinks

195
codex-cli/src/components/chat/terminal-chat-input.tsx
View File

@@ -1,5 +1,6 @@
import type { MultilineTextEditorHandle } from "./multiline-editor";
import type { ReviewDecision } from "../../utils/agent/review.js";
import type { FileSystemSuggestion } from "../../utils/file-system-suggestions.js";
import type { HistoryEntry } from "../../utils/storage/command-history.js";
import type {
ResponseInputItem,
@@ -11,6 +12,7 @@ import { TerminalChatCommandReview } from "./terminal-chat-command-review.js";
import TextCompletions from "./terminal-chat-completions.js";
import { loadConfig } from "../../utils/config.js";
import { getFileSystemSuggestions } from "../../utils/file-system-suggestions.js";
import { expandFileTags } from "../../utils/file-tag-utils";
import { createInputItem } from "../../utils/input-utils.js";
import { log } from "../../utils/logger/log.js";
import { setSessionId } from "../../utils/session.js";
@@ -92,16 +94,120 @@ export default function TerminalChatInput({
const [historyIndex, setHistoryIndex] = useState<number | null>(null);
const [draftInput, setDraftInput] = useState<string>("");
const [skipNextSubmit, setSkipNextSubmit] = useState<boolean>(false);
const [fsSuggestions, setFsSuggestions] = useState<Array<string>>([]);
const [fsSuggestions, setFsSuggestions] = useState<
Array<FileSystemSuggestion>
>([]);
const [selectedCompletion, setSelectedCompletion] = useState<number>(-1);
// Multiline text editor key to force remount after submission
const [editorKey, setEditorKey] = useState(0);
const [editorState, setEditorState] = useState<{
key: number;
initialCursorOffset?: number;
}>({ key: 0 });
// Imperative handle from the multiline editor so we can query caret position
const editorRef = useRef<MultilineTextEditorHandle | null>(null);
// Track the caret row across keystrokes
const prevCursorRow = useRef<number | null>(null);
const prevCursorWasAtLastRow = useRef<boolean>(false);
// --- Helper for updating input, remounting editor, and moving cursor to end ---
const applyFsSuggestion = useCallback((newInputText: string) => {
setInput(newInputText);
setEditorState((s) => ({
key: s.key + 1,
initialCursorOffset: newInputText.length,
}));
}, []);
// --- Helper for updating file system suggestions ---
function updateFsSuggestions(
txt: string,
alwaysUpdateSelection: boolean = false,
) {
// Clear file system completions if a space is typed
if (txt.endsWith(" ")) {
setFsSuggestions([]);
setSelectedCompletion(-1);
} else {
// Determine the current token (last whitespace-separated word)
const words = txt.trim().split(/\s+/);
const lastWord = words[words.length - 1] ?? "";
const shouldUpdateSelection =
lastWord.startsWith("@") || alwaysUpdateSelection;
// Strip optional leading '@' for the path prefix
let pathPrefix: string;
if (lastWord.startsWith("@")) {
pathPrefix = lastWord.slice(1);
// If only '@' is typed, list everything in the current directory
pathPrefix = pathPrefix.length === 0 ? "./" : pathPrefix;
} else {
pathPrefix = lastWord;
}
if (shouldUpdateSelection) {
const completions = getFileSystemSuggestions(pathPrefix);
setFsSuggestions(completions);
if (completions.length > 0) {
setSelectedCompletion((prev) =>
prev < 0 || prev >= completions.length ? 0 : prev,
);
} else {
setSelectedCompletion(-1);
}
} else if (fsSuggestions.length > 0) {
// Token cleared → clear menu
setFsSuggestions([]);
setSelectedCompletion(-1);
}
}
}
/**
* Result of replacing text with a file system suggestion
*/
interface ReplacementResult {
/** The new text with the suggestion applied */
text: string;
/** The selected suggestion if a replacement was made */
suggestion: FileSystemSuggestion | null;
/** Whether a replacement was actually made */
wasReplaced: boolean;
}
// --- Helper for replacing input with file system suggestion ---
function getFileSystemSuggestion(
txt: string,
requireAtPrefix: boolean = false,
): ReplacementResult {
if (fsSuggestions.length === 0 || selectedCompletion < 0) {
return { text: txt, suggestion: null, wasReplaced: false };
}
const words = txt.trim().split(/\s+/);
const lastWord = words[words.length - 1] ?? "";
// Check if @ prefix is required and the last word doesn't have it
if (requireAtPrefix && !lastWord.startsWith("@")) {
return { text: txt, suggestion: null, wasReplaced: false };
}
const selected = fsSuggestions[selectedCompletion];
if (!selected) {
return { text: txt, suggestion: null, wasReplaced: false };
}
const replacement = lastWord.startsWith("@")
? `@${selected.path}`
: selected.path;
words[words.length - 1] = replacement;
return {
text: words.join(" "),
suggestion: selected,
wasReplaced: true,
};
}
// Load command history on component mount
useEffect(() => {
async function loadHistory() {
@@ -223,21 +329,12 @@ export default function TerminalChatInput({
}
if (_key.tab && selectedCompletion >= 0) {
const words = input.trim().split(/\s+/);
const selected = fsSuggestions[selectedCompletion];
if (words.length > 0 && selected) {
words[words.length - 1] = selected;
const newText = words.join(" ");
setInput(newText);
// Force remount of the editor with the new text
setEditorKey((k) => k + 1);
// We need to move the cursor to the end after editor remounts
setTimeout(() => {
editorRef.current?.moveCursorToEnd?.();
}, 0);
const { text: newText, wasReplaced } =
getFileSystemSuggestion(input);
// Only proceed if the text was actually changed
if (wasReplaced) {
applyFsSuggestion(newText);
setFsSuggestions([]);
setSelectedCompletion(-1);
}
@@ -277,7 +374,7 @@ export default function TerminalChatInput({
setInput(history[newIndex]?.command ?? "");
// Re-mount the editor so it picks up the new initialText
setEditorKey((k) => k + 1);
setEditorState((s) => ({ key: s.key + 1 }));
return; // handled
}
@@ -296,28 +393,23 @@ export default function TerminalChatInput({
if (newIndex >= history.length) {
setHistoryIndex(null);
setInput(draftInput);
setEditorKey((k) => k + 1);
setEditorState((s) => ({ key: s.key + 1 }));
} else {
setHistoryIndex(newIndex);
setInput(history[newIndex]?.command ?? "");
setEditorKey((k) => k + 1);
setEditorState((s) => ({ key: s.key + 1 }));
}
return; // handled
}
// Otherwise let it propagate
}
if (_key.tab) {
const words = input.split(/\s+/);
const mostRecentWord = words[words.length - 1];
if (mostRecentWord === undefined || mostRecentWord === "") {
return;
}
const completions = getFileSystemSuggestions(mostRecentWord);
setFsSuggestions(completions);
if (completions.length > 0) {
setSelectedCompletion(0);
}
// Defer filesystem suggestion logic to onSubmit if enter key is pressed
if (!_key.return) {
// Pressing tab should trigger the file system suggestions
const shouldUpdateSelection = _key.tab;
const targetInput = _key.delete ? input.slice(0, -1) : input + _input;
updateFsSuggestions(targetInput, shouldUpdateSelection);
}
}
@@ -599,7 +691,10 @@ export default function TerminalChatInput({
);
text = text.trim();
const inputItem = await createInputItem(text, images);
// Expand @file tokens into XML blocks for the model
const expandedText = await expandFileTags(text);
const inputItem = await createInputItem(expandedText, images);
submitInput([inputItem]);
// Get config for history persistence.
@@ -673,28 +768,30 @@ export default function TerminalChatInput({
setHistoryIndex(null);
}
setInput(txt);
// Clear tab completions if a space is typed
if (txt.endsWith(" ")) {
setFsSuggestions([]);
setSelectedCompletion(-1);
} else if (fsSuggestions.length > 0) {
// Update file suggestions as user types
const words = txt.trim().split(/\s+/);
const mostRecentWord =
words.length > 0 ? words[words.length - 1] : "";
if (mostRecentWord !== undefined) {
setFsSuggestions(getFileSystemSuggestions(mostRecentWord));
}
}
}}
key={editorKey}
key={editorState.key}
initialCursorOffset={editorState.initialCursorOffset}
initialText={input}
height={6}
focus={active}
onSubmit={(txt) => {
onSubmit(txt);
setEditorKey((k) => k + 1);
// If final token is an @path, replace with filesystem suggestion if available
const {
text: replacedText,
suggestion,
wasReplaced,
} = getFileSystemSuggestion(txt, true);
// If we replaced @path token with a directory, don't submit
if (wasReplaced && suggestion?.isDirectory) {
applyFsSuggestion(replacedText);
// Update suggestions for the new directory
updateFsSuggestions(replacedText, true);
return;
}
onSubmit(replacedText);
setEditorState((s) => ({ key: s.key + 1 }));
setInput("");
setHistoryIndex(null);
setDraftInput("");
@@ -741,7 +838,7 @@ export default function TerminalChatInput({
</Text>
) : fsSuggestions.length > 0 ? (
<TextCompletions
completions={fsSuggestions}
completions={fsSuggestions.map((suggestion) => suggestion.path)}
selectedCompletion={selectedCompletion}
displayLimit={5}
/>

3
codex-cli/src/components/chat/terminal-chat-response-item.tsx
View File

@@ -10,6 +10,7 @@ import type {
} from "openai/resources/responses/responses";
import { useTerminalSize } from "../../hooks/use-terminal-size";
import { collapseXmlBlocks } from "../../utils/file-tag-utils";
import { parseToolCall, parseToolCallOutput } from "../../utils/parsers";
import chalk, { type ForegroundColorName } from "chalk";
import { Box, Text } from "ink";
@@ -137,7 +138,7 @@ function TerminalChatResponseMessage({
: c.type === "refusal"
? c.refusal
: c.type === "input_text"
? c.text
? collapseXmlBlocks(c.text)
: c.type === "input_image"
? "<Image>"
: c.type === "input_file"