104 lines
3.3 KiB
Markdown
104 lines
3.3 KiB
Markdown
# Prompt‑Clustering Utility
|
||
|
||
This repository contains a small utility (`cluster_prompts.py`) that embeds a
|
||
list of prompts with the OpenAI Embedding API, discovers natural groupings with
|
||
unsupervised clustering, lets ChatGPT name & describe each cluster and finally
|
||
produces a concise Markdown report plus a couple of diagnostic plots.
|
||
|
||
The default input file (`prompts.csv`) ships with the repo so you can try the
|
||
script immediately, but you can of course point it at your own file.
|
||
|
||
---
|
||
|
||
## 1. Setup
|
||
|
||
1. Install the Python dependencies (preferably inside a virtual env):
|
||
|
||
```bash
|
||
pip install pandas numpy scikit-learn matplotlib openai
|
||
```
|
||
|
||
2. Export your OpenAI API key (**required**):
|
||
|
||
```bash
|
||
export OPENAI_API_KEY="sk‑..."
|
||
```
|
||
|
||
---
|
||
|
||
## 2. Basic usage
|
||
|
||
```bash
|
||
# Minimal command – runs on prompts.csv and writes analysis.md + plots/
|
||
python cluster_prompts.py
|
||
```
|
||
|
||
This will
|
||
|
||
* create embeddings with the `text-embedding-3-small` model,
|
||
* pick a suitable number *k* via silhouette score (K‑Means),
|
||
* ask `gpt‑4o‑mini` to label & describe each cluster,
|
||
* store the results in `analysis.md`,
|
||
* and save two plots to `plots/` (`cluster_sizes.png` and `tsne.png`).
|
||
|
||
The script prints a short success message once done.
|
||
|
||
---
|
||
|
||
## 3. Command‑line options
|
||
|
||
| flag | default | description |
|
||
|------|---------|-------------|
|
||
| `--csv` | `prompts.csv` | path to the input CSV (must contain a `prompt` column; an `act` column is used as context if present) |
|
||
| `--cache` | _(none)_ | embedding cache path (JSON). Speeds up repeated runs – new texts are appended automatically. |
|
||
| `--cluster-method` | `kmeans` | `kmeans` (with automatic *k*) or `dbscan` |
|
||
| `--k-max` | `10` | upper bound for *k* when `kmeans` is selected |
|
||
| `--dbscan-min-samples` | `3` | min samples parameter for DBSCAN |
|
||
| `--embedding-model` | `text-embedding-3-small` | any OpenAI embedding model |
|
||
| `--chat-model` | `gpt-4o-mini` | chat model used to generate cluster names / descriptions |
|
||
| `--output-md` | `analysis.md` | where to write the Markdown report |
|
||
| `--plots-dir` | `plots` | directory for generated PNGs |
|
||
|
||
Example with customised options:
|
||
|
||
```bash
|
||
python cluster_prompts.py \
|
||
--csv my_prompts.csv \
|
||
--cache .cache/embeddings.json \
|
||
--cluster-method dbscan \
|
||
--embedding-model text-embedding-3-large \
|
||
--chat-model gpt-4o \
|
||
--output-md my_analysis.md \
|
||
--plots-dir my_plots
|
||
```
|
||
|
||
---
|
||
|
||
## 4. Interpreting the output
|
||
|
||
### analysis.md
|
||
|
||
* Overview table: cluster label, generated name, member count and description.
|
||
* Detailed section for every cluster with five representative example prompts.
|
||
* Separate lists for
|
||
* **Noise / outliers** (label `‑1` when DBSCAN is used) and
|
||
* **Potentially ambiguous prompts** (only with K‑Means) – these are items that
|
||
lie almost equally close to two centroids and might belong to multiple
|
||
groups.
|
||
|
||
### plots/cluster_sizes.png
|
||
|
||
Quick bar‑chart visualisation of how many prompts ended up in each cluster.
|
||
|
||
---
|
||
|
||
## 5. Troubleshooting
|
||
|
||
* **Rate‑limits / quota errors** – lower the number of prompts per run or switch
|
||
to a larger quota account.
|
||
* **Authentication errors** – make sure `OPENAI_API_KEY` is exported in the
|
||
shell where you run the script.
|
||
* **Inadequate clusters** – try the other clustering method, adjust `--k-max`
|
||
or tune DBSCAN parameters (`eps` range is inferred, `min_samples` exposed via
|
||
CLI).
|