Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions manual-testing/eliminated-candidate-tracking.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Eliminated Candidate Tracking

## Issue
Previously, when a candidate was eliminated from contention, the system would stop updating its upvotes and downvotes count. Additionally, eliminated candidates would not appear in the `matchingRegexes` field in the export, making it unclear which eliminated candidates matched specific words.

## Fix
Modified the `applyClassification` method in `pickController.ts` to:
1. Continue tracking all candidates (including eliminated ones) in the `matchingRegexes` field
2. Continue updating `positiveVotes` and `negativeVotes` for eliminated candidates
3. Prevent re-elimination by checking `!candidate.eliminated` before setting `eliminated = true`

## Changes Made
- [src/pickController.ts](../src/pickController.ts#L385-L387): Removed filter that excluded eliminated candidates from `matchingRegexes`
- [src/pickController.ts](../src/pickController.ts#L407-L429): Removed `continue` statement that skipped eliminated candidates in ACCEPT classification
- [src/pickController.ts](../src/pickController.ts#L436-L456): Removed `continue` statement that skipped eliminated candidates in REJECT classification
- [src/test/pickController.test.ts](../src/test/pickController.test.ts): Added comprehensive test suite "Eliminated Candidate Tracking" with 4 tests

## Tests Added
All tests pass (57/57):
1. **Eliminated candidates should continue to have votes updated** - Verifies negative votes continue to accumulate
2. **Eliminated candidates should appear in matchingRegexes for export** - Ensures export includes all matching candidates
3. **Eliminated candidates should continue to receive positive votes** - Verifies positive votes are tracked
4. **Eliminated candidates should be properly tracked through REJECT classifications** - Verifies REJECT path works correctly

## Behavior
- **UI Display**: Eliminated candidates will show updated vote counts in real-time as users continue classifying words
- **Export**: The JSON export will include eliminated candidates in the `matchingRegexes` array for each classification, providing complete tracking of which candidates (active or eliminated) matched each word

## Verification
Run automated tests:
```bash
npm test
```

To manually verify:
1. Generate candidates and eliminate one by classifying words it doesn't match/matches incorrectly
2. Continue classifying more words that would match/not match the eliminated candidate
3. Observe that the eliminated candidate's vote counts continue to update in the UI
4. Export the history and verify that `matchingRegexes` includes the eliminated candidate where applicable
107 changes: 107 additions & 0 deletions manual-testing/load-session.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
# Load Session Feature

## Feature
Added the ability to load a previously exported session from a JSON file. This is gated power-user functionality that allows users to:
- Start a session without using an LLM
- Share sessions with others
- Resume work on complex regex patterns
- Reproduce and debug specific scenarios
- Use exported data for testing or documentation

## Key Improvements
1. **Session load is accessible from the initial prompt screen** - Users can load a session before ever calling an LLM
2. **Used words are tracked** - Words from loaded sessions are tracked as "used" to prevent them from being resurfaced in pair generation

## Implementation

### UI Changes
- **[media/pickView.html](../media/pickView.html#L95-L103)**: Added "Load Session" button on the initial prompt screen
- Placed below the prompt input with explanatory text "Or load a previous session"
- Allows users to start with candidates + classifications without calling LLM
- **[media/pickView.html](../media/pickView.html#L219-L221)**: "Load" button in the Word Classification History section (during voting)
- Allows reloading a session mid-workflow

### Backend (pickController.ts)
- **[addUsedWords method](../src/pickController.ts#L976-L984)**: New method to add words to the used set
- Prevents loaded session words from being resurfaced in pair generation
- Used for programmatic addition of used words

### Backend (pickViewProvider.ts)
- **[handleLoadSession](../src/pickViewProvider.ts#L1024-L1155)**: Session loading handler
- Validates data structure
- Generates candidates from loaded data
- Applies classifications (which automatically marks words as used)
- Sends `showVoting` message to transition UI

### Format Compatibility
The loader accepts the same format as the export:
```json
{
"candidates": [
{
"regex": "[a-z]+",
"explanation": "lowercase letters",
"confidence": 0.9,
"equivalents": ["[a-z]*[a-z]"]
}
],
"classifications": [
{
"word": "abc",
"classification": "in",
"matchingRegexes": ["[a-z]+"]
}
]
}
```

**Classification format conversion**:
- Export: `"in"` / `"out"` / `"unsure"`
- Internal: `ACCEPT` / `REJECT` / `UNSURE`

### Status Updates
Enhanced `setHistoryCopyStatus` function to support different styling:
- Normal messages (default color)
- Error messages (red)
- Muted messages (description foreground)

## Tests
- **[src/test/loadSession.test.ts](../src/test/loadSession.test.ts)**: 6 comprehensive tests covering:
- Session data structure validation
- Classification format conversion
- Empty classifications handling
- Optional candidate fields (explanation, confidence, equivalents)
- Case-insensitive classification normalization
- Invalid input handling

All tests pass (136/136 in pickController suite, 6/6 in loadSession suite).

## Usage

1. **Export a session**: Click the "Export" button in the Word Classification History section
2. **Save the JSON file**: Copy is saved to clipboard, paste into a `.json` file
3. **Load the session**: Click the "Load" button and select the JSON file
4. **Session restored**: All candidates and classifications are restored, and you can continue voting or see the final result

## Error Handling

Clear error messages are shown for:
- Invalid JSON syntax
- Missing required fields (candidates, classifications)
- Empty candidates array
- Invalid data types
- File read failures

## Manual Verification

To manually verify:
1. Start a PICK session and classify several words
2. Export the history to clipboard
3. Save to a JSON file
4. Reset PICK
5. Click "Load" and select the saved JSON file
6. Verify:
- All candidates are restored with correct explanations and confidence scores
- All classifications are restored and reflected in vote counts
- UI shows correct active/eliminated candidate states
- Can continue voting if not in final state
54 changes: 54 additions & 0 deletions manual-testing/session.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
{
"candidates": [
{
"regex": "(?:\\./|/)?(?:[\\w.-]+/)*[\\w.-]+",
"explanation": "Matches relative or absolute Unix filepaths, allowing dot, dash, and underscore in names. Handles paths like './file', '/usr/bin', or 'folder/file'.",
"confidence": 0.8
},
{
"regex": "/(?:[^/]+/)*[^/]+",
"explanation": "Matches absolute Unix filepaths starting with '/', with segments separated by '/'. Does not match relative paths.",
"confidence": 0.7
},
{
"regex": "(?:[\\w.-]+/)*[\\w.-]+",
"explanation": "Matches relative filepaths without leading './' or '/', allowing multiple segments. Does not match absolute paths.",
"confidence": 0.7,
"equivalents": [
"(?:\\./)?(?:[\\w.-]+/)*[\\w.-]+(?:\\.[\\w]+)?"
]
},
{
"regex": "(?:/|\\./)?(?:[^/]+/)*[^/]+",
"explanation": "Matches both absolute and relative filepaths, allowing any character except '/' in segments. More permissive than others.",
"confidence": 0.6
}
],
"classifications": [
{
"word": "./a",
"classification": "in",
"matchingRegexes": [
"(?:\\./|/)?(?:[\\w.-]+/)*[\\w.-]+",
"(?:[\\w.-]+/)*[\\w.-]+",
"(?:/|\\./)?(?:[^/]+/)*[^/]+"
]
},
{
"word": "/a\u0000",
"classification": "in",
"matchingRegexes": [
"/(?:[^/]+/)*[^/]+",
"(?:/|\\./)?(?:[^/]+/)*[^/]+"
]
},
{
"word": "/a/a\u0000",
"classification": "out",
"matchingRegexes": [
"/(?:[^/]+/)*[^/]+",
"(?:/|\\./)?(?:[^/]+/)*[^/]+"
]
}
]
}
20 changes: 18 additions & 2 deletions media/pickView.html
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,18 @@
></textarea>

<div class="prompt-actions" aria-live="polite">
<button
id="loadSessionBtnPrompt"
class="icon-btn subtle"
title="Load session from JSON"
aria-label="Load session from exported JSON file"
>
<svg viewBox="0 0 24 24" xmlns="http://www.w3.org/2000/svg" aria-hidden="true" width="14" height="14">
<path d="M20 6h-8l-2-2H4c-1.1 0-2 .9-2 2v12c0 1.1.9 2 2 2h16c1.1 0 2-.9 2-2V8c0-1.1-.9-2-2-2zm0 12H4V6h5.17l2 2H20v10z" fill="currentColor" />
</svg>
</button>
<input type="file" id="loadSessionFilePrompt" accept=".json,application/json" style="display:none;" aria-hidden="true" />

<button
id="recentPromptsBtn"
class="icon-btn subtle"
Expand Down Expand Up @@ -98,6 +110,7 @@
<div class="history-menu__header">Recent prompts</div>
<div id="recentPromptList" class="history-menu__items"></div>
</div>
<span id="loadSessionStatus" class="history-copy-status" role="status" aria-live="polite"></span>
</div>
</div>

Expand Down Expand Up @@ -209,10 +222,13 @@ <h4 style="margin:0;">Should the generated regex match these words?</h4>
<div class="word-history__header">
<h3>Word Classification History</h3>
<div class="history-actions">
<button id="copyHistoryBtn" class="icon-btn subtle" title="Copy history as JSON" aria-label="Copy classification history as JSON">
<button id="copyHistoryBtn" class="icon-btn subtle" title="Export session as JSON" aria-label="Export session as JSON">
<span class="codicon codicon-clippy" aria-hidden="true"></span>
<span class="btn-label">Export</span>
</button>
<button id="loadSessionBtn" class="icon-btn subtle" title="Load session from JSON" aria-label="Load session from exported JSON file">
<span class="codicon codicon-folder-opened" aria-hidden="true"></span>
</button>
<input type="file" id="loadSessionFile" accept=".json,application/json" style="display:none;" aria-hidden="true" />
<span id="historyCopyStatus" class="history-copy-status" role="status" aria-live="polite"></span>
</div>
</div>
Expand Down
Loading