Skip to content

Commit e8fabbe

Browse files
committed
feat: add AI photo organizer with face recognition and multi-language support
Add the `pdx organize` command that uses a vision-language model via Ollama to describe photos and group them into named folders by date. - YAML-based configuration with example template - Family context file for personalized AI descriptions - Folder history as style examples for AI naming - Face recognition via insightface with reference photo directories - HEIC/HEIF conversion for VLM compatibility - Multi-language prompts (Czech and English) - Video discovery and organization alongside photos - GPS/EXIF-based location extraction with home location filtering - Helpful error when config.yaml is missing - Documentation for setup, configuration, and all features Bug fixes: - fix: move text tokens to GPU and results back to CPU in prompt_to_vector - fix: filter files by supported extensions in handle_path to prevent indexing non-photo files passed as arguments
1 parent da37de7 commit e8fabbe

13 files changed

Lines changed: 712 additions & 15 deletions

.gitignore

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,9 @@
1+
# Configuration and Personal Data
2+
config.yaml
3+
family_context.txt
4+
history_cache.json
5+
6+
# Python artifacts
7+
.venv/
18
/pdx.egg-info/
29
__pycache__/

README-Windows.md

Lines changed: 70 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,9 +43,78 @@ cd pdx
4343
cd /mnt/c/Users/<YourWindowsUser>/
4444
```
4545

46+
## Install exiftool
47+
48+
The `organize` command requires `exiftool` for EXIF and GPS metadata extraction.
49+
Without it, all photos will show as "Unknown location".
50+
51+
```
52+
sudo apt install -y libimage-exiftool-perl
53+
```
54+
55+
## Connect to Ollama on Windows
56+
57+
If Ollama is installed on Windows (not inside WSL), you need to make it accessible from WSL.
58+
59+
### 1. Make Ollama listen on all interfaces
60+
61+
Set a Windows environment variable:
62+
63+
- **Settings → System → About → Advanced system settings → Environment Variables**
64+
- Add a new variable: `OLLAMA_HOST` = `0.0.0.0`
65+
- Fully quit Ollama from the system tray (right-click → Quit) and relaunch it
66+
67+
Verify in PowerShell:
68+
69+
```
70+
netstat -an | findstr 11434
71+
```
72+
73+
You should see `0.0.0.0:11434` in the output.
74+
75+
### 2. Add a firewall rule
76+
77+
Find the WSL network address. In WSL, run:
78+
79+
```
80+
ip -4 addr show eth0 | grep -oP 'inet \K[\d.]+'
81+
```
82+
83+
Replace the host part of the IP with `0` to get the network address (e.g. if the
84+
command prints `a.b.c.d`, use `a.b.0.0`). Then open PowerShell as Administrator:
85+
86+
```
87+
netsh advfirewall firewall add rule name="Ollama WSL" dir=in action=allow protocol=TCP localport=11434 remoteip=<a.b.0.0>/20
88+
```
89+
90+
This restricts access to the WSL subnet only.
91+
92+
### 3. Update config.yaml
93+
94+
Find your WSL gateway IP (this is the Windows host as seen from WSL):
95+
96+
```
97+
ip route show default | awk '{print $3}'
98+
```
99+
100+
Update `config.yaml` with that IP:
101+
102+
```yaml
103+
ai:
104+
ollama_url: "http://<gateway-ip>:11434/api/chat"
105+
```
106+
107+
### 4. Verify connectivity
108+
109+
```
110+
curl http://<gateway-ip>:11434/api/version
111+
```
112+
113+
You should get a JSON response with the Ollama version.
114+
46115
## Additional Debian packages (optional)
47116

48-
- To view the photos selected after a `pdx` query, use `qimgv` instead of `gwenview` due to stability in Windows
117+
- To view the photos selected after a `pdx` query, use `qimgv` instead of `gwenview` due to stability in Windows
49118
```
50119
sudo apt install -y qimgv
51120
```

README.md

Lines changed: 69 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,15 @@
11
# Photo inDeXer (pdx)
22

3+
## Prerequisites
4+
5+
- Python 3.13+
6+
- [Ollama](https://ollama.com) with a vision-language model pulled (e.g. `ollama pull gemma4:26b`)
7+
- [Podman](https://podman.io) for running Qdrant
8+
- `exiftool` for EXIF/GPS metadata extraction (`sudo apt install -y libimage-exiftool-perl`)
9+
- `libGL` for face recognition (`sudo apt install -y libgl1` on Debian/Ubuntu, `sudo dnf install -y mesa-libGL` on Fedora) — optional, only needed if using the `faces` config
10+
11+
See [README-Windows.md](README-Windows.md) for Windows/WSL-specific setup.
12+
313
## Set up a virtual Python environment
414

515
```sh
@@ -8,6 +18,60 @@ source .venv/bin/activate
818
pip install -e .
919
```
1020

21+
## Configuration
22+
23+
Copy the example config and adjust it for your setup:
24+
25+
```sh
26+
cp config.example.yaml config.yaml
27+
```
28+
29+
| Section | Key | Description |
30+
|---------|-----|-------------|
31+
| `ai` | `language` | Output language: `cs` (Czech) or `en` (English) |
32+
| `ai` | `ollama_url` | URL of the Ollama API endpoint |
33+
| `ai` | `model_name` | Vision-language model to use (e.g. `gemma4:26b`) |
34+
| `location` | `home_names` | List of city names considered "home" — photos taken here won't have the location in the folder name |
35+
| `faces` | `reference_dir` | Directory with reference face photos for recognition (one subdirectory per person) |
36+
| `faces` | `similarity_threshold` | Face matching threshold (lower = stricter, default `0.4`) |
37+
| `faces` | `name_map` | Map directory names to display names (e.g. `john: "Johnny"`) |
38+
| `storage` | `context_file` | Path to a text file with family/personal context for the AI |
39+
| `storage` | `history_file` | JSON list of past folder names (e.g. `"210619 - Beach volleyball"`) used as style examples for AI naming |
40+
41+
### Face recognition (optional)
42+
43+
To enable face recognition, create a reference directory with one subdirectory per person, each containing a few clear photos of their face (one face per photo):
44+
45+
```
46+
~/results/pdx/faces/
47+
├── john/
48+
│ ├── photo1.jpg
49+
│ ├── photo2.jpg
50+
│ └── photo3.jpg
51+
└── jane/
52+
├── photo1.jpg
53+
└── photo2.jpg
54+
```
55+
56+
Directory names are used as identifiers. Use `name_map` in the config to map them to display names (e.g. `john: "Johnny"`). 3-5 reference photos per person is usually enough.
57+
58+
### Family context (optional)
59+
60+
The `context_file` (default: `family_context.txt`) gives the AI background knowledge about your family — names, hobbies, sports, travel habits. This helps it generate more accurate photo descriptions and folder names. Write it in the same language as your `language` setting. Example:
61+
62+
```
63+
FAMILY MEMBERS:
64+
- Dad: Born 1985. Hobbies, sports.
65+
- Mom: Born 1987. Hobbies, interests.
66+
- Child1: Born 2013. Sport (team name, jersey color).
67+
68+
SPORTS:
69+
- Sport1 (Child1 only): Jersey description, equipment.
70+
- Sport2 (Dad): Gear, typical events.
71+
```
72+
73+
If the file is missing, the AI falls back to generic descriptions.
74+
1175
## Start/stop Qdrant (podman)
1276

1377
Storage is in the `pdx` directory under XDG data home (default: `~/.local/share/pdx`).
@@ -29,6 +93,11 @@ pdx index -c private /path/to/private_photos # create or extend the `private`
2993
pdx erase -c private # delete the `private` collection
3094
```
3195

96+
## Organize photos
97+
```sh
98+
pdx organize -c private /path/to/organized_folder # Use AI and EXIF to group photos into a structured directory tree.
99+
```
100+
32101
## Query photos
33102

34103
```sh

config.example.yaml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,22 @@
1+
# PDX Configuration Template
2+
# Copy this file to config.yaml and adjust for your setup.
3+
4+
ai:
5+
language: en # cs or en
6+
ollama_url: "http://localhost:11434/api/chat"
7+
model_name: "gemma4:26b"
8+
9+
location:
10+
# Add names of cities or districts you consider "Home"
11+
home_names:
12+
- "City"
13+
14+
faces:
15+
reference_dir: "faces"
16+
similarity_threshold: 0.4
17+
name_map: # optional: folder name -> display name
18+
john: "John"
19+
20+
storage:
21+
context_file: "family_context.txt"
22+
history_file: "folder_history.json"

pdx/cli.py

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,23 @@ def index(collection: str, force_cpu: bool, real_path: bool, paths: tuple[str, .
5555
idx = Indexer(force_cpu=force_cpu)
5656
idx.index_photos(collection, photos)
5757

58+
@pdx.command()
59+
@click.argument("target", type=click.Path())
60+
@click.option("--collection", "-c", default="default", help="Qdrant collection name.")
61+
@click.option("--config", "-f", default="config.yaml", help="Path to config file.")
62+
def organize(target: str, collection: str, config: str):
63+
"""Organize indexed photos into a structured directory tree."""
64+
from pdx.organizer import Organizer
65+
logging.basicConfig(level=logging.INFO)
66+
config_path = os.path.realpath(config)
67+
if not os.path.isfile(config_path):
68+
raise click.BadParameter(
69+
f"Config file not found: {config}\n"
70+
"Copy config.example.yaml to config.yaml and adjust it for your setup.",
71+
param_hint="'--config'",
72+
)
73+
org = Organizer(collection, target, config_path=config_path)
74+
org.organize()
5875

5976
@pdx.command()
6077
@click.argument("query_args", nargs=-1)

pdx/faces.py

Lines changed: 88 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,88 @@
1+
import logging
2+
import warnings
3+
from pathlib import Path
4+
5+
import cv2
6+
import numpy as np
7+
8+
warnings.filterwarnings("ignore", message=".*estimate.*deprecated.*", category=FutureWarning)
9+
from insightface.app import FaceAnalysis
10+
11+
12+
PHOTO_EXTENSIONS = {'.jpg', '.jpeg', '.png', '.heic', '.heif'}
13+
14+
15+
class FaceRecognizer:
16+
def __init__(self, reference_dir, similarity_threshold=0.4, det_size=(640, 640), name_map=None):
17+
self.similarity_threshold = similarity_threshold
18+
self.name_map = name_map or {}
19+
20+
self.app = FaceAnalysis(name='buffalo_l', providers=['CPUExecutionProvider'])
21+
self.app.prepare(ctx_id=-1, det_size=det_size)
22+
23+
self.centroids = self._load_references(Path(reference_dir))
24+
if self.centroids:
25+
logging.info(f"Face recognition: loaded {len(self.centroids)} persons: {', '.join(self.centroids.keys())}")
26+
else:
27+
logging.warning("Face recognition: no reference embeddings loaded")
28+
29+
def _load_references(self, reference_dir):
30+
centroids = {}
31+
for person_dir in sorted(reference_dir.iterdir()):
32+
if not person_dir.is_dir():
33+
continue
34+
embeddings = []
35+
for img_file in sorted(person_dir.iterdir()):
36+
if img_file.suffix.lower() not in PHOTO_EXTENSIONS:
37+
continue
38+
img = cv2.imread(str(img_file))
39+
if img is None:
40+
logging.warning(f"Face ref: cannot read {img_file}")
41+
continue
42+
faces = self.app.get(img)
43+
if len(faces) == 0:
44+
logging.warning(f"Face ref: no face in {img_file}")
45+
elif len(faces) > 1:
46+
logging.warning(f"Face ref: multiple faces in {img_file}, skipping")
47+
else:
48+
embeddings.append(faces[0].embedding)
49+
if embeddings:
50+
centroids[person_dir.name] = np.mean(embeddings, axis=0)
51+
else:
52+
logging.warning(f"Face ref: no usable faces for '{person_dir.name}'")
53+
return centroids
54+
55+
def _resolve_name(self, folder_name):
56+
if folder_name in self.name_map:
57+
return self.name_map[folder_name]
58+
return folder_name[0].upper() + folder_name[1:]
59+
60+
def identify_faces(self, image_path):
61+
try:
62+
img = cv2.imread(str(image_path))
63+
if img is None:
64+
return []
65+
faces = self.app.get(img)
66+
if not faces:
67+
return []
68+
except Exception as e:
69+
logging.warning(f"Face detection failed for {image_path}: {e}")
70+
return []
71+
72+
identified = []
73+
for face in faces:
74+
best_name = None
75+
best_score = -1
76+
for name, centroid in self.centroids.items():
77+
score = np.dot(face.embedding, centroid) / (
78+
np.linalg.norm(face.embedding) * np.linalg.norm(centroid)
79+
)
80+
if score > best_score:
81+
best_score = score
82+
best_name = name
83+
if best_name and best_score >= self.similarity_threshold:
84+
display_name = self._resolve_name(best_name)
85+
if display_name not in identified:
86+
identified.append(display_name)
87+
88+
return identified

pdx/find.py

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,7 @@
88

99

1010
PHOTOS_EXTS = ("heic", "heif", "jpeg", "jpg", "png")
11+
VIDEO_EXTS = ("mp4", "mov", "avi", "mkv", "mts", "m4v")
1112

1213

1314
class Finder:
@@ -42,8 +43,11 @@ def find_photos_in_dir(self, path: Path) -> None:
4243

4344
def handle_path(self, path: Path) -> None:
4445
if path.is_file():
45-
# take regular files as photos to index
46-
self._photos.append(str(path))
46+
ext = path.suffix[1:].lower() if path.suffix else ""
47+
if ext in PHOTOS_EXTS:
48+
self._photos.append(str(path))
49+
else:
50+
logging.warning(f"skipping unsupported file: {path}")
4751
elif path.is_dir():
4852
# traverse directories recursively and look for files matching PHOTOS_EXTS
4953
self.find_photos_in_dir(path)
@@ -59,3 +63,14 @@ def find_photos(paths: tuple[str, ...], include_symlinks: bool = False) -> list[
5963
finder.handle_path(path)
6064

6165
return finder.photos
66+
67+
68+
def find_videos(directories: set[Path]) -> list[Path]:
69+
videos = []
70+
for d in directories:
71+
if not d.is_dir():
72+
continue
73+
for f in d.iterdir():
74+
if f.is_file() and f.suffix and f.suffix[1:].lower() in VIDEO_EXTS:
75+
videos.append(f)
76+
return videos

pdx/model.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -86,9 +86,13 @@ def tensors_to_vectors(self, tensors):
8686
return tensor.cpu().numpy()
8787

8888
def prompt_to_vector(self, prompt):
89+
"""Fixed: Moves tokens to GPU and results to CPU."""
8990
with torch.no_grad():
90-
text = self._tokenizer([prompt])
91+
# Move text tokens to the device (GPU)
92+
text = self._tokenizer([prompt]).to(self._device)
9193
encode_text_fn = cast(
9294
Callable[..., torch.Tensor], getattr(self._model, "encode_text")
9395
)
94-
return encode_text_fn(text).numpy().flatten().tolist()
96+
# Move the resulting vector back to CPU
97+
vector = encode_text_fn(text)
98+
return vector.cpu().numpy().flatten().tolist()

0 commit comments

Comments
 (0)