[codex] Add persistent GPU exact-key lookup stage

nemo_curator/stages/interleaved/README.md

@@ -160,6 +160,67 @@ into validated `InterleavedBatch` tasks. Together, the two stages support:
 InterleavedLanceReader -> LanceColumnFetchStage -> annotator
 ```
 
+### GPU exact-key presence lookup
+
+For bulk presence-only workflows, `GpuExactKeyLookupStage` loads immutable
+Parquet key segments into GPU memory and builds persistent RAPIDS
+`FilteredJoin` objects once per actor. It probes exact values without reading
+payload columns or rebuilding the GPU hash tables for each task. Install
+`cudf-cu12>=26.6,<26.7`; earlier cuDF releases do not expose the persistent
+`FilteredJoin` Python API.
+
+The stage has two inputs:
+
+| Input | Required content |
+|-------|------------------|
+| Curator task | An `InterleavedBatch` with `input_key_column`; its Arrow type must match the reference key type (regular and large UTF-8 strings are compatible) |
+| Reference set | One or more immutable Parquet files containing a non-null `reference_key_column`; the union of these columns is the exact membership set |
+
+It returns one `InterleavedBatch` for every input task. Every input row and
+column is preserved, and one nullable boolean column is appended:
+
+| Input key | Output `image_present` |
+|-----------|------------------------|
+| Null or empty string | Null; no GPU lookup is performed |
+| Exact key in any reference file | `True` |
+| Non-empty key absent from every reference file | `False` |
+
+If the destination presence column already exists, the task fails rather than
+silently overwriting it. Duplicate input keys are allowed and preserve their
+individual rows. The reference files are a set for membership purposes, so a
+key appearing in more than one reference segment still produces one boolean
+result per input row.
+
+```python
+from nemo_curator.stages.interleaved import GpuExactKeyLookupStage
+
+stage = GpuExactKeyLookupStage(
+    reference_files=[
+        "/local/image-urls/segment-000.parquet",
+        "/local/image-urls/segment-001.parquet",
+    ],
+    reference_key_column="url",
+    input_key_column="source_ref",
+    presence_column="image_present",
+    expected_reference_rows=355_952_746,
+).with_(num_workers=8, batch_size=8)
+```
+
+Each reference file is retained as an independent hash table. This avoids a
+second full-size allocation to concatenate large string columns during actor
+setup. `process_batch()` concatenates eligible input keys for one probe and
+then restores the original task boundaries and row order. Null and empty
+string keys are not queried and receive null presence; all other absent keys
+receive `False`.
+
+The stage consumes a stable Parquet key sidecar rather than Lance's private
+on-disk scalar-index files. Build that sidecar from a pinned source table using
+its normal public reader API. Use `LanceColumnFetchStage` instead when payload
+columns or stable row IDs must be returned. See the
+[MINT-1T HTML GPU presence tutorial](../../../tutorials/interleaved/mint_html_gpu_presence/)
+for a complete reader → lookup → writer pipeline and a public Lance-to-Parquet
+sidecar builder.
+
 ## Usage
 
 ```python
@@ -185,6 +246,7 @@ pipeline.run()
 ```
 stages/interleaved/
 ├── __init__.py                     # Exports filter/annotator stages
+├── gpu_key_lookup.py               # Persistent GPU exact-key membership
 ├── lance.py                        # Lance column fetch and interleaved Lance reader
 ├── stages.py                       # BaseInterleavedAnnotatorStage, BaseInterleavedFilterStage,
 │                                   # InterleavedAspectRatioFilterStage

nemo_curator/stages/interleaved/__init__.py

@@ -12,6 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
+from nemo_curator.stages.interleaved.gpu_key_lookup import GpuExactKeyLookupStage
 from nemo_curator.stages.interleaved.lance import (
     InterleavedLanceReader,
     InterleavedLanceReaderStage,
@@ -28,6 +29,7 @@
 __all__ = [
     "BaseInterleavedAnnotatorStage",
     "BaseInterleavedFilterStage",
+    "GpuExactKeyLookupStage",
     "InterleavedAspectRatioFilterStage",
     "InterleavedLanceReader",
     "InterleavedLanceReaderStage",