[ENH] Support schema in local chroma #5626

Sicheng-Pan · 2025-10-17T00:10:44Z

Description of changes

Summarize the changes made by this PR.

Improvements & Bug fixes
- N/A
New functionality
- ^

Test plan

How are these changes tested?

Tests pass locally with pytest for python, yarn test for js, cargo test for rust

Migration plan

Are there any migrations, or any forwards/backwards compatibility changes needed in order to make sure this change deploys reliably?

Observability plan

What is the plan to instrument and monitor this change?

Documentation Changes

Are all docstrings for user-facing APIs updated if required? Do we need to make documentation changes in the docs section?

Sicheng-Pan · 2025-10-17T00:10:59Z

[ENH] Support schema in local chroma #5626 👈 (View in Graphite)
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

github-actions · 2025-10-17T00:11:27Z

Sicheng-Pan · 2025-10-17T00:16:22Z

16 Jobs Failed:

PR checks / all-required-pr-checks-passed failed on "Decide whether the needed jobs succeeded or failed"

[...]
}
EOM
)"
shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
env:
  GITHUB_REPO_NAME: chroma-core/chroma
  PYTHONPATH: /home/runner/_work/_actions/re-actors/alls-green/release/v1/src
# ❌ Some of the required to succeed jobs failed 😢😢😢

📝 Job statuses:
📝 python-tests → ❌ failure [required to succeed or be skipped]
📝 python-vulnerability-scan → ✓ success [required to succeed or be skipped]
📝 javascript-client-tests → ❌ failure [required to succeed or be skipped]
📝 rust-tests → ❌ failure [required to succeed or be skipped]
📝 go-tests → ✓ success [required to succeed or be skipped]
📝 lint → ✓ success [required to succeed]
📝 check-helm-version-bump → ⬜ skipped [required to succeed or be skipped]
📝 delete-helm-comment → ✓ success [required to succeed or be skipped]
Error: Process completed with exit code 1.

PR checks / JavaScript client tests / test failed on "Test"

[...]
      79 |   );
      80 | };

      at chromaFetch (src/chroma-fetch.ts:77:9)
      at async o (../../node_modules/.pnpm/@[email protected]_@[email protected][email protected]_/node_modules/@hey-api/client-fetch/src/client.ts:91:11)
      at async ChromaClient.deleteCollection (src/chroma-client.ts:427:5)
      at async Object.<anonymous> (test/peek.collection.test.ts:28:5)

PASS test/offline.test.ts

Test Suites: 9 failed, 4 passed, 13 total
Tests:       52 failed, 34 passed, 86 total
Snapshots:   1 passed, 1 total
Time:        5.097 s
Ran all test suites.
🔧 rust-server: 2025-10-17T00:16:01.808539Z  INFO  Received SIGTERM, shutting down service
    at rust/frontend/src/server.rs:96
🔧 rust-server: 
 ELIFECYCLE  Test failed. See above for more details.
Error: Process completed with exit code 1.

PR checks / Python tests / test-python-cli (3.9) failed on "Integration Test"

[...]
E           assert 0 == 1
E            +  where 0 = len([])

chromadb/test/test_cli.py:109: AssertionError
----------------------------- Captured stdout call -----------------------------


Chroma Vacuum


Purging the log...


----------------------------- Captured stderr call -----------------------------
Failed to vacuum Chroma
=========================== short test summary info ============================
FAILED chromadb/test/test_cli.py::test_vacuum[python_sqlite_persistent] - assert 0 == 1
 +  where 0 = len([])
========================= 1 failed, 2 passed in 16.93s =========================
Error: Process completed with exit code 1.

PR checks / Python tests / test-rust-bindings (3.9, --ignore-glob 'chromadb/test/property/*' --ignore-glob 'chromadb/test/st... failed on "Test"

[...]
    record_set={'ids': ['vgHf'],
     'embeddings': [[0.0976270078546495,
       0.43037873274483895,
       0.20552675214328775,
       0.08976636599379373,
       -0.15269040132219058,
       0.29178822613331223,
       -0.12482557747461498,
       0.7835460015641595,
       0.9273255210020586,
       -0.2331169623484446,
       0.5834500761653292,
       0.05778983950580896,
       0.13608912218786462,
       0.8511932765853221,
       -0.8579278836042261,
       -0.8257414005969186,
       -0.9595632051193486,
       0.665239691095876,
       0.556313501899701,

PR checks / Python tests / test-rust-bindings (3.9, chromadb/test/property --ignore-glob chromadb/test/property/test_cross_v... failed on "Test"

[...]
        caplog: pytest.LogCaptureFixture,
        client: ClientAPI,
        collection: strategies.Collection,
        record_set: strategies.RecordSet,
        filters: List[strategies.Filter],
        should_compact: bool,
        data: st.DataObject,
    ) -> None:
        caplog.set_level(logging.ERROR)
    
        reset(client)
>       coll = client.create_collection(
            name=collection.name,
            metadata=collection.metadata,  # type: ignore
            embedding_function=collection.embedding_function,
        )

chromadb/test/property/test_filtering.py:504: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
chromadb/api/client.py:183: in create_collection

PR checks / Python tests / test-rust-bindings (3.9, chromadb/test/property/test_cross_version_persist.py) failed on "Test"

[...]
      [0.49471491735316087,
       0.6626073181945178,
       -0.6743019079659223,
       0.5038120720135397,
       0.34141991677633876,
       0.7813747838277567,
       0.9375825078903437,
       -0.7510787823801179,
       -0.8386522771583667,
       0.5994602579040187,
       -0.6756582538282832,
       -0.6108509514855862,
       0.7640718450516537,
       0.8779297942050499,
       -0.363057512154052,
       0.6843362008759879,
       0.717713819127302,
       0.2107809610040725,
       0.1321768961262899,
       0.1516768184912194,

PR checks / Python tests / test-rust-bindings-stress (3.9) failed on "Test"

[...]
>           serialized_schema=collection.schema,
            metadata=collection.metadata,
            dimension=collection.dimension,
            tenant=collection.tenant,
            database=collection.database,
        )
E       AttributeError: 'builtins.Collection' object has no attribute 'schema'

chromadb/api/rust.py:250: AttributeError
============================= slowest 10 durations =============================
0.23s setup    chromadb/test/stress/test_many_collections.py::test_many_collections[rust_sqlite_ephemeral]
0.06s call     chromadb/test/stress/test_many_collections.py::test_many_collections[rust_sqlite_persistent]
0.03s setup    chromadb/test/stress/test_many_collections.py::test_many_collections[rust_sqlite_persistent]
0.01s call     chromadb/test/stress/test_many_collections.py::test_many_collections[rust_sqlite_ephemeral]

(2 durations < 0.005s hidden.  Use -vv to show these durations.)
=========================== short test summary info ============================
FAILED chromadb/test/stress/test_many_collections.py::test_many_collections[rust_sqlite_persistent] - AttributeError: 'builtins.Collection' object has no attribute 'schema'
========================= 1 failed, 1 passed in 0.40s ==========================
Error: Process completed with exit code 1.

PR checks / Python tests / test-rust-single-node-integration (3.9, --ignore-glob 'chromadb/test/property/*' --ignore-glob 'c... failed on "Rust Integration Test"

[...]
       0.6701075434684753,
       -0.5605696439743042,
       -0.9741194248199463,
       0.28687378764152527,
       -0.6557893753051758,
       -0.8765866756439209,
       0.8675418496131897,
       -0.7030428051948547,
       0.04009866341948509,
       -0.2654014229774475,
       -0.9105188250541687,
       -0.37129247188568115,
       -0.2990592122077942,
       -0.013124626129865646,
       0.413469135761261,
       0.9999558925628662,
       0.037142541259527206,
       0.8855062127113342,
       0.05490194633603096,
       0.1873646378517151,

PR checks / Python tests / test-rust-single-node-integration (3.9, chromadb/test/property/test_add.py) failed on "Rust Integration Test"

[...]
       -0.736328125,
       0.24462890625,
       0.86376953125,
       -0.22314453125,
       -0.80322265625,
       0.2283935546875,
       -0.6025390625,
       -0.253662109375,
       -0.64599609375,
       0.63037109375,
       0.64990234375,
       -0.64013671875,
       0.640625,
       -0.33837890625,
       0.122314453125,
       0.471435546875,
       -0.388916015625,
       -0.6552734375,
       0.113525390625,
       0.035400390625,

PR checks / Python tests / test-rust-single-node-integration (3.9, chromadb/test/property/test_collections.py) failed on "Rust Integration Test"

[...]
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:444
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:633
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:808
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:816
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:829
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/typing_extensions.py:761

You can reproduce this example by temporarily adding @reproduce_failure('6.112.2', b'AXicY2AAAhYgZmRAAwAAfwAG') as a decorator on your test case
FAILED chromadb/test/property/test_collections.py::test_previously_failing_one[integration] - chromadb.errors.InternalError: Could not fetch collections: Could not deserialize database ID
FAILED chromadb/test/property/test_collections.py::test_previously_failing_one[async_integration] - chromadb.errors.InternalError: Could not fetch collections: Could not deserialize database ID
FAILED chromadb/test/property/test_collections.py::test_previously_failing_two[integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_collections.py::test_previously_failing_two[async_integration] - chromadb.errors.InternalError: Could not deserialize database ID
======================= 6 failed, 114 warnings in 56.83s =======================
  2025-10-17T00:15:36.873935Z ERROR  , name: "PeriodicReader.ExportFailed", Failed to export metrics, reason: "Metrics exporter otlp failed with the grpc server returns error (The service is currently unavailable): , detailed error message: dns error: failed to lookup address information: Temporary failure in name resolution"
    at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.27.0/src/metrics/periodic_reader.rs:262

  2025-10-17T00:15:36.876475Z ERROR  , name: "BatchSpanProcessor.Flush.ExportError", reason: "ExportFailed(Status { code: Unavailable, message: \", detailed error message: dns error: failed to lookup address information: Temporary failure in name resolution\" })", Failed during the export process
    at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.27.0/src/trace/span_processor.rs:334

Error: Process completed with exit code 1.

PR checks / Python tests / test-rust-single-node-integration (3.9, chromadb/test/property/test_collections_with_database_ten... failed on "Rust Integration Test"

[...]
        /home/runner/_work/chroma/chroma/chromadb/test/property/strategies.py:383
        /home/runner/_work/chroma/chroma/chromadb/test/property/strategies.py:387
        /home/runner/_work/chroma/chroma/chromadb/test/property/strategies.py:389
        /home/runner/_work/chroma/chroma/chromadb/test/property/test_collections.py:232
        /home/runner/_work/chroma/chroma/chromadb/test/property/test_collections.py:233
        /home/runner/_work/chroma/chroma/chromadb/utils/async_to_sync.py:59
        /home/runner/_work/chroma/chroma/chromadb/utils/async_to_sync.py:61
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/collections/__init__.py:598
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_content.py:210
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:302
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:444
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:633
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:808
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:816
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/httpx/_models.py:829
        /opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/typing_extensions.py:761

You can reproduce this example by temporarily adding @reproduce_failure('6.112.2', b'AXicY2AAAjYgZmRAAwAAqQAI') as a decorator on your test case
================= 2 failed, 151 warnings in 142.23s (0:02:22) ==================
Error: Process completed with exit code 1.

PR checks / Python tests / test-rust-single-node-integration (3.9, chromadb/test/property/test_embeddings.py) failed on "Rust Integration Test"

[...]
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[integration-kwargs3] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[integration-kwargs4] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[integration-kwargs5] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs0] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs1] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs2] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs3] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs4] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs5] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_no_op_compaction[integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_no_op_compaction[async_integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_add_then_purge[integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_add_then_purge[async_integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_encompassing_delete[integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_encompassing_delete[async_integration] - chromadb.errors.InternalError: Could not deserialize database ID
======= 34 failed, 8 passed, 2 skipped, 2 xfailed, 14 warnings in 7.27s ========
  2025-10-17T00:14:54.708710Z ERROR  , name: "BatchSpanProcessor.Flush.ExportError", reason: "ExportFailed(Status { code: Unavailable, message: \", detailed error message: dns error: failed to lookup address information: Temporary failure in name resolution\" })", Failed during the export process
    at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.27.0/src/trace/span_processor.rs:334

Error: Process completed with exit code 1.

PR checks / Python tests / test-rust-single-node-integration (3.9, chromadb/test/property/test_filtering.py) failed on "Rust Integration Test"

[...]
       0.53369140625,
       -0.9697265625,
       0.094970703125,
       0.61767578125,
       0.97705078125,
       0.52001953125,
       0.50634765625,
       -0.9111328125,
       0.87451171875,
       -0.60498046875,
       -0.25,
       0.705078125,
       0.3955078125,
       0.529296875,
       0.1751708984375,
       -0.5205078125,
       0.87890625,
       -0.580078125,
       -0.91015625,
       -0.97412109375,

PR checks / Python tests / test-rust-thin-client (3.9, chromadb/test/property/test_embeddings.py) failed on "Test"

[...]
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[integration-kwargs3] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[integration-kwargs4] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[integration-kwargs5] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs0] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs1] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs2] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs3] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs4] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_delete_success[async_integration-kwargs5] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_no_op_compaction[integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_no_op_compaction[async_integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_add_then_purge[integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_add_then_purge[async_integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_encompassing_delete[integration] - chromadb.errors.InternalError: Could not deserialize database ID
FAILED chromadb/test/property/test_embeddings.py::test_encompassing_delete[async_integration] - chromadb.errors.InternalError: Could not deserialize database ID
======= 34 failed, 8 passed, 2 skipped, 2 xfailed, 11 warnings in 6.28s ========
  2025-10-17T00:14:53.542872Z ERROR  , name: "BatchSpanProcessor.Flush.ExportError", reason: "ExportFailed(Status { code: Unavailable, message: \", detailed error message: dns error: failed to lookup address information: Temporary failure in name resolution\" })", Failed during the export process
    at /home/runner/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/opentelemetry_sdk-0.27.0/src/trace/span_processor.rs:334

Error: Process completed with exit code 1.

PR checks / Rust tests / test (blacksmith-8vcpu-ubuntu-2404) failed on "Test"

[...]
      28: core::ops::function::FnOnce::call_once
                 at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/core/src/ops/function.rs:250:5
      29: core::ops::function::FnOnce::call_once
                 at /rustc/4d91de4e48198da2e33413efdcd9cd2cc0c46688/library/core/src/ops/function.rs:250:5
    note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.

  Cancelling due to test failure: 7 tests still running
        PASS [   0.029s] chroma-frontend quota::tests::test_quota_exceeded_error_message_empty
        PASS [   1.967s] chroma-blockstore arrow::blockfile::tests::test_get_by_prefix
        PASS [   1.012s] chroma-distance distance_sse::tests::test_sse_random_sizes
        PASS [   2.032s] chroma-cache foyer::test::test_foyer_hybrid_cache_can_recover
        PASS [   2.054s] chroma-cache foyer::test::test_writing_only_to_disk_works
        PASS [  16.613s] chroma-blockstore::blockfile_writer_test tests::blockfile_writer_test
        PASS [  40.782s] chroma-blockstore arrow::concurrency_test::tests::test_blockfile_shuttle
────────────
     Summary [  41.322s] 135/655 tests run: 134 passed, 1 failed, 124 skipped
        FAIL [   0.227s] chroma-frontend executor::local::tests::test_query
warning: 520/655 tests were not run due to test failure (run with --no-fail-fast to run all tests, or run with --max-fail)
error: test run failed
Error: Process completed with exit code 100.

1 job failed running on non-Blacksmith runners.

Summary: 1 successful workflow, 1 failed workflow

✅ Check PR Title (1 job succeeded)
❌ PR checks (35 jobs succeeded, 16 jobs failed, 4 jobs cancelled)

Last updated: 2025-10-17 01:14:05 UTC

[ENH] Support schema in local chroma

f68cf25

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[ENH] Support schema in local chroma #5626

[ENH] Support schema in local chroma #5626

Uh oh!

Sicheng-Pan commented Oct 17, 2025 •

edited

Loading

Uh oh!

Sicheng-Pan commented Oct 17, 2025

Uh oh!

github-actions bot commented Oct 17, 2025

Uh oh!

Sicheng-Pan commented Oct 17, 2025 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

[ENH] Support schema in local chroma #5626

Are you sure you want to change the base?

[ENH] Support schema in local chroma #5626

Uh oh!

Conversation

Sicheng-Pan commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description of changes

Test plan

Migration plan

Observability plan

Documentation Changes

Uh oh!

Sicheng-Pan commented Oct 17, 2025

Uh oh!

github-actions bot commented Oct 17, 2025

Reviewer Checklist

Testing, Bugs, Errors, Logs, Documentation

System Compatibility

Quality

Uh oh!

Sicheng-Pan commented Oct 17, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Sicheng-Pan commented Oct 17, 2025 •

edited

Loading

Sicheng-Pan commented Oct 17, 2025 •

edited

Loading