Skip to content

Rewrite SQLAlchemy dialect to support IoTDB 2.0+ table model#17537

Open
JackieTien97 wants to merge 3 commits intomasterfrom
ty/sqlalchemy
Open

Rewrite SQLAlchemy dialect to support IoTDB 2.0+ table model#17537
JackieTien97 wants to merge 3 commits intomasterfrom
ty/sqlalchemy

Conversation

@JackieTien97
Copy link
Copy Markdown
Contributor

Summary

Rewrite the Python SQLAlchemy dialect to support the IoTDB 2.0+ table model, replacing the old tree-model dialect.

Changes

  • DBAPI layer: Add sql_dialect and database parameters to Connection; remove silent exception swallowing in Cursor.execute()
  • IoTDBDialect: Complete rewrite with table model reflection (SHOW TABLES, SHOW COLUMNS FROM), construct_arguments for column categories and TTL
  • IoTDBDDLCompiler (new): Generates CREATE TABLE with TAG/ATTRIBUTE/FIELD/TIME column categories and WITH (TTL=...) clause
  • IoTDBSQLCompiler: Simplified — table model supports standard SQL, removed the old Time column encoding hack
  • IoTDBTypeCompiler: Updated type mappings for table model types (STRING, INT64, BLOB, TIMESTAMP, DATE)
  • Tests: Rewritten integration tests covering DDL, DML, reflection, and TIME column handling

Usage

from sqlalchemy import create_engine, Column, String, Float, Table, MetaData

engine = create_engine("iotdb://root:root@localhost:6667/test_db")

metadata = MetaData()
sensors = Table('sensors', metadata,
    Column('region', String, iotdb_category='TAG'),
    Column('device', String, iotdb_category='TAG'),
    Column('temperature', Float, iotdb_category='FIELD'),
    schema='test_db',
    iotdb_ttl=3600000
)

metadata.create_all(engine)

Column Categories

Columns are categorized via iotdb_category dialect argument:

  • TAG — indexed identity columns
  • ATTRIBUTE — metadata columns (not indexed)
  • FIELD — measurement/value columns
  • TIME — explicit time column (optional; auto-generated if omitted)

The old SQLAlchemy dialect was built for the tree model (path-based
schema). This rewrites it to support the table model with standard
relational SQL, including:
- Column categories (TAG, ATTRIBUTE, FIELD, TIME) via dialect-specific args
- DDL generation with CREATE TABLE categories and TTL support
- Table model reflection (SHOW TABLES, SHOW COLUMNS FROM)
- Updated type mappings (STRING, BLOB, TIMESTAMP, DATE)
- Simplified SQL compiler (table model supports standard SQL)
- DBAPI layer: add sql_dialect parameter, propagate exceptions

Co-Authored-By: Claude Opus 4.7 (1M context) <[email protected]>
@codecov
Copy link
Copy Markdown

codecov Bot commented Apr 22, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 39.80%. Comparing base (ce103a3) to head (ae5bc1b).
⚠️ Report is 4 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #17537      +/-   ##
============================================
- Coverage     39.86%   39.80%   -0.06%     
  Complexity      312      312              
============================================
  Files          5137     5142       +5     
  Lines        347310   347862     +552     
  Branches      44268    44397     +129     
============================================
+ Hits         138444   138475      +31     
- Misses       208866   209387     +521     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- Add sqlalchemy_example.py covering DDL/DML/reflection/raw SQL
- Add README_ZH.md as full Chinese translation of README.md
- Update README.md SQLAlchemy section for IoTDB 2.0+ table model
- Add tests for all data types, advanced queries (aggregation, AND/OR,
  NULL, batch insert, LIMIT+OFFSET), schema operations (has_schema,
  has_table, multiple databases, column category reflection, get_view_names),
  raw SQL, and URL-with-database
@sonarqubecloud
Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant