Skip to content

RisingWave: SQLMesh plan fails due to an attempt to create the same MV twice #5184

New issue
New issue
Closed
Bug
#5195
Closed
RisingWave: SQLMesh plan fails due to an attempt to create the same MV twice#5184
Bug
#5195
@xardasos

Description

@xardasos
xardasos
opened on Aug 19, 2025

SQLMesh is unable to execute any plan with materialized views on RisingWave DB, as it fails due to an attempt to create the same MV twice.

How to reproduce

  1. Create a RW project with a MV
  2. Execute sqlmesh plan

You can follow the steps from this Tobiko blog post.

Current bug behavior

When following the steps from the Tobiko blog post, sqlmesh plan fails with the following error: materialized view with name reporting__event_summary_tumbling__3036268106 exists.

Log

2025-08-19 09:49:07,830 - MainThread - sqlmesh.core.plan.evaluator - INFO - Evaluating plan stage CreateSnapshotRecordsStage (evaluator.py:125)
2025-08-19 09:49:07,974 - MainThread - sqlmesh.core.plan.evaluator - INFO - Evaluating plan stage PhysicalLayerUpdateStage (evaluator.py:125)
2025-08-19 09:49:07,977 - ThreadPoolExecutor-1_0 - sqlmesh.core.snapshot.evaluator - INFO - Listing data objects in schema dev.sqlmesh__sqlmesh_example (evaluator.py:361)
2025-08-19 09:49:07,977 - ThreadPoolExecutor-1_1 - sqlmesh.core.snapshot.evaluator - INFO - Listing data objects in schema dev.sqlmesh__reporting (evaluator.py:361)
2025-08-19 09:49:07,977 - ThreadPoolExecutor-1_0 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: /* SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT current_catalog (base.py:2260)
2025-08-19 09:49:07,978 - ThreadPoolExecutor-1_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT current_catalog (base.py:2260)
2025-08-19 09:49:07,998 - MainThread - sqlmesh.core.snapshot.evaluator - INFO - Creating schema 'dev.sqlmesh__reporting' (evaluator.py:1176)
2025-08-19 09:49:07,999 - MainThread - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / CREATE SCHEMA IF NOT EXISTS "sqlmesh__reporting" (base.py:2260)
2025-08-19 09:49:08,055 - MainThread - sqlmesh.core.snapshot.evaluator - INFO - Creating schema 'dev.sqlmesh__sqlmesh_example' (evaluator.py:1176)
2025-08-19 09:49:08,056 - MainThread - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / CREATE SCHEMA IF NOT EXISTS "sqlmesh__sqlmesh_example" (base.py:2260)
2025-08-19 09:49:08,113 - ThreadPoolExecutor-2_0 - sqlmesh.core.snapshot.evaluator - INFO - Creating table 'dev.sqlmesh__sqlmesh_example.sqlmesh_example__seed_model__2185867172' (evaluator.py:1538)
2025-08-19 09:49:08,115 - ThreadPoolExecutor-2_0 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / CREATE TABLE IF NOT EXISTS "sqlmesh__sqlmesh_example"."sqlmesh_example__seed_model__2185867172" ("id" INT, "item_id" INT, "event_date" DATE) (base.py:2260)
2025-08-19 09:49:08,117 - ThreadPoolExecutor-2_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT 1 FROM "information_schema"."tables" WHERE "table_name" = 'reporting__event_summary_tumbling__3036268106' AND "table_schema" = 'sqlmesh__reporting' (base.py:2260)
2025-08-19 09:49:08,124 - ThreadPoolExecutor-2_1 - sqlmesh.core.snapshot.evaluator - INFO - Creating view 'dev.sqlmesh__reporting.reporting__event_summary_tumbling__3036268106' (evaluator.py:1979)
2025-08-19 09:49:08,131 - ThreadPoolExecutor-2_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / CREATE MATERIALIZED VIEW "sqlmesh__reporting"."reporting__event_summary_tumbling__3036268106" AS SELECT "window_start" AS "window_start", "window_end" AS "window_end", "event_type" AS "event_type", COUNT() AS "event_count", SUM("event_value") AS "total_value" FROM TUMBLE("click_events", "event_timestamp", INTERVAL '5 MINUTES') AS "_q_0" GROUP BY "window_start", "window_end", "event_type" (base.py:2260)
2025-08-19 09:49:08,442 - ThreadPoolExecutor-2_0 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: /* SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT 1 FROM "information_schema"."tables" WHERE "table_name" = 'sqlmesh_example__seed_model__2185867172' AND "table_schema" = 'sqlmesh__sqlmesh_example' (base.py:2260)
2025-08-19 09:49:08,446 - ThreadPoolExecutor-2_0 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / DELETE FROM "sqlmesh__sqlmesh_example"."sqlmesh_example__seed_model__2185867172" WHERE TRUE (base.py:2260)
2025-08-19 09:49:08,518 - ThreadPoolExecutor-2_0 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: INSERT INTO "sqlmesh__sqlmesh_example"."sqlmesh_example__seed_model__2185867172" ("id", "item_id", "event_date") SELECT CAST("id" AS INT) AS "id", CAST("item_id" AS INT) AS "item_id", CAST("event_date" AS DATE) AS "event_date" FROM (VALUES "") AS "t"("id", "item_id", "event_date") (base.py:2260)
2025-08-19 09:49:08,677 - ThreadPoolExecutor-2_2 - sqlmesh.core.snapshot.evaluator - INFO - Creating table 'dev.sqlmesh__sqlmesh_example.sqlmesh_example__incremental_model__2906841977' (evaluator.py:1538)
2025-08-19 09:49:08,678 - ThreadPoolExecutor-2_2 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / CREATE TABLE IF NOT EXISTS "sqlmesh__sqlmesh_example"."sqlmesh_example__incremental_model__2906841977" ("id" INT, "item_id" INT, "event_date" DATE) (base.py:2260)
2025-08-19 09:49:10,102 - ThreadPoolExecutor-2_2 - sqlmesh.core.snapshot.evaluator - INFO - Dry running model 'sqlmesh_example.incremental_model' (evaluator.py:1560)
2025-08-19 09:49:10,103 - ThreadPoolExecutor-2_2 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT "seed_model"."id" AS "id", "seed_model"."item_id" AS "item_id", "seed_model"."event_date" AS "event_date" FROM "dev"."sqlmesh__sqlmesh_example"."sqlmesh_example__seed_model__2185867172" AS "seed_model" WHERE ("seed_model"."event_date" <= CAST('1970-01-01' AS DATE) AND "seed_model"."event_date" >= CAST('1970-01-01' AS DATE)) AND FALSE LIMIT 0 (base.py:2260)
2025-08-19 09:49:10,112 - ThreadPoolExecutor-2_2 - sqlmesh.core.snapshot.evaluator - INFO - Creating table 'dev.sqlmesh__sqlmesh_example.sqlmesh_example__full_model__3332047715' (evaluator.py:1538)
2025-08-19 09:49:10,113 - ThreadPoolExecutor-2_2 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / CREATE TABLE IF NOT EXISTS "sqlmesh__sqlmesh_example"."sqlmesh_example__full_model__3332047715" ("item_id" INT, "num_orders" BIGINT) (base.py:2260)
2025-08-19 09:49:10,411 - ThreadPoolExecutor-2_2 - sqlmesh.core.snapshot.evaluator - INFO - Dry running model 'sqlmesh_example.full_model' (evaluator.py:1560)
2025-08-19 09:49:10,412 - ThreadPoolExecutor-2_2 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT "incremental_model"."item_id" AS "item_id", COUNT(DISTINCT "incremental_model"."id") AS "num_orders" FROM "dev"."sqlmesh__sqlmesh_example"."sqlmesh_example__incremental_model__2906841977" AS "incremental_model" WHERE FALSE GROUP BY "incremental_model"."item_id" LIMIT 0 (base.py:2260)
2025-08-19 09:49:10,417 - MainThread - sqlmesh.core.plan.evaluator - INFO - Evaluating plan stage BackfillStage (evaluator.py:125)
2025-08-19 09:49:10,486 - ThreadPoolExecutor-3_0 - sqlmesh.core.snapshot.evaluator - INFO - Evaluating snapshot SnapshotId<"dev"."reporting"."event_summary_tumbling": 50710608> (evaluator.py:648)
2025-08-19 09:49:10,487 - ThreadPoolExecutor-3_1 - sqlmesh.core.state_sync.db.facade - INFO - Adding interval (2025-08-18 00:00:00, 2025-08-19 00:00:00) for snapshot SnapshotId<"dev"."sqlmesh_example"."seed_model": 3467833469> (facade.py:619)
2025-08-19 09:49:10,493 - ThreadPoolExecutor-3_0 - sqlmesh.core.snapshot.evaluator - INFO - Inserting batch (2025-08-18 00:00:00, 2025-08-19 00:00:00) into dev.sqlmesh__reporting.reporting__event_summary_tumbling__3036268106' (evaluator.py:703)
2025-08-19 09:49:10,494 - ThreadPoolExecutor-3_1 - sqlmesh.core.state_sync.db.interval - INFO - Pushing intervals for snapshot SnapshotId<"dev"."sqlmesh_example"."seed_model": 3467833469> (interval.py:213)
2025-08-19 09:49:10,494 - ThreadPoolExecutor-3_0 - sqlmesh.core.snapshot.evaluator - INFO - Replacing view 'dev.sqlmesh__reporting.reporting__event_summary_tumbling__3036268106' (evaluator.py:1933)
2025-08-19 09:49:10,499 - ThreadPoolExecutor-3_0 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / CREATE MATERIALIZED VIEW "sqlmesh__reporting"."reporting__event_summary_tumbling__3036268106" AS SELECT "window_start" AS "window_start", "window_end" AS "window_end", "event_type" AS "event_type", COUNT() AS "event_count", SUM("event_value") AS "total_value" FROM TUMBLE("click_events", "event_timestamp", INTERVAL '5 MINUTES') AS "_q_0" GROUP BY "window_start", "window_end", "event_type" (base.py:2260)
2025-08-19 09:49:10,604 - ThreadPoolExecutor-3_1 - sqlmesh.core.snapshot.evaluator - INFO - Evaluating snapshot SnapshotId<"dev"."sqlmesh_example"."incremental_model": 4283318177> (evaluator.py:648)
2025-08-19 09:49:10,627 - ThreadPoolExecutor-3_1 - sqlmesh.core.snapshot.evaluator - INFO - Inserting batch (2020-01-01 00:00:00, 2025-08-19 00:00:00) into dev.sqlmesh__sqlmesh_example.sqlmesh_example__incremental_model__2906841977' (evaluator.py:703)
2025-08-19 09:49:10,633 - ThreadPoolExecutor-3_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: /* SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / DELETE FROM "sqlmesh__sqlmesh_example"."sqlmesh_example__incremental_model__2906841977" WHERE "event_date" BETWEEN CAST('2020-01-01' AS DATE) AND CAST('2025-08-18' AS DATE) (base.py:2260)
2025-08-19 09:49:10,681 - ThreadPoolExecutor-3_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / INSERT INTO "sqlmesh__sqlmesh_example"."sqlmesh_example__incremental_model__2906841977" ("id", "item_id", "event_date") SELECT "id", "item_id", "event_date" FROM (SELECT "seed_model"."id" AS "id", "seed_model"."item_id" AS "item_id", "seed_model"."event_date" AS "event_date" FROM "dev"."sqlmesh__sqlmesh_example"."sqlmesh_example__seed_model__2185867172" AS "seed_model" WHERE "seed_model"."event_date" <= CAST('2025-08-18' AS DATE) AND "seed_model"."event_date" >= CAST('2020-01-01' AS DATE)) AS "_subquery" WHERE "event_date" BETWEEN CAST('2020-01-01' AS DATE) AND CAST('2025-08-18' AS DATE) (base.py:2260)
2025-08-19 09:49:10,754 - ThreadPoolExecutor-3_1 - sqlmesh.core.state_sync.db.facade - INFO - Adding interval (2020-01-01 00:00:00, 2025-08-19 00:00:00) for snapshot SnapshotId<"dev"."sqlmesh_example"."incremental_model": 4283318177> (facade.py:619)
2025-08-19 09:49:10,754 - ThreadPoolExecutor-3_1 - sqlmesh.core.state_sync.db.interval - INFO - Pushing intervals for snapshot SnapshotId<"dev"."sqlmesh_example"."incremental_model": 4283318177> (interval.py:213)
2025-08-19 09:49:10,859 - ThreadPoolExecutor-3_1 - sqlmesh.core.snapshot.evaluator - INFO - Evaluating snapshot SnapshotId<"dev"."sqlmesh_example"."full_model": 237739027> (evaluator.py:648)
2025-08-19 09:49:10,869 - ThreadPoolExecutor-3_1 - sqlmesh.core.snapshot.evaluator - INFO - Inserting batch (2020-01-01 00:00:00, 2025-08-19 00:00:00) into dev.sqlmesh__sqlmesh_example.sqlmesh_example__full_model__3332047715' (evaluator.py:703)
2025-08-19 09:49:10,871 - ThreadPoolExecutor-3_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT 1 FROM "information_schema"."tables" WHERE "table_name" = 'sqlmesh_example__full_model__3332047715' AND "table_schema" = 'sqlmesh__sqlmesh_example' (base.py:2260)
2025-08-19 09:49:10,874 - ThreadPoolExecutor-3_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / DELETE FROM "sqlmesh__sqlmesh_example"."sqlmesh_example__full_model__3332047715" WHERE TRUE (base.py:2260)
2025-08-19 09:49:11,010 - ThreadPoolExecutor-3_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / INSERT INTO "sqlmesh__sqlmesh_example"."sqlmesh_example__full_model__3332047715" ("item_id", "num_orders") SELECT "incremental_model"."item_id" AS "item_id", COUNT(DISTINCT "incremental_model"."id") AS "num_orders" FROM "dev"."sqlmesh__sqlmesh_example"."sqlmesh_example__incremental_model__2906841977" AS "incremental_model" GROUP BY "incremental_model"."item_id" (base.py:2260)
2025-08-19 09:49:11,111 - ThreadPoolExecutor-3_1 - sqlmesh.core.snapshot.evaluator - INFO - Auditing snapshot SnapshotId<"dev"."sqlmesh_example"."full_model": 237739027> (evaluator.py:553)
2025-08-19 09:49:11,115 - ThreadPoolExecutor-3_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT "rw_columns"."name" AS "column_name", "rw_columns"."data_type" AS "data_type" FROM "rw_catalog"."rw_columns" JOIN "rw_catalog"."rw_relations" ON "rw_relations"."id" = "rw_columns"."relation_id" JOIN "rw_catalog"."rw_schemas" ON "rw_schemas"."id" = "rw_relations"."schema_id" WHERE ("rw_relations"."name" = 'sqlmesh_example__full_model__3332047715' AND "rw_columns"."name" <> '_row_id' AND "rw_columns"."name" <> '_rw_timestamp') AND "rw_schemas"."name" = 'sqlmesh__sqlmesh_example' (base.py:2260)
2025-08-19 09:49:11,124 - ThreadPoolExecutor-3_1 - sqlmesh.core.engine_adapter.base - INFO - Executing SQL: / SQLMESH_PLAN: 5484d15aea174b18ad893df45542011a / SELECT COUNT() FROM (SELECT * FROM "dev"."sqlmesh__sqlmesh_example"."sqlmesh_example__full_model__3332047715" AS "sqlmesh_example__full_model__3332047715" WHERE "item_id" < 0) AS "audit" (base.py:2260)
2025-08-19 09:49:11,129 - ThreadPoolExecutor-3_1 - sqlmesh.core.state_sync.db.facade - INFO - Adding interval (2020-01-01 00:00:00, 2025-08-19 00:00:00) for snapshot SnapshotId<"dev"."sqlmesh_example"."full_model": 237739027> (facade.py:619)
2025-08-19 09:49:11,129 - ThreadPoolExecutor-3_1 - sqlmesh.core.state_sync.db.interval - INFO - Pushing intervals for snapshot SnapshotId<"dev"."sqlmesh_example"."full_model": 237739027> (interval.py:213)
2025-08-19 09:49:11,213 - MainThread - sqlmesh.core.scheduler - INFO - Execution failed for node ('"dev"."reporting"."event_summary_tumbling"', ((1755475200000, 1755561600000), 0)) (scheduler.py:499)
Traceback (most recent call last):
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/utils/concurrency.py", line 69, in _process_node
self.fn(node)
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/scheduler.py", line 457, in evaluate_node
audit_results = self.evaluate(
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/scheduler.py", line 188, in evaluate
wap_id = self.snapshot_evaluator.evaluate(
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/snapshot/evaluator.py", line 161, in evaluate
result = self._evaluate_snapshot(
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/snapshot/evaluator.py", line 802, in _evaluate_snapshot
apply(query_or_df, index)
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/snapshot/evaluator.py", line 709, in apply
evaluation_strategy.insert(
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/snapshot/evaluator.py", line 1934, in insert
self.adapter.create_view(
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/engine_adapter/shared.py", line 341, in internal_wrapper
return func(*list_args, **kwargs)
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/engine_adapter/base_postgres.py", line 113, in create_view
super().create_view(
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/engine_adapter/shared.py", line 326, in internal_wrapper
return func(*list_args, **kwargs)
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/engine_adapter/base.py", line 1122, in create_view
self.execute(
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/engine_adapter/base.py", line 2237, in execute
self._execute(sql, **kwargs)
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/engine_adapter/base.py", line 2263, in execute
self.cursor.execute(sql, **kwargs)
psycopg2.errors.InternalError: Failed to run the query

Caused by these errors (recent errors listed first):
1: Catalog error
2: materialized view with name reporting__event_summary_tumbling__3036268106 exists

The above exception was the direct cause of the following exception:

sqlmesh.utils.concurrency.NodeExecutionFailedError: Execution failed for node ('"dev"."reporting"."event_summary_tumbling"', ((1755475200000, 1755561600000), 0))
2025-08-19 09:49:11,226 - MainThread - sqlmesh.core.context - INFO - Plan application failed. (context.py:1702)
Traceback (most recent call last):
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/context.py", line 1694, in apply
self._apply(plan, circuit_breaker)
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/context.py", line 2487, in _apply
self._scheduler.create_plan_evaluator(self).evaluate(
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/plan/evaluator.py", line 107, in evaluate
self._evaluate_stages(plan_stages, plan)
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/plan/evaluator.py", line 127, in _evaluate_stages
handler(stage, plan)
File "<...>/sqlmesh-risingwave-demo/venv/lib/python3.10/site-packages/sqlmesh/core/plan/evaluator.py", line 254, in visit_backfill_stage
raise PlanError("Plan application failed.")
sqlmesh.utils.errors.PlanError: Plan application failed.

Affected SQLMesh versions

0.200.0 and probably all the later versions including the latest one

The bug doesn't appear in 0.197.4

Possible cause

A likely cause of this bug appears to be the changes introduced in PR #4908, as reverting these changes from 0.200.0 seems to resolve the issue.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Bug

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions