Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move materialize issue references to database-issues #29773

Merged
merged 1 commit into from
Sep 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
34 changes: 17 additions & 17 deletions ci/nightly/pipeline.template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -656,7 +656,7 @@ steps:
- id: checks-parallel-drop-create-default-replica
label: "Checks parallel + DROP/CREATE replica %N"
depends_on: build-aarch64
skip: "Affected by materialize#23882"
skip: "Affected by database-issues#7172"
timeout_in_minutes: 180
parallelism: 2
agents:
Expand All @@ -669,7 +669,7 @@ steps:
- id: checks-parallel-restart-clusterd-compute
label: "Checks parallel + restart compute clusterd %N"
depends_on: build-aarch64
skip: "Affected by materialize#23882"
skip: "Affected by database-issues#7172"
timeout_in_minutes: 180
parallelism: 2
agents:
Expand All @@ -682,7 +682,7 @@ steps:
- id: checks-parallel-restart-entire-mz
label: "Checks parallel + restart of the entire Mz %N"
depends_on: build-aarch64
skip: "Affected by materialize#23882"
skip: "Affected by database-issues#7172"
timeout_in_minutes: 180
parallelism: 2
agents:
Expand All @@ -695,7 +695,7 @@ steps:
- id: checks-parallel-restart-environmentd-clusterd-storage
label: "Checks parallel + restart of environmentd & storage clusterd %N"
depends_on: build-aarch64
skip: "Affected by materialize#23882"
skip: "Affected by database-issues#7172"
timeout_in_minutes: 180
parallelism: 2
agents:
Expand All @@ -708,7 +708,7 @@ steps:
- id: checks-parallel-kill-clusterd-storage
label: "Checks parallel + kill storage clusterd %N"
depends_on: build-aarch64
skip: "Affected by materialize#23882"
skip: "Affected by database-issues#7172"
timeout_in_minutes: 180
parallelism: 2
agents:
Expand All @@ -721,7 +721,7 @@ steps:
- id: checks-parallel-restart-redpanda
label: "Checks parallel + restart Redpanda & Debezium %N"
depends_on: build-aarch64
skip: "Affected by materialize#23882"
skip: "Affected by database-issues#7172"
timeout_in_minutes: 180
parallelism: 2
agents:
Expand Down Expand Up @@ -873,7 +873,7 @@ steps:
label: "K8s recovery: storage on failing node"
depends_on: build-aarch64
timeout_in_minutes: 60
# TODO: materialize#25108 (k8s node recovery tests flaky)
# TODO: database-issues#7499 (k8s node recovery tests flaky)
retry:
automatic:
- exit_status: 1
Expand All @@ -899,7 +899,7 @@ steps:
label: "K8s recovery: compute on failing node"
depends_on: build-aarch64
timeout_in_minutes: 60
# TODO: materialize#25108 (k8s node recovery tests flaky)
# TODO: database-issues#7499 (k8s node recovery tests flaky)
retry:
automatic:
- exit_status: 1
Expand All @@ -925,7 +925,7 @@ steps:
label: "K8s recovery: replicated compute on failing node"
depends_on: build-aarch64
timeout_in_minutes: 60
# TODO: materialize#25108 (k8s node recovery tests flaky)
# TODO: database-issues#7499 (k8s node recovery tests flaky)
retry:
automatic:
- exit_status: 1
Expand All @@ -951,7 +951,7 @@ steps:
label: "K8s recovery: envd on failing node"
depends_on: build-aarch64
timeout_in_minutes: 60
# TODO: materialize#25108 (k8s node recovery tests flaky)
# TODO: database-issues#7499 (k8s node recovery tests flaky)
retry:
automatic:
- exit_status: 1
Expand Down Expand Up @@ -1157,7 +1157,7 @@ steps:
plugins:
- ./ci/plugins/mzcompose:
composition: sqlsmith
# TODO(def-) Increase number of joins when materialize#23451 is fixed
# TODO(def-) Increase number of joins when database-issues#7046 is fixed
args: [--max-joins=1, --runtime=1500]

- id: sqlsmith-explain
Expand All @@ -1169,7 +1169,7 @@ steps:
plugins:
- ./ci/plugins/mzcompose:
composition: sqlsmith
# TODO(def-) Increase number of joins when materialize#23451 is fixed
# TODO(def-) Increase number of joins when database-issues#7046 is fixed
args: [--max-joins=5, --explain-only, --runtime=1500]

- group: SQLancer
Expand Down Expand Up @@ -1232,7 +1232,7 @@ steps:
- ./ci/plugins/mzcompose:
composition: rqg
args: ["simple-aggregates", "--seed=$BUILDKITE_JOB_ID"]
skip: "flaky until materialize#27954 is fixed"
skip: "flaky until database-issues#8197 is fixed"

- id: rqg-db3-joins
label: "RQG dbt3-joins workload"
Expand All @@ -1249,7 +1249,7 @@ steps:
- id: rqg-lateral-joins
label: "RQG lateral-joins workload"
depends_on: build-aarch64
skip: "flaky until materialize#25865 is fixed"
skip: "flaky until database-issues#7713 is fixed"
timeout_in_minutes: 45
agents:
queue: hetzner-aarch64-4cpu-8gb
Expand Down Expand Up @@ -1283,7 +1283,7 @@ steps:
- id: rqg-wmr
label: "RQG WMR workload"
depends_on: build-aarch64
skip: "flaky until materialize#24904 is fixed"
skip: "flaky until database-issues#7433 is fixed"
timeout_in_minutes: 45
agents:
queue: hetzner-aarch64-8cpu-16gb
Expand Down Expand Up @@ -1339,7 +1339,7 @@ steps:
- id: retain-history
label: "Check retain history"
depends_on: build-aarch64
skip: "materialize#24479"
skip: "database-issues#7310"
timeout_in_minutes: 15
agents:
queue: hetzner-aarch64-8cpu-16gb
Expand Down Expand Up @@ -1440,7 +1440,7 @@ steps:
depends_on: build-aarch64
artifact_paths: [parallel-workload-queries.log.zst]
timeout_in_minutes: 90
skip: "TODO(def-): Reenable when materialize#2392 is fixed"
skip: "TODO(def-): Reenable when database-issues#835 is fixed"
agents:
queue: hetzner-aarch64-16cpu-32gb
plugins:
Expand Down
2 changes: 1 addition & 1 deletion ci/plugins/cloudtest/hooks/post-command
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ buildkite-agent artifact upload "$artifacts_str"
bin/ci-builder run stable bin/ci-annotate-errors --test-cmd="$(cat test_cmd)" "${artifacts[@]}" > ci-annotate-errors.log || CI_ANNOTATE_ERRORS_RESULT=$?
buildkite-agent artifact upload "ci-annotate-errors.log"

# File should not be empty, see materialize#25369
# File should not be empty, see database-issues#7569
test -s kubectl-get-logs-previous.log

if [ "$CI_ANNOTATE_ERRORS_RESULT" -ne 0 ]; then
Expand Down
8 changes: 4 additions & 4 deletions ci/release-qualification/pipeline.template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -115,7 +115,7 @@ steps:
plugins:
- ./ci/plugins/mzcompose:
composition: zippy
# Execution times longer than 4h are apparently not feasible at this time due to materialize#17845
# Execution times longer than 4h are apparently not feasible at this time due to database-issues#5201
args: [--scenario=ClusterReplicas, --actions=10000, --max-execution-time=4h]

- id: zippy-user-tables-large
Expand Down Expand Up @@ -166,7 +166,7 @@ steps:
plugins:
- ./ci/plugins/mzcompose:
composition: zippy
# TODO(def-) Increase number of actions when materialize#24250 is fixed
# TODO(def-) Increase number of actions when database-issues#7257 is fixed
args: [--scenario=KafkaParallelInsert, --transaction-isolation=serializable, --actions=50000, --max-execution-time=4h]

- group: Benchmarks
Expand Down Expand Up @@ -216,7 +216,7 @@ steps:
- ./ci/plugins/mzcompose:
composition: sqlsmith
args: [--max-joins=2, --runtime=6000]
skip: "Reenable when materialize#23451 is fixed"
skip: "Reenable when database-issues#7046 is fixed"

- id: sqlsmith-explain-long
label: "Longer SQLsmith explain"
Expand All @@ -228,7 +228,7 @@ steps:
- ./ci/plugins/mzcompose:
composition: sqlsmith
args: [--max-joins=15, --explain-only, --runtime=6000]
skip: "Reenable when materialize#23451 is fixed"
skip: "Reenable when database-issues#7046 is fixed"

- id: test-preflight-check-rollback
label: Test with preflight check and rollback
Expand Down
2 changes: 1 addition & 1 deletion ci/test/cargo-test/mzcompose.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ def workflow_default(c: Composition, parser: WorkflowArgumentParser) -> None:
args = parser.parse_args()
c.up("zookeeper", "kafka", "schema-registry", "postgres", "cockroach", "minio")
# Heads up: this intentionally runs on the host rather than in a Docker
# image. See materialize#13010.
# image. See database-issues#3739.
postgres_url = (
f"postgres://postgres:postgres@localhost:{c.default_port('postgres')}"
)
Expand Down
2 changes: 1 addition & 1 deletion ci/test/pipeline.template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -747,7 +747,7 @@ steps:
composition: rtr-combined
agents:
queue: hetzner-aarch64-8cpu-16gb
skip: "Flakes because of materialize#29289"
skip: "Flakes because of database-issues#8489"

- id: skip-version-upgrade
label: "Skip Version Upgrade"
Expand Down
2 changes: 1 addition & 1 deletion clippy.toml
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ disallowed-methods = [

disallowed-macros = [
{ path = "proptest::prop_oneof", reason = "use `proptest::strategy::Union::new` instead" },
{ path = "log::log", reason = "use the macros provided by `tracing` instead (materialize#9992)" },
{ path = "log::log", reason = "use the macros provided by `tracing` instead (database-issues#3001)" },
{ path = "tracing::instrument", reason = "use `mz_ore::instrument` instead" },
]

Expand Down
2 changes: 1 addition & 1 deletion doc/developer/design/20210311_prometheus_source.md
Original file line number Diff line number Diff line change
Expand Up @@ -134,4 +134,4 @@ I think if we want to go down this road, we'll want something like an "is a" tab
// These questions may be technical, product, or anything in-between.
-->

[#5825]: https://github.com/MaterializeInc/materialize/issues/5825
[#5825]: https://github.com/MaterializeInc/database-issues/issues/1793
Original file line number Diff line number Diff line change
Expand Up @@ -287,5 +287,5 @@ CREATE OR REPLACE VIEW kafka_view AS SELECT col1, col2, col3 FROM kafka_src;
CREATE VIEW view3 AS SELECT col3 FROM kafka_view;
```

[#3791]: https://github.com/MaterializeInc/materialize/issues/3791
[#3791]: https://github.com/MaterializeInc/database-issues/issues/1182
[jettison-discussion]: https://github.com/MaterializeInc/materialize/pull/6450/files#r612735779
2 changes: 1 addition & 1 deletion doc/developer/design/20210601_build_mirrelationexpr.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ Figuring out how to more efficiently implement the syntax has been an ongoing
issue. There has been some earlier design discussions on this, which are
documented in the comments of [#5684].

[#5684]: https://github.com/MaterializeInc/materialize/issues/5684
[#5684]: https://github.com/MaterializeInc/database-issues/issues/1750

## Goals

Expand Down
22 changes: 11 additions & 11 deletions doc/developer/design/20210707_qgm_sql_high_level_representation.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,16 +32,16 @@ and the one proposed in this document, which adds normalization at the SQL level
* be easy to use,
* be normalization-friendly,
* allow supporting complex features such as recursion in CTEs,
* Proper support of `LATERAL` joins ([#6875](https://github.com/MaterializeInc/materialize/issues/6875))
* Proper support of `LATERAL` joins ([#6875](https://github.com/MaterializeInc/database-issues/issues/2143))
* Support for functional dependency analysis during name resolution
([#7647](https://github.com/MaterializeInc/materialize/issues/7647)).
([#7647](https://github.com/MaterializeInc/database-issues/issues/2369)).

Existing issues that should be addressed sooner rather than later:
* [sql: handle aggregates that refer exclusively to outer columns #3720](https://github.com/MaterializeInc/materialize/issues/3720)
* [sql: handle aggregates that refer exclusively to outer columns #3720](https://github.com/MaterializeInc/database-issues/issues/1163)
* [sql: make SQL responsible for its equality semantics #7597](https://github.com/MaterializeInc/materialize/pull/7597)
* [sql: wrong results with outer join due to incorrect name resolution #7618](https://github.com/MaterializeInc/materialize/issues/7618)
* [sql: ambiguous column name error reported for equivalent expressions in the selection list](https://github.com/MaterializeInc/materialize/issues/7549)
* [Ignore the selection list in EXISTS subqueries](https://github.com/MaterializeInc/materialize/issues/6490)
* [sql: wrong results with outer join due to incorrect name resolution #7618](https://github.com/MaterializeInc/database-issues/issues/2364)
* [sql: ambiguous column name error reported for equivalent expressions in the selection list](https://github.com/MaterializeInc/database-issues/issues/2340)
* [Ignore the selection list in EXISTS subqueries](https://github.com/MaterializeInc/database-issues/issues/2020)

## Non-Goals

Expand Down Expand Up @@ -168,7 +168,7 @@ the query or discovered via some query transformation.

As shown above, there aren't many different types of operators, since QGM is meant to be a representation for
query normalization. The set of operators listed above is very close to the one suggested in
[#692](https://github.com/MaterializeInc/materialize/issues/692).
[#692](https://github.com/MaterializeInc/database-issues/issues/224).

The core operator is represented by the `Select` box, which represents a whole query block (sub-block).

Expand Down Expand Up @@ -405,7 +405,7 @@ A `SELECT` query is a grouping query if any of the following conditions is met:
* the projection of the query contains any aggregate which parameters are constant,
* the projection of the query contains any aggregate where all columns referenced within it come from the tables
in the `FROM` clause, either directly in the projection or within a subquery
(see [#3720](https://github.com/MaterializeInc/materialize/issues/3720))
(see [#3720](https://github.com/MaterializeInc/database-issues/issues/1163))

That means that in order to determine whether a query is a grouping query or not, we must inspect the projection
of the query first. For this reason, after having processed the `FROM` clause and the `WHERE` clause, we will
Expand Down Expand Up @@ -502,9 +502,9 @@ the nested `Select` box would not be guaranteed and hence, it could not be merge

Some normalization transformations are better/easier done with a representation at a higher level than our current
`MirRelationExpr` representation. Specially those around SQL-specific concepts such as outer joins that are
lost during lowering. Several examples of this are [#6932](https://github.com/MaterializeInc/materialize/issues/6932),
[#6987](https://github.com/MaterializeInc/materialize/issues/6987) or
[#6988](https://github.com/MaterializeInc/materialize/issues/6988), but the list of unsupported cases that are
lost during lowering. Several examples of this are [#6932](https://github.com/MaterializeInc/database-issues/issues/2155),
[#6987](https://github.com/MaterializeInc/database-issues/issues/2174) or
[#6988](https://github.com/MaterializeInc/database-issues/issues/2175), but the list of unsupported cases that are
hard to support at the moment is much longer.

### Query decorrelation during normalization
Expand Down
2 changes: 1 addition & 1 deletion doc/developer/design/20210713_S3_sources_with_headers.md
Original file line number Diff line number Diff line change
Expand Up @@ -209,7 +209,7 @@ schema we will put its dataflow into an error state to prevent users from
reading invalid data. Currently there is no way to recover from this state.
There are a few options:

* Issue [#6605](https://github.com/MaterializeInc/materialize/issues/6605#issuecomment-880101445)
* Issue [#6605](https://github.com/MaterializeInc/database-issues/issues/2060#issuecomment-880101445)
suggests that with a full sql-accessible catalog we could use standard sql to
retract S3 objects that are invalid, thereby recovering.
* We could add specific syntax for S3 sources that allow retracting invalid CSV
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -166,20 +166,20 @@ There were some, which lead to the addition of a `DEGRADED` state above.
## References

- Product thoughts from Nicolle: https://docs.google.com/document/d/10RsEnpJJBN-lQKyYl08KRQ15Ckn-_gCWa5MAnZcc-ts/edit#heading=h.yvax2aldz1n9
- Meta-issue by Eli: https://github.com/MaterializeInc/materialize/issues/7115
- Meta-issue by Eli: https://github.com/MaterializeInc/database-issues/issues/2208

Critical:
- https://github.com/MaterializeInc/materialize/issues/7396: sink error during startup kills materialized
- https://github.com/MaterializeInc/materialize/issues/7009: failure in S3 source is printed to log but source doesn't error out
- https://github.com/MaterializeInc/materialize/issues/6586: transient broker failure leads to error in log, but source doesn't report as errored and doesn't continue to produce data
- https://github.com/MaterializeInc/database-issues/issues/2304: sink error during startup kills materialized
- https://github.com/MaterializeInc/database-issues/issues/2180: failure in S3 source is printed to log but source doesn't error out
- https://github.com/MaterializeInc/database-issues/issues/2051: transient broker failure leads to error in log, but source doesn't report as errored and doesn't continue to produce data

Unknown:
- https://github.com/MaterializeInc/materialize/issues/7035: fetching in source doesn't continue after transient failure
- https://github.com/MaterializeInc/materialize/issues/6916: metadata fetch errors in log
- https://github.com/MaterializeInc/database-issues/issues/2187: fetching in source doesn't continue after transient failure
- https://github.com/MaterializeInc/database-issues/issues/2153: metadata fetch errors in log

Related:
- https://github.com/MaterializeInc/materialize/issues/3122: "no complete timestamps" message is not useful
- related: https://github.com/MaterializeInc/materialize/issues/2917
- https://github.com/MaterializeInc/database-issues/issues/1024: "no complete timestamps" message is not useful
- related: https://github.com/MaterializeInc/database-issues/issues/978

Future Work, aka. non-goals:
- https://github.com/MaterializeInc/materialize/discussions/5340: more structured user errors (error, details, hints)
Expand Down
Loading
Loading