Skip to content

branch-4.0: [fix](Rollup) When creating a table with rollup and setting a sequence column, we should correctly configure the uniqueId of the sequence column #56143 #57389

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Oct 28, 2025
Merged

branch-4.0: [fix](Rollup) When creating a table with rollup and setting a sequence column, we should correctly configure the uniqueId of the sequence column #56143 #57389

merged 1 commit into from
Oct 28, 2025
+61 −7

Conversation

@github-actions
Copy link
Contributor

Cherry-picked from #56143

@qzsee
[fix](Rollup) When creating a table with rollup and setting a sequenc…
4aef4f5 
…e column, we should correctly configure the uniqueId of the sequence column (#56143)

Doris version master 

```sql
CREATE TABLE IF NOT EXISTS users( 
    user_id VARCHAR(32) NOT NULL,
    event_time DATETIME NOT NULL,
    val1 BIGINT NOT NULL
) ENGINE=OLAP
UNIQUE KEY(user_id, event_time)
DISTRIBUTED BY HASH(user_id) BUCKETS 1
ROLLUP ( rlp_spacetime_collision(event_time, user_id) )
PROPERTIES (
   "replication_allocation" = "tag.location.default: 1",
   "function_column.sequence_col" = "event_time"
);

mysql> insert into users values(1,'2025-01-01 00:00:00',1);Query OK, 1 row affected (0.18 sec)

mysql> select * from users1;
ERROR 1105 (HY000): errCode = 2, detailMessage = (9.134.213.24)[E-207]slot(id: 3, name: __DORIS_DELETE_SIGN__)'s nullable does not match column(tablet id: -1, index: 5, name: __DORIS_SEQUENCE_COL__) 

```

The reasons are as follows.

all `MaterializedIndexMeta` shared one `sequenceCol`

```java

public void setSequenceInfo(Type type, Column refColumn) {
        this.hasSequenceCol = true;
        this.sequenceType = type;

        Column sequenceCol;
        if (getEnableUniqueKeyMergeOnWrite()) {
            // sequence column is value column with NONE aggregate type for
            // unique key table with merge on write
            sequenceCol = ColumnDef.newSequenceColumnDef(type, AggregateType.NONE).toColumn();
        } else {
            // sequence column is value column with REPLACE aggregate type for
            // unique key table
            sequenceCol = ColumnDef.newSequenceColumnDef(type, AggregateType.REPLACE).toColumn();
        }
        if (refColumn != null) {
            sequenceCol.setDefaultValueInfo(refColumn);
        }
        // add sequence column at last
        fullSchema.add(sequenceCol);
        nameToColumn.put(Column.SEQUENCE_COL, sequenceCol);
        for (MaterializedIndexMeta indexMeta : indexIdToMeta.values()) {
            List<Column> schema = indexMeta.getSchema();
            schema.add(sequenceCol);
        }
    }

```

when init column `uniqueId`, base index sequence column unique id
override roullup index sequence column `uniqueId `
```java
    public void initSchemaColumnUniqueId() {
        maxColUniqueId = Column.COLUMN_UNIQUE_ID_INIT_VALUE;
        this.schema.forEach(column -> {
            column.setUniqueId(incAndGetMaxColUniqueId());
            if (LOG.isDebugEnabled()) {
                LOG.debug("indexId: {},  column:{}, uniqueId:{}",
                        indexId, column, column.getUniqueId());
            }
        });
    }

```

then BE  will make an incorrect judgment.

### Release note

None

### Check List (For Author)

- Test <!-- At least one of them must be included. -->
    - [ ] Regression test
    - [ ] Unit Test
    - [ ] Manual test (add detailed scripts or steps below)
    - [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
        - [ ] Previous test can cover this change.
        - [ ] No code files have been changed.
        - [ ] Other reason <!-- Add your reason?  -->

- Behavior changed:
    - [ ] No.
    - [ ] Yes. <!-- Explain the behavior change -->

- Does this need documentation?
    - [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
apache/doris-website#1214 -->

### Check List (For Reviewer who merge this PR)

- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR
should merge into -->

---------

Co-authored-by: garenshi <[email protected]>
@github-actions github-actions bot requested a review from yiguolei as a code owner October 27, 2025 17:23
@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@dataroaring dataroaring reopened this Oct 27, 2025
@hello-stephen
Copy link
Contributor

run buildall

@hello-stephen
Copy link
Contributor

FE UT Coverage Report

Increment line coverage `` 🎉
Increment coverage report
Complete coverage report

@doris-robot
Copy link

ClickBench: Total hot run time: 30.01 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 4aef4f58461341f54e8165445217ee721abaab95, data reload: true

query1	0.04	0.03	0.03
query2	0.11	0.05	0.03
query3	0.25	0.07	0.07
query4	1.62	0.11	0.11
query5	0.26	0.26	0.27
query6	1.19	0.65	0.64
query7	0.04	0.02	0.02
query8	0.06	0.04	0.04
query9	0.62	0.52	0.54
query10	0.57	0.58	0.57
query11	0.15	0.12	0.11
query12	0.15	0.12	0.12
query13	0.63	0.61	0.61
query14	0.79	0.84	0.82
query15	0.87	0.88	0.86
query16	0.38	0.40	0.39
query17	1.03	1.05	1.07
query18	0.19	0.19	0.18
query19	1.92	1.87	1.89
query20	0.02	0.02	0.01
query21	15.40	0.92	0.57
query22	0.77	1.22	0.73
query23	15.03	1.40	0.59
query24	16.29	0.62	0.30
query25	0.30	0.14	0.08
query26	0.42	0.17	0.12
query27	0.05	0.04	0.04
query28	8.26	0.98	0.93
query29	12.60	3.99	3.26
query30	0.29	0.12	0.12
query31	2.88	0.58	0.37
query32	3.26	0.56	0.48
query33	3.04	3.06	3.09
query34	17.13	5.84	5.09
query35	5.26	5.08	5.21
query36	0.73	0.53	0.53
query37	0.12	0.07	0.07
query38	0.07	0.05	0.03
query39	0.04	0.03	0.03
query40	0.18	0.16	0.14
query41	0.09	0.03	0.06
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 113.18 s
Total hot run time: 30.01 s

@yiguolei yiguolei merged commit cbf7e54 into branch-4.0 Oct 28, 2025
25 of 27 checks passed
@github-actions github-actions bot deleted the auto-pick-56143-branch-4.0 branch October 28, 2025 03:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yiguolei yiguolei yiguolei approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hello-stephen @doris-robot @yiguolei @dataroaring @qzsee