Table Visualization distortion of big numbers on UInt64 type

[Nopius]

Issue Summary

Bug appers with clickhouse data source and big numbers with UInt64 data type. Problem is proved to be on frontend/Visualizer side, because the returned XHR result is in correct format and with correct data.

Steps to Reproduce

1. Create clickhouse data source.
2. Create query in clickhouse data source:

SELECT 8722821678000304785

Note, this value is lower than max UInt64 (18,446,744,073,709,551,615)
3. Execute the query with network debug turned on
4. Check the network response:

{
    "query_result": {
        "id": 7932109,
        "query_hash": "38c14fd87388f3b88a159d215bfac6c8",
        "query": "select 8722821678000304785 LIMIT 1000",
        "data": {
            "columns": [
                {
                    "name": "8722821678000304785",
                    "friendly_name": "8722821678000304785",
                    "type": "integer"
                }
            ],
            "rows": [
                {
                    "8722821678000304785": 8722821678000304785
                }
            ]
        },
        "data_source_id": 67,
        "runtime": 0.4866828918457031,
        "retrieved_at": "2023-07-10T01:20:52.933Z"
    }
}

5. Check visualizated value (wrong): 8,722,821,678,000,305,152
   

NOTES

1. When saving in CSV/TSV the values are correct, the problem appears only in a browser.
2. The same distortion happens on real data when we select from a clickhouse table UInt64 field with a big value
3. The value is also lower than signed Int64 (which is 9,223,372,036,854,775,807)
4. Changing field type to 'text' in visualizer doesn't help.

Technical details:

• Redash 10/Linux
• Chrome/Windows:
• How did you install Redash: docker image, runs in k8s

[justinclift]

I remember running into something very like this a while back, and I think from rough memory it's this:

https://discuss.redash.io/t/a-numeric-column-in-sql-shows-different-value-than-it-actually-contains-in-redash-display/9587/14

[justinclift]

There is a suggested way forward for fixing it too, we'd just need someone to do the implementation. Might not even be too hard either. 😄

[eradman]

The root cause is that the underlying number parsing and formatting library (numeral.js) only work with standard javascript numbers, not BigInt. The same is true for numbro.js.

[justinclift]

Yeah. We'd need to figure out a "better" approach to solve this. It might be a substantial piece of work, or could turn out to not be. No idea personally. 😄

[justinclift]

@eradman Would this be of use for solving the problem?

https://github.com/MikeMcl/bignumber.js

[eradman]

The effort to solving this would be huge and there is very little benefit. People cannot read such large numbers, hence a number such as 87228216780304785 will nearly always formatted in a human readable form, such as 77.47 PiB.

This is surprising at first, but this is how JavaScript numbers work, and it is not unique to Redash. Also CSV/TSV export is generated in Python without a loss of precision.

[justinclift]

People cannot read such large numbers ...

Ahhh, thanks. We have a customer affected by the incorrect display of large values.

They're working around it currently by casting it to a string for the text representation, but they'd like it fixed at some point.

When I have spare time to properly get up to speed with this part of things I should have a go at fixing it. 😄