Dbeaver CE is very slow on some Redshift quieries

[klingsoft]

Description

Suddenly, maybe after last dbeaver Community Edition upgrade, or after some update in the database, I got very long response times on some queries running against our RedShift database which exposes external tables for data stored in S3.

The query is a plain “select * from” which normally takes under a second.

My colleague is running an older version of dbeaver CE, and has normal response times. I tried to run this in dbeaver Ultimate, and had normal response times again. Also running dbVisualizer is normal in response times.

I have switched back and forth between the dbeaver versions, and always have slow responses in CE.

As we can see in the attached screenshot, most of the time is spent to get the data (“Read data from container”/”Fetch result set”). First step to execute the query is sub-second.

Is there any other logs I can provide to show what is going on or to understand the delays? I could not see any traces in the application log.

This does not apply to all databases with similar setup, or even for all schemas in the problematic database.

DBeaver Version

Community Edision 24.2.5.202411171748

Operating System

Windows 11

Database and driver

Server:
Redshift 8.0.2
PostgreSQL 8.0.2 on i686-pc-linux-gnu, compiled by GCC gcc (GCC) 3.4.2 20041017 (Red Hat 3.4.2-6.fc3), Redshift 1.0.78987

Driver:
Redshift JDBC Driver 2.1.0.3

Steps to reproduce

No response

Additional context

No response

[Akmanar]

We are also facing this problem with a external database with files in S3. Approximately 150M rows a day during a year, so you know how big we are talking. We would deeply appreciate a fix for this one team, thanks for your hard work.

[ShadelessFox]

Hello @klingsoft @Akmanar,

Do you have antivirus enabled? Can you try adding DBeaver's executable to the exclusion list to see if it somehow affects results?

[Akmanar]

@ShadelessFox thanks for coming back to us. We are in a a few tricky weeks as we go into Winter PTOs but I will most definitely involve our IT department and try to test this out for you asap. Will circle back here once we have tried that approach.

[Akmanar]

@ShadelessFox The one thing that I've noticed we didn't yet add to this thread is the fact that we have tried alternatives such as https://marketplace.visualstudio.com/items?itemName=cweijan.dbclient-jdbc for vcode and with that extension we are able to query the table and it works seamlessly.
Needlessly to say that vcode is widely used in our company and hence your antivirus test suggestion would still make sense, as problems with data coming and going to/from vcode would have affected many more teams, whereas dbeaver is only used by some selected teams such as mine.

[Akmanar]

In the meantime for what is worth, our antivirus is Crowdstrike Falcon

[klingsoft]

Hello @klingsoft @Akmanar,

Do you have antivirus enabled? Can you try adding DBeaver's executable to the exclusion list to see if it somehow affects results?

I am afraid that is not possible due to company policies.

[E1izabeth]

Is this Thread safe driver setting turned off?

[klingsoft]

It is not checked:

The difference I can see is that my driver says "PostgreSQL" in the "Driver type" field

[E1izabeth]

The difference I can see is that my driver says "PostgreSQL" in the "Driver type" field

That's because I made a screenshot of the DBeaver Ultimate version. We have more advanced support for Redshift there.

[shuiting]

Having a similar issue with a Redshift database. Recently updated the CE software to the newest version and a simple 'select * from table' type of query that should run in seconds is taking anywhere from 3 - 30+ minutes in the 'Read data from container' stage. It only happens with DBeaver CE as I can use other ways of connecting to and querying against the database (like TablePlus and Python) to get the same data in a matter of seconds.

I've tried uninstalling DBeaver CE, restarting my machine, and then re-installing it with various versions. Additionally, I've reset all of my settings after the issue started happening and that appears to have done nothing or possibly made it worse.

I've also tried toggling the 'Thread safe driver' on and off, but that doesn't appear to change anything.

Any help would be appreciated as the current issue makes the software unusable on my end.

[klingsoft]

To clarify: DBeaver Ultimate (Trial version) was better than Community edition (version 24).

Reverting back to Community edition version 23.3.5 fixed the problem.

[E1izabeth]

Could you try turning ON this Thread safe driver setting? Does it help?

[klingsoft]

No difference with that setting in version 24.
Switching back to version 23 and it is also much slower. Not as slow as in version 24, but over a minute to get the data.
After this, nothing helps with this setting to get the speed back in version 23.

However, after downloading a new Redshift JDBC driver it looks more promising.
I downloaded the redshift-jdbc42-2.1.0.32.jar, and created a new driver in dbeaver with this jar.
This runs in a second in a new connection I created with the new driver.

But I do not get it to work when I try to update the current Redshift driver to use the new jar. This is to avoid having to recreate all current connections to different RedShift databases.
The same jar is now used in both drivers, but for some reason dbeaver thinks one is a postgres driver (and this is fast), and the old one is a Redshift driver (and is slow).

New Driver with the fast query:

Old Driver (slow) with the updated JAR:

When i test the old connection, I can see that the new driver is used (Redshift JDBC Driver 2.1.0.32):

Any ideas?

[E1izabeth]

I see. Thanks

[raphaelauv]

I have exactly the same problem on external tables ( spectrum ) with a AWS / Redshift connection ( not a postgresql )

using CE Version 24.3.1.202412221611 with redshift-jdbc42-2.1.0.32 on ubuntu 24.04

[kzito-fn]

Similar problem here as well, not specific to including spectrum tables, just happening with complex or slower queries. I can see them complete in the "Queries and loads" view in AWS Console after a few minutes, then DBeaver remains stuck in "Read data from container" for any number of minutes after, or indefinitely. Tried various versions of DBeaver and the driver with no improvement.

[nilsoncunha]

@klingsoft I rolled back DBeaver to version 23.3.5, as you mentioned, and it worked. I'm not experiencing any slowdown when retrieving data from Redshift Spectrum.

The driver version I'm using is 2.1.0.29.

Thank you very much!

[Akmanar]

@E1izabeth @ShadelessFox
We have finally had the opportunity to get our hands on a laptop not enrolled on our antivirus policies and could test our problematic queries. The results were the same as with the ones with the antivirus enabled, no change in behaviour. Hope this helps.

[Yigitttttttt]

I'm experiencing the same problem right now in Oracle 19c. I'm waiting too much Select * from * and it writes: Read data from container and sometimes it's takes 2 minutes. But, Toad always get in seconds.

[serge-rider]

The problem is caused by new Redshift driver version.

Original issue is here: aws/amazon-redshift-jdbc-driver#115

Workaround: switch to driver 2.1.0.8 or earlier.

[klingsoft]

@serge-rider: Thanks for the update.
If it is of any help to you, it works better on my side if I create a new connection in dBeaver as a Postgres connection, but still use the RedShift driver (Redshift JDBC Driver 2.1.0.32).
No unexpected delays as in the old RedShift connection using this driver.