Skip to content

Conversation

@rustprooflabs
Copy link
Owner

@rustprooflabs rustprooflabs commented Feb 4, 2024

This PR started by testing osm2pgsql-dev/osm2pgsql#2134. Those results were encouraging and is now this is a PR addressing a couple other minor improvements.

@rustprooflabs rustprooflabs added the external External project or constraint. Blocked, related or otherwise label Feb 4, 2024
@rustprooflabs rustprooflabs added this to the 1.0.0 milestone Feb 4, 2024
@rustprooflabs
Copy link
Owner Author

Connections before -- 41 idle!

image

Connections after -- 3 idle.

image

@rustprooflabs
Copy link
Owner Author

Remove extra -d from osm2pgsql-replication command after more strict option checking implemented osm2pgsql-dev/osm2pgsql#2115

Sets repo/branch back to production setting. Leaving the new repo URL.

@rustprooflabs rustprooflabs added Technical Debt Not broken, not new, but not right either! Docker Docker image and/or related script(s) labels Feb 10, 2024
@rustprooflabs
Copy link
Owner Author

Adding a few more notes about connection volume with replication. The initial load had a connection pattern similar to a single load w/out replication (sustained 41 idle connections). Where things could get really nasty connection-wise was the replication update steps.

Before the patch the default PgOSM Flex layerset would consume 207 connections for replication updates. It's a relatively short time, but compared to Postgres' default max_connections=100 that can be shocking.

image

After the change to osm2pgsql, the total connection load drops to 17 (-92% diff)

image

@rustprooflabs rustprooflabs merged commit 0afd698 into main Feb 10, 2024
@rustprooflabs rustprooflabs deleted the reduced-conns-test branch February 10, 2024 16:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Docker Docker image and/or related script(s) external External project or constraint. Blocked, related or otherwise Technical Debt Not broken, not new, but not right either!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants