Skip to content

0.2.0
Compare
Choose a tag to compare
@rustprooflabs rustprooflabs released this 26 May 23:01
· 565 commits to main since this release
0.2.0
03238bc
This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode.

PgOSM Flex 0.2.0 Release Notes

This release includes data quality improvements along with a major overhaul to the in-Docker process. The improvements in Docker are tied to recent improvements to osm2pgsql.

A big part of the improvements in this release are related to the underlying improvements in osm2pgsql made in the master branch after the osm2pgsql v1.4.2 release. Those changes include a major overhaul of the middle processing, making the RAM requirements a) significantly lower, and b) predictable. See the blog post OpenStreetMap to PostGIS is getting lighter for more. This key improvement cascaded into a new open source API (https://osm2pgsql-tuner.com/) to provide recommended osm2pgsql commands for a given amount of system RAM and the size of the PBF being processed. This allowed the script in Docker to be reworked to determine the combination of osm2pgsql switches to use for processing (--slim, --drop, --flat-nodes, --cache) instead of always using --slim --drop --cache=N>0.

Multiple regions were tested and found to run 10-40% faster than the previous script. Previously the recommendation for the Docker image was to avoid for 2+ GB PBF files, now the Docker image makes it easy and fast for all PBF sizes tested so far.

⚠️ Breaking change: Docker Exec - Different 3rd Param

The Docker exec command had a change in the 3rd parameter. Previously the 3rd parameter set the cache value for osm2pgsql (in MB), it now expects the system RAM (in GB!) to use to determine the osm2pgsql command to run. This example command sets the system RAM to 8 GB.

docker exec -it \
    -e POSTGRES_PASSWORD=mysecretpassword -e POSTGRES_USER=postgres \
    pgosm bash docker/run_pgosm_flex.sh \
    north-america/us \
    district-of-columbia \
    8 \
    run-all

⚠️ Breaking change: JSON handling in osm2pgsql

The osm2pgsql project built in automatic JSON handling when table columns are declared as JSON/JSONB. This required remove
the Lua JSON script that was previously added.

Bump required osm2pgsql version

The two breaking changes listed mean this version of PgOSM Flex requires a currently non-tagged release of osm2pgsql. In the future when the osm2pgsql version following 1.4.2 is released, this project will pin to that for the minimum version. With the improvements described above, the new recommended way to use PgOSM Flex is to use the Docker image to handle the processing. Using Docker makes the version of osm2pgsql installed a non-issue.

If you are using osm2pgsql installed on a server, you must install the latest master branch of osm2pgsql. The instructions in MANUAL-STEPS-RUN.md cover this install.

Data related improvements

  • Infrastructure improvements - Add line, polygon types, plus generic utility keys - #133
  • Add road_polygon to handle elevators #127
  • Improve wheelchair from simple boolean to match OSM defined values #117 / #125
  • Add door/entrance to Layers Building and Indoor (#108 / #121)
  • Fixed Bench processing bug - #119
  • Additional amenity keys included, code improvements - #110, #114, #120
  • Add relations to building layer #109 / #111

Backend related

  • Docker uses dynamic osm2pgsql command via osm2pgsql-tuner.com API (BREAKING CHANGE) - #129
  • Swapped JSON to osm2pgsql built-in support (BREAKING CHANGE) #122
  • Testing / QC - #112 / #116
Assets 2
Loading