proof of concept

[Alessandro100]

Summary:

NOTE: None of this is final code, just to give an idea

Demonstrates a proof of concept where

• Stops.txt is pre-process into a .pmtiles file
• The .pmtiles file is stored in google cloud storage
• The frontend reads the .pmtiles file from google cloud storage and displays 9000 stops flawlessly

Quick UI prototype of what the Feed detail page could look like

Expected behavior:

On the feed detail page, the map should appear with the STM stops displayed where you can hover over the stops and see the stop name.

Testing tips:

Go to feeds/mdb-1179 and check out the UI. Should be able to hover hove the stops and get the data. Also to get a quick idea of the UI for routes

Please make sure these boxes are checked before submitting your pull request - thanks!

• Run the unit tests with ./scripts/api-tests.sh to make sure you didn't break anything
• Add or update any needed documentation to the repo
• Format the title like "feat: [new feature short description]". Title must follow the Conventional Commit Specification(https://www.conventionalcommits.org/en/v1.0.0/).
• Linked all relevant issues
• Include screenshot(s) showing how this pull request works and fixes the issue(s)

Spike Findings

High Level (Map Visualization)

1. When validation report is being generated -> the map files will be generated (.pmtiles)
2. These map files will be stored in google cloud storage, with the URL stored in datasets
3. Frontend will load the latest dataset .pmtile onto the map where the user will be able to interact with the map

Generating .pmtiles

When the validation report is being created

• Stops.txt -> stops.geojson -> stops.pmtiles (file) -> upload into google cloud storage
  • Save the url into the datasets table
• Same thing for routes.txt
  Once the .pmtiles are in the datasets table, frontend will be able to access it and display it on the map

.pmtiles Script

Stops.txt -> stops.geojson
python gtfs_routes_to_geojson.py stops.txt output.geoson
TO NOTE: we can add any data we want to expose in properties
stops.geojson -> stops.pmtiles
tippecanoe -o stops.pmtiles --force --no-tile-size-limit -zg stops.geojson

Decisions

Pre-processing map file vs client managed map elements

• In stops.txt there can be many elements, adding these elements to the map without any optimizations will result in bad performance

Choosing .pmtiles as the map data format

• Pmtiles is compressed and optimized to be served from a storage bucket (GCP Storage). It handles callback events on actions (click, mouseover). Great for a map with has static data
• Raster tiles: Great performance, no interactivity. This is used to render the base map of OSM
• Vector Tile Map (mbtiles): Needs to run from a server ($) and is great for a map with dynamic elements
  More info on .pmtiles: https://docs.protomaps.com/pmtiles/

Each feed will have it’s own .pmtile file vs 1 large map file for all feeds

• Loading times would be much better per feed
• ~ STM stops.txt .pmtile = 285kb
• [discussion] should the url be on dataset or do we only care about the latest one (and don’t want to store the previous ones)

Important notes

• When access the .pmtile out of GCP the client needed the file to have CORS *

Displaying Route Data

• we'll need to access data from trip.txt, calendar.txt, and routes.txt
• During the validation report it, we should add the needed fields in our database (new table: FeedRoute)
• Depending on if this data is worth persisting, we can only keep the latest route data.
• For team discussion

Screenshots

Route Table (prototype)

Detailed map with stops (prototype)

[emmambd]

on the topic of historical data, I think ideal state would be the most recent dataset -> the second most recent, so it's possible for users to compare

[Alessandro100]

on the topic of historical data, I think ideal state would be the most recent dataset -> the second most recent, so it's possible for users to compare

Depending on the use case there are few ways this can be done UI

1. Toggle button showing only 1 at a time (Latest / Previous)
2. "Compare" button: which would show them side by side
3. A diff info section ex: "New" badge on routes which changed from the previous, highlighting the changes

[emmambd]

3 is the juicy/high value one :)

[github-actions]

Preview Firebase Hosting URL: https://mobility-feeds-dev--pr-909-70gjn23t.web.app

[github-actions]

Preview Firebase Hosting URL: https://mobility-feeds-dev--pr-909-uhoeo9vt.web.app

[github-actions]

Preview Firebase Hosting URL: https://mobility-feeds-dev--pr-909-r4cf34ep.web.app