-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
77 weighted points #82
base: main
Are you sure you want to change the base?
Changes from all commits
ea43364
0112bfe
5b0b80a
b4b5a4b
ab8c1f3
80ca8e8
4b53b33
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,3 @@ | ||
README_files/ | ||
input/north-yorkshire-latest.osm.pbf | ||
input/inputs.html |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,150 @@ | ||
|
||
|
||
The setup information is contained within the `setup.py` file, which | ||
generates minimal input files. | ||
|
||
``` bash | ||
python setup.py | ||
``` | ||
|
||
We’ll get a sample of 2 schools in York (York High School and Huntington | ||
School) using the `osmextract` package. | ||
|
||
``` r | ||
library(osmextract) | ||
``` | ||
|
||
Data (c) OpenStreetMap contributors, ODbL 1.0. https://www.openstreetmap.org/copyright. | ||
Check the package website, https://docs.ropensci.org/osmextract/, for more details. | ||
|
||
``` r | ||
q = "SELECT * FROM multipolygons WHERE amenity='school'" | ||
schools_york = osmextract::oe_get("York", query = q, extra_tags = "amenity") | ||
``` | ||
|
||
No exact match found for place = York and provider = geofabrik. Best match is Corse. | ||
Checking the other providers. | ||
|
||
No exact match found in any OSM provider data. Searching for the location online. | ||
|
||
The input place was matched with North Yorkshire. | ||
|
||
The chosen file was already detected in the download directory. Skip downloading. | ||
|
||
The corresponding gpkg file was already detected. Skip vectortranslate operations. | ||
|
||
Reading query `SELECT * FROM multipolygons WHERE amenity='school'' | ||
from data source `/home/robin/data/osm/geofabrik_north-yorkshire-latest.gpkg' | ||
using driver `GPKG' | ||
Simple feature collection with 603 features and 25 fields | ||
Geometry type: MULTIPOLYGON | ||
Dimension: XY | ||
Bounding box: xmin: -2.546044 ymin: 53.6425 xmax: -0.2912398 ymax: 54.61681 | ||
Geodetic CRS: WGS 84 | ||
|
||
``` r | ||
# schools_york$name | ||
destinations = dplyr::filter( | ||
schools_york, | ||
name %in% c("York High School", "Huntington School") | ||
) |> | ||
dplyr::select(name, everything()) | ||
destinations$name | ||
``` | ||
|
||
[1] "York High School" "Huntington School" | ||
|
||
``` r | ||
# Remove columns that only contain NA: | ||
destinations = destinations[, colSums(is.na(destinations)) < nrow(destinations)] | ||
destinations = sf::st_centroid(destinations) | ||
``` | ||
|
||
Warning: st_centroid assumes attributes are constant over geometries | ||
|
||
``` r | ||
sf::write_sf(destinations, "input/destinations.geojson", delete_dsn = TRUE) | ||
``` | ||
|
||
We’ll also create a sample of subpoints in York, taking 3 random points | ||
from each zone. | ||
|
||
``` r | ||
zones = sf::st_read("input/zones.geojson") | ||
``` | ||
|
||
Reading layer `zones' from data source | ||
`/home/robin/github/Urban-Analytics-Technology-Platform/od2net/examples/york_minimal/input/zones.geojson' | ||
using driver `GeoJSON' | ||
Simple feature collection with 3 features and 1 field | ||
Geometry type: POLYGON | ||
Dimension: XY | ||
Bounding box: xmin: -1.146752 ymin: 53.92474 xmax: -1.025942 ymax: 54.01074 | ||
Geodetic CRS: WGS 84 | ||
|
||
``` r | ||
set.seed(123) | ||
subpoints = sf::st_sample(zones, size = rep(3, nrow(zones))) |> | ||
sf::st_sf() | ||
# Let's add provide the subpoints with values representing their importance: | ||
subpoints$size = runif(nrow(subpoints), 1, 10) |> | ||
round(1) | ||
sf::write_sf(subpoints, "input/subpoints.geojson", delete_dsn = TRUE) | ||
``` | ||
|
||
We can visualise these as follows: | ||
|
||
``` python | ||
import geopandas as gpd | ||
import pandas as pd | ||
zones = gpd.read_file("input/zones.geojson") | ||
destinations = gpd.read_file("input/destinations.geojson") | ||
subpoints = gpd.read_file("input/subpoints.geojson") | ||
od = pd.read_csv("input/od.csv") | ||
ax = zones.plot() | ||
destinations.plot(ax=ax, color='red') | ||
subpoints.plot(ax=ax, color='blue', markersize=subpoints['size'] * 3) | ||
ax.set_title("Origins and Destinations") | ||
``` | ||
|
||
![](README_files/figure-commonmark/origins_destinations_plot-1.png) | ||
|
||
Let’s visualise the flows between the origins and destinations: | ||
|
||
``` r | ||
library(ggplot2) | ||
od = readr::read_csv("input/od.csv") | ||
``` | ||
|
||
Rows: 6 Columns: 3 | ||
── Column specification ──────────────────────────────────────────────────────── | ||
Delimiter: "," | ||
chr (2): from, to | ||
dbl (1): count | ||
|
||
ℹ Use `spec()` to retrieve the full column specification for this data. | ||
ℹ Specify the column types or set `show_col_types = FALSE` to quiet this message. | ||
|
||
``` r | ||
od_geo = od::od_to_sf(od, zones, destinations) | ||
``` | ||
|
||
0 origins with no match in zone ids | ||
0 destinations with no match in zone ids | ||
points not in od data removed. | ||
|
||
``` r | ||
ggplot() + | ||
geom_sf(data = zones, fill = "grey") + | ||
geom_sf(data = subpoints, aes(size = size), color = "blue") + | ||
geom_sf(data = destinations, color = "red") + | ||
geom_sf(data = od_geo, aes(size = count), color = "black") | ||
``` | ||
|
||
![](README_files/figure-commonmark/flows_plot-3.png) | ||
|
||
We can then run the od2net command as follows: | ||
|
||
``` bash | ||
docker run -v $(pwd):/app ghcr.io/urban-analytics-technology-platform/od2net:main /app/config.json | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
--- | ||
format: gfm | ||
--- | ||
|
||
The setup information is contained within the `setup.py` file, which generates minimal input files. | ||
|
||
```{bash} | ||
#| eval: false | ||
python setup.py | ||
``` | ||
|
||
We'll get a sample of 2 schools in York (York High School and Huntington School) using the `osmextract` package. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm fine showing examples like this here, but FWIW, something like Overpass can be quicker/more interactive. Ultimately a user guide could point to a variety of methods for generating inputs like this. https://overpass-turbo.eu/s/1RL9 as an example There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yes Overpass is fine for this also. Happy for this to be tweaked, just using what I know best. I don't think it matters here but happy to use another tool to get the schools if there are ways of doing that with reproducible and readable code. Doesn't seem like a biggy or blocker at this stage but let me know if you'd like to see changes to this bit. |
||
|
||
```{r} | ||
library(osmextract) | ||
q = "SELECT * FROM multipolygons WHERE amenity='school'" | ||
schools_york = osmextract::oe_get("York", query = q, extra_tags = "amenity") | ||
# schools_york$name | ||
destinations = dplyr::filter( | ||
schools_york, | ||
name %in% c("York High School", "Huntington School") | ||
) |> | ||
dplyr::select(name, everything()) | ||
destinations$name | ||
# Remove columns that only contain NA: | ||
destinations = destinations[, colSums(is.na(destinations)) < nrow(destinations)] | ||
destinations = sf::st_centroid(destinations) | ||
sf::write_sf(destinations, "input/destinations.geojson", delete_dsn = TRUE) | ||
``` | ||
|
||
We'll also create a sample of subpoints in York, taking 3 random points from each zone. | ||
|
||
```{r} | ||
zones = sf::st_read("input/zones.geojson") | ||
set.seed(123) | ||
subpoints = sf::st_sample(zones, size = rep(3, nrow(zones))) |> | ||
sf::st_sf() | ||
# Let's add provide the subpoints with values representing their importance: | ||
subpoints$size = runif(nrow(subpoints), 1, 10) |> | ||
round(1) | ||
sf::write_sf(subpoints, "input/subpoints.geojson", delete_dsn = TRUE) | ||
``` | ||
|
||
We can visualise these as follows: | ||
|
||
```{python} | ||
#| label: origins_destinations_plot | ||
import geopandas as gpd | ||
import pandas as pd | ||
zones = gpd.read_file("input/zones.geojson") | ||
destinations = gpd.read_file("input/destinations.geojson") | ||
subpoints = gpd.read_file("input/subpoints.geojson") | ||
od = pd.read_csv("input/od.csv") | ||
ax = zones.plot() | ||
destinations.plot(ax=ax, color='red') | ||
subpoints.plot(ax=ax, color='blue', markersize=subpoints['size'] * 3) | ||
ax.set_title("Origins and Destinations") | ||
``` | ||
|
||
Let's visualise the flows between the origins and destinations: | ||
|
||
```{r} | ||
#| label: flows_plot | ||
library(ggplot2) | ||
od = readr::read_csv("input/od.csv") | ||
od_geo = od::od_to_sf(od, zones, destinations) | ||
ggplot() + | ||
geom_sf(data = zones, fill = "grey") + | ||
geom_sf(data = subpoints, aes(size = size), color = "blue") + | ||
geom_sf(data = destinations, color = "red") + | ||
geom_sf(data = od_geo, aes(size = count), color = "black") | ||
``` | ||
|
||
```{r} | ||
#| eval: false | ||
#| echo: false | ||
library(tmap) | ||
tmap_mode("view") | ||
m = qtm(zones) + | ||
tm_shape(subpoints) + | ||
tm_dots(size = "size", col = "blue") + | ||
tm_shape(destinations) + | ||
tm_dots(fill = "red", size = 5) + | ||
tm_shape(od_geo) + | ||
tm_lines(lwd = "count", col = "black", scale = 9) | ||
tmap_save(m, "input/inputs.html") | ||
browseURL("input/inputs.html") | ||
``` | ||
|
||
We can then run the od2net command as follows: | ||
|
||
|
||
```{bash} | ||
#| eval: false | ||
docker run -v $(pwd):/app ghcr.io/urban-analytics-technology-platform/od2net:main /app/config.json | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
{ | ||
"requests": { | ||
"description": "Manually drawn zones and flows to schools, demonstrating weighted subpoints", | ||
"pattern": { | ||
"ZoneToPoint": { | ||
"zones_path": "zones.geojson", | ||
"destinations_path": "destinations.geojson", | ||
"csv_path": "od.csv", | ||
"origin_zone_centroid_fallback": false | ||
} | ||
}, | ||
"origins_path": "subpoints.geojson", | ||
"destinations_path": "destinations.geojson" | ||
}, | ||
"uptake": "Identity", | ||
"cost": { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It'd be simpler to not use a custom external command for lts and cost in this example. That's causing the Docker issue, and other examples demonstrate how to do this already. How about following the Liverpool example for cost and lts? Do you want something like quiet routes for this, just distance / direct roues, or something else? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Yeah, simple is good, clearly just copy-pasted from previous York example. Can update that, good suggestion. |
||
"ExternalCommand": "python3 cost.py" | ||
}, | ||
"lts": { | ||
"ExternalCommand": "python3 lts.py" | ||
} | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
import json | ||
import sys | ||
|
||
# Output a numeric edge cost | ||
def calculate(edge): | ||
tags = edge["osm_tags"] | ||
length_meters = edge["length_meters"] | ||
lts = edge["lts"] | ||
nearby_amenities = edge["nearby_amenities"] | ||
slope = edge["slope"] | ||
|
||
# Return None to not use the edge at all | ||
|
||
if tags["highway"] == "residential": | ||
return [round(length_meters), round(length_meters)] | ||
else: | ||
# Strongly avoid non-residential roads | ||
return [round(10 * length_meters), round(10 * length_meters)] | ||
|
||
|
||
# Read an array of JSON dictionaries from STDIN | ||
input_batch = json.loads(sys.stdin.read()) | ||
# Calculate an edge cost for each one | ||
results = list(map(calculate, input_batch)) | ||
# Write a JSON array of the resulting numbers | ||
print(json.dumps(results)) |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
{ | ||
"type": "FeatureCollection", | ||
"name": "destinations", | ||
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } }, | ||
"features": [ | ||
{ "type": "Feature", "properties": { "name": "York High School", "osm_way_id": "29110900", "amenity": "school", "other_tags": "\"addr:city\"=>\"York\",\"addr:country\"=>\"GB\",\"addr:postcode\"=>\"YO24 3WZ\",\"addr:street\"=>\"Cornlands Road\",\"addr:suburb\"=>\"Acomb\",\"capacity\"=>\"991\",\"email\"=>\"[email protected]\",\"isced:level\"=>\"2\",\"max_age\"=>\"16\",\"min_age\"=>\"11\",\"phone\"=>\"+44 1904 555500\",\"ref:GB:uprn\"=>\"100052163540\",\"ref:edubase\"=>\"144652\",\"ref:edubase:group\"=>\"16073\",\"school:trust\"=>\"yes\",\"school:trust:name\"=>\"South Bank Multi Academy Trust\",\"school:trust:type\"=>\"multi_academy\",\"school:type\"=>\"academy\",\"website\"=>\"https://www.yorkhighschool.co.uk/\",\"wikidata\"=>\"Q8055461\",\"wikipedia\"=>\"en:York High School, York\"" }, "geometry": { "type": "Point", "coordinates": [ -1.128900930965928, 53.947613356147485 ] } }, | ||
{ "type": "Feature", "properties": { "name": "Huntington School", "osm_way_id": "122135723", "amenity": "school", "other_tags": "\"addr:country\"=>\"GB\",\"addr:postcode\"=>\"YO32 9WT\",\"addr:street\"=>\"Huntington Road\",\"capacity\"=>\"1545\",\"email\"=>\"[email protected]\",\"isced:level\"=>\"2;3\",\"max_age\"=>\"18\",\"min_age\"=>\"11\",\"phone\"=>\"+44 1904 752100\",\"ref:GB:uprn\"=>\"100052170371\",\"ref:edubase\"=>\"121673\",\"school:trust\"=>\"no\",\"school:type\"=>\"community\",\"website\"=>\"https://huntingtonschool.co.uk/\"" }, "geometry": { "type": "Point", "coordinates": [ -1.063642017028779, 53.990093958328686 ] } } | ||
] | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
from,to,count | ||
south,York High School,500 | ||
center,York High School,100 | ||
north,York High School,200 | ||
south,Huntington School,800 | ||
center,Huntington School,300 | ||
north,Huntington School,600 |
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No objection to origins.geojson, other than slight name clash with 'zones of origin': you can have zones in which flows originate and origins that specificy the exact points they must flow from. But as long as it's documented could well be the least bad options, I have no better suggestions, and happy with origins.geojson if you are. Subpoints comes from the jittering paper and it seems we're using the word in a similar way, what's the objection to subpoints out of interest? 👍 to clarify terms upfront and happy to adapt. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
{ | ||
"type": "FeatureCollection", | ||
"name": "subpoints", | ||
"crs": { "type": "name", "properties": { "name": "urn:ogc:def:crs:OGC:1.3:CRS84" } }, | ||
"features": [ | ||
{ "type": "Feature", "properties": { "size": 7.2 }, "geometry": { "type": "Point", "coordinates": [ -1.084948428827924, 53.959860019753116 ] } }, | ||
{ "type": "Feature", "properties": { "size": 8.2 }, "geometry": { "type": "Point", "coordinates": [ -1.065675923641903, 53.968255508127534 ] } }, | ||
{ "type": "Feature", "properties": { "size": 1.2 }, "geometry": { "type": "Point", "coordinates": [ -1.08027588725639, 53.960397590493173 ] } }, | ||
{ "type": "Feature", "properties": { "size": 5.3 }, "geometry": { "type": "Point", "coordinates": [ -1.048145775760174, 53.988466436780065 ] } }, | ||
{ "type": "Feature", "properties": { "size": 7.8 }, "geometry": { "type": "Point", "coordinates": [ -1.055372173407674, 54.009228087733909 ] } }, | ||
{ "type": "Feature", "properties": { "size": 2.9 }, "geometry": { "type": "Point", "coordinates": [ -1.077859414261401, 53.998822595720604 ] } }, | ||
{ "type": "Feature", "properties": { "size": 3.9 }, "geometry": { "type": "Point", "coordinates": [ -1.11062851182091, 53.93422678953317 ] } }, | ||
{ "type": "Feature", "properties": { "size": 3.1 }, "geometry": { "type": "Point", "coordinates": [ -1.107718347978171, 53.929568710302227 ] } }, | ||
{ "type": "Feature", "properties": { "size": 2.3 }, "geometry": { "type": "Point", "coordinates": [ -1.116778858632627, 53.956331735484653 ] } } | ||
] | ||
} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
{"type":"FeatureCollection","features":[{"type":"Feature","properties":{"name":"center"},"geometry":{"coordinates":[[[-1.08285,53.970735],[-1.096017,53.958917],[-1.075591,53.947693],[-1.057528,53.967309],[-1.08285,53.970735]]],"type":"Polygon"}},{"type":"Feature","properties":{"name":"north"},"geometry":{"coordinates":[[[-1.094806,53.998733],[-1.068685,53.977605],[-1.025942,53.9965],[-1.056812,54.010736],[-1.094806,53.998733]]],"type":"Polygon"}},{"type":"Feature","properties":{"name":"south"},"geometry":{"coordinates":[[[-1.146752,53.957545],[-1.139312,53.924745],[-1.091661,53.9281],[-1.1067,53.956427],[-1.146752,53.957545]]],"type":"Polygon"}}]} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,17 @@ | ||
import json | ||
import sys | ||
|
||
# Output 0 (not allowed), 1 (suitable for children), 2 (low stress), 3 (low stress), or 4 (high stress) | ||
def calculate(tags): | ||
if tags["highway"] == "residential": | ||
return 2 | ||
else: | ||
return 4 | ||
|
||
|
||
# Read an array of JSON dictionaries from STDIN | ||
tags_batch = json.loads(sys.stdin.read()) | ||
# Calculate LTS for each one | ||
lts_results = list(map(calculate, tags_batch)) | ||
# Write a JSON array of the resulting numbers | ||
print(json.dumps(lts_results)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm hesitant to check both
qmd
andmd
into git, because the second is derived from the first, and it contains lots of log-spam. Looking at https://quarto.org/docs/publishing/github-pages.html, checking in both is option 1, right? Publishing to GH Pages is more work upfront, but feels like the better option. I can set that up after we get this first PR merged?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, happy to remove the .md. Glad I didn't check in the rest of the stuff that quarto outputs based on this comment!