Merge remote-tracking branch 'upstream/master'

Author: didi

README.md

@@ -38,7 +38,7 @@ View the help message output:
 	  --report              output report to stdout (default)
 	  --ids-only            output all way and relation IDs from given diff
 	  --ql-only             output Overpass QL
-	  --rss-file RSS_FILE   output report as rss file
+	  --rss-file RSS_FILE   output / append report as rss file
 
 `regional-diff.py` by default reports ways and relations with bicylce-tags[1] in
 Vorarlberg which were modified in the last minute.
@@ -80,11 +80,28 @@ created on `#Wed Jun 25 20:02:07 UTC 2014`. This should be the sample output:
 	  http://www.openstreetmap.org/relation/116941/history
 	  http://www.openstreetmap.org/changeset/23158682	
 
+RSS-Feed
+--------
+
 To generate an RSS-file run:
 
 	$ scripts/regional-diff.py -f scripts/637.osc.gz --rss-file regional-diff.rss
 
-Put the RSS-file to your `htdocs` folder and point your favourite RSS-Client to that URL to subscribe that feed.
+Put the RSS-file to your `htdocs` folder and point your favourite RSS-Client to
+that URL to subscribe that feed.
+
+The RSS-File with an RSS-Item is only created if modification exist (in
+Vorarlberg). An RSS-Item (of a modification) is created and appended on every
+run of `regional-diff.py` (even if the modification is part of a previous item).
+
+Note
+----
+
+Please note that processing a daily-diff file is a very CPU and memory hungry
+task. Working with less than 16GB of RAM could cause the script to crash.
+
+Used libraries
+--------------
 
-Please be aware that the RSS-File will be overwriten on every run of `regional-diff.py`
+* *PyRSS2Gen* (version 1.1) taken from http://www.dalkescientific.com/Python/PyRSS2Gen.html
 

doc/notes.txt

@@ -179,4 +179,32 @@ Show only ways (from a given list of ways) which are part of a bicycle route
   way for our task is to select only the parent ways for all nodes from the input
   set with *way(bn*);
 
-
+Thu 10 Jul 16:04:02 CEST 2014 (and before) -MM
+==============================================
+Problems fixed:
+• Ways without node-changes have no spatial information in diff file, so get removed by crop
+    · pick a node for each way missing spatial information
+    · download them via overpass in bulk
+        · there was need to limit the download to only 1000 nodes at once, and merge the results later
+    · merge them into osm-file before crop
+
+• filter out non-highway ways with osmosis instead of overpass → reduces load on overpass server
+    · allow only highway types with --tag-filter that are allowed for bicyclists
+
+• added handling of deleted ways: osmosis removes deleted items when converting osc→osm
+    ·  regex <deleted> → <modified> in osc after simplify change → fixed it.
+
+Testcases:
+• ./regional-diff.py -f minutely-796.osc.gz 
+    this includes changeset http://www.openstreetmap.org/api/0.6/changeset/24005065 from user species
+  there should be 89 ways and 2 relations (all from this changeset)
+• ./regional-diff.py -f scripts/637.osc.gz
+  there should be 5 ways and 2 relations
+
+Notes for the future :
+
+• overpass API does NOT return deleted nodes.
+  · for ways without spatial information, some nodes (if deleted meanwhile on the server) will not come back on overpass quere → way gets removed (wrongly) on crop
+  But: this should happen only (rareley) on fresh changesets.
+
+• if a relation modified is only sorted or tag-changed without having modified members in the changesets, this is not reported.