Errors running makefile

[stepps00]

I'm trying to setup a PIP service, via the cookbook documentation, but running into issues with this repo. I'm unable to build wof-pgis-index needed to index WOF records.

Running make or make bin, I see the following errors:

if test -d pkg; then rm -rf pkg; fi
if test -d src/github.com/whosonfirst/go-whosonfirst-pgis; then rm -rf src/github.com/whosonfirst/go-whosonfirst-pgis; fi
mkdir -p src/github.com/whosonfirst/go-whosonfirst-pgis
cp -r client src/github.com/whosonfirst/go-whosonfirst-pgis/client
cp -r vendor/* src/
# github.com/whosonfirst/go-whosonfirst-placetypes
src/github.com/whosonfirst/go-whosonfirst-placetypes/specification.go:13:6: WOFPlacetypeSpecification redeclared in this block
	previous declaration at src/github.com/whosonfirst/go-whosonfirst-placetypes/placetypes.go:27:6
# github.com/whosonfirst/go-whosonfirst-uri
src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:50:3: unknown field 'IsAlternate' in struct literal of type URIArgs
src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:57:16: undefined: AltGeom
src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:71:7: args.AltGeom undefined (type *URIArgs has no field or method AltGeom)
src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:72:7: args.IsAlternate undefined (type *URIArgs has no field or method IsAlternate)
src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:105:20: uri_args.IsAlternate undefined (type *URIArgs has no field or method IsAlternate)
src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:109:37: undefined: AltGeom
src/github.com/whosonfirst/go-whosonfirst-uri/uri.go:162:6: IsWOFFile redeclared in this block
	previous declaration at src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:80:36
src/github.com/whosonfirst/go-whosonfirst-uri/uri.go:183:6: IsAltFile redeclared in this block
	previous declaration at src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:97:36
src/github.com/whosonfirst/go-whosonfirst-uri/uri.go:204:6: IdFromPath redeclared in this block
	previous declaration at src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:124:38
src/github.com/whosonfirst/go-whosonfirst-uri/uri.go:245:6: RepoFromPath redeclared in this block
	previous declaration at src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:130:41
src/github.com/whosonfirst/go-whosonfirst-uri/parse.go:105:20: too many errors
# github.com/lib/pq
src/github.com/lib/pq/connector.go:27:2: not enough arguments to return
src/github.com/lib/pq/connector.go:27:10: c.open undefined (type *Connector has no field or method open)
src/github.com/lib/pq/notice.go:14:18: c.(*conn).noticeHandler undefined (type *conn has no field or method noticeHandler)
src/github.com/lib/pq/notice.go:25:11: c.(*conn).noticeHandler undefined (type *conn has no field or method noticeHandler)
src/github.com/lib/pq/rows.go:46:9: undefined: oid.TypeName
src/github.com/lib/pq/rows.go:74:26: rs.colTyps[index].Type undefined (type oid.Oid has no field or method Type)
src/github.com/lib/pq/rows.go:79:26: rs.colTyps[index].Name undefined (type oid.Oid has no field or method Name)
src/github.com/lib/pq/rows.go:86:2: not enough arguments to return
src/github.com/lib/pq/rows.go:86:26: rs.colTyps[index].Length undefined (type oid.Oid has no field or method Length)
src/github.com/lib/pq/rows.go:92:26: rs.colTyps[index].PrecisionScale undefined (type oid.Oid has no field or method PrecisionScale)
src/github.com/lib/pq/rows.go:92:26: too many errors
make: *** [bin] Error 2

[thisisaaronland]

Hrm, it looks like the schema needs to be updated. I will try to investigate shortly.

[straup]

@stepps00 It is not a schema thing, I just read the issue quickly in between a number of other things.

The problem is caused by the fact that this package has not been updated to account for Go modules (which is a better way, overall, to handle dependencies). Can you please try the following:

• Start from a fresh clone of the repo
• Checkout the gomod branch
• Run make cli-tools (which is the new make bin)
• Try running the tool(s) that you've used in the past

Assuming that works I'll merged to master and publish a new release.

[stepps00]

From a fresh clone of the repo, I ran the commands:

$ git checkout gomod
    Branch 'gomod' set up to track remote branch 'gomod' from 'origin'.
    Switched to a new branch 'gomod'
$ make cli-tools
    go build -mod vendor -o bin/wof-pgis-connect cmd/wof-pgis-connect/main.go
    go build -mod vendor -o bin/wof-pgis-dump cmd/wof-pgis-dump/main.go
    go build -mod vendor -o bin/wof-pgis-index cmd/wof-pgis-index/main.go
    go build -mod vendor -o bin/wof-pgis-prune cmd/wof-pgis-prune/main.go

This seemed to install without issue, but when I run the command to index a single repository I get the following error:

$ ./bin/wof-pgis-index -verbose -mode directory /path/to/repo/whosonfirst-data-admin-ro
    18:31:53.849023 [wof-pgis-index] FATAL Failed to create new indexer because Unknown driver

[thisisaaronland]

Ah, yes. More (past) changes to the go-whosonfirst-index interfaces. I will investigate shortyl.

[thisisaaronland]

Try it now? (Don't forget to pull from gomod)

BTW, you should specify -mode repo.

[stepps00]

$ git clone https://github.com/whosonfirst/go-whosonfirst-pgis.git
$ cd go-whosonfirst-pgis
$ git checkout gomod
    Branch 'gomod' set up to track remote branch 'gomod' from 'origin'.
    Switched to a new branch 'gomod'
$ make cli-tools
    go build -mod vendor -o bin/wof-pgis-connect cmd/wof-pgis-connect/main.go
    go build -mod vendor -o bin/wof-pgis-dump cmd/wof-pgis-dump/main.go
    go build -mod vendor -o bin/wof-pgis-index cmd/wof-pgis-index/main.go
    go build -mod vendor -o bin/wof-pgis-prune cmd/wof-pgis-prune/main.go

This seems to work as expected, I no longer see the same errors. The next step would be to index the repo I'm interested in working with, as follows:

$ ./bin/wof-pgis-index -verbose -mode repo my/path/to/whosonfirst-data-admin-ro

This also seems to work, I don't see any issues. However, when I try to use the wof-hierarchy-rebuild tool, I now see postgis errors:

$ wof-hierarchy-rebuild -C postgis my/path/to/whosonfirst-data-admin-ro/data/101/751/953/101751953.geojson -U -v
    INFO:root:rebuild hierarchy for my/path/to/whosonfirst-data-admin-ro/data/101/751/953/101751953.geojson
    DEBUG:root:[hierarchy][101751953][Slatina] rebuild feature
    DEBUG:root:[hierarchy][101751953][Slatina] controlled is []
    DEBUG:root:wof:parent_id NOT IN in controlled

    ...

    DEBUG:root:[spatial][postgis][pip] SELECT id, parent_id, placetype_id, meta, ST_AsGeoJSON(geom), ST_AsGeoJSON(centroid) FROM whosonfirst WHERE ST_Intersects(geom, ST_GeomFromGeoJSON(%s)) AND placetype_id=%s AND is_superseded=%s AND is_deprecated=%s with args: ['{"type": "Point", "coordinates": [24.36667, 44.43333]}', '102312307', 0, 0]
    DEBUG:root:[spatial][postgis][pip] time to execute query (point-in-polygon for 44.43333,24.36667) : 0.03873920440673828
    DEBUG:root:[spatial][postgis][inflate_row] {'filters': {'wof:placetype_id': '102312307', 'wof:is_superseded': 0, 'wof:is_deprecated': 0, 'wof:is_ceased': 0}, 'as_feature': True}
    ERROR:root:False
    ERROR:root:[spatial][postgis][inflate_row] BANDAID is wof:is_ceased == 0: True ('0')
    ERROR:root:[spatial][postgis][inflate_row] failed to convert row to feature
    DEBUG:root:[spatial][postgis][pip] failed to inflate row
    DEBUG:root:ensure hierarchy for 101751953 with placetype country : 0 possible
    DEBUG:root:[hierarchy][101751953][Slatina] append 0 possible hierarchies
    DEBUG:root:[hierarchy][101751953][Slatina] no append but ensure hierarchy - matches: False
    DEBUG:root:[hierarchy][101751953][Slatina] ensure common ancestors (is a locality) : region;continent;country
    INFO:root:nothing has changed when rebuilding the hierarchy for 101751953
    INFO:root:Slatina (101751953) has no changes (parent ID is still -1)
    DEBUG:root:[{'locality_id': 101751953}]
    DEBUG:root:writing /my/path/to/whosonfirst-data-admin-ro/data/101/751/953/101751953.geojson

With the hierarchy now only containing the feature itself:

    "wof:hierarchy":[
        {
            "locality_id":101751953
        }
    ],

I've passed both -mode repo and -mode directory, both give the same result. So this is probably an issue with the way I'm indexing the repo in postgres on my machine, I need to poke around.

Either way, I'm able to now run the makefile, so I think issue is fixed - thank you @thisisaaronland!