Features client #1073

tbarsballe · 2024-10-29T23:24:30Z

Includes: list_collections, create_collection, list_features, create_features

This came out of an internal need for a project I was working on incorporating the SDK, and represents about the extent of time I have to contribute to Features API support at this time. If someone else can pick up from where this started, that would be great.

I have programmatically and manually tested all of the the FeaturesClient methods in the context of the project described above, and everything seems to be working. In particular, the API has different pagination scheme, currently handled by overriding Paged.__next_link, that could perhaps be incorporated directly into Paged, since the as-of-yet-unimplemented Analytics API uses the same pagination scheme.

Related Issue(s):

Oddly enough, we do not seem to have an issue for Features API support yet.

Proposed Changes:

For inclusion in changelog (if applicable):

Add FeaturesClient to support the Features API

PR Checklist:

This PR is as small and focused as possible
If this PR includes proposed changes for inclusion in the changelog, the title of this PR summarizes those changes and is ready for inclusion in the Changelog.
I have updated docstrings for function changes and docs in the 'docs' folder for user interface / behavior changes
This PR does not break any examples or I have updated them

planet/clients/features.py

stephenhillier · 2024-12-18T22:56:26Z

I added some minor suggestions for the type errors. Just remembered your request for somebody picking this up so no need to follow up.

Looks like the biggest piece remaining is some test coverage. Maybe one of us can tackle that during one of the working session time blocks.

tbarsballe · 2024-12-19T16:47:46Z

Thanks for taking a look. Test coverage is definitely a big missing piece.

CLI integration would be the other major part that's lacking, since this realy only covers the SDK part right now.

There's also a number of other Features API endpoints to implement - so far this covers all the major ones, but not any of the less-used ones (e.g. "alternates" or "validate")

stephenhillier · 2025-01-21T00:23:39Z

I'm spending some time finishing this up, current progress:

added tests for all methods
- Trying a new thing where we have one test for each piece of functionality, which calls both sync and async methods (uses parameterize if we need multiple test cases). @ischneider
added sync client
added docstrings with examples to all methods (sync and async)

I'm also proposing to add a new class Feature, which is a subclass of dict. Feature has a .ref convenience method/property which makes it easier to grab the feature reference (required for using features with Data API, Subscriptions API etc.). With this we can allow users to pass Features directly to other methods e.g. pl.data.search(["PSScene"], geometry=feature). Subclassing dict makes it easier to extend this pattern to other object types in the future, since most client methods return dicts right now.

Another proposal is that add_features accepts geojson Features, Geometries as well as objects that implement __geo_interface__. This means users can import features directly from other spatial libraries.

stephenhillier · 2025-01-27T17:53:57Z

Added CLI commands with tests

tbarsballe · 2025-01-29T00:11:30Z

I did a little bit of testing of the CLI here (mostly because I happened to need to list some features today), and have a little bit of feedback on the user experience:

The command planet features feature-list [collection-id] seems overly verbose / redundant due to the "features feature" part (though it makes sense from an API structure perspective). Also since the base command (features) is plural, I found myself mistyping feature-list as features-list frequently (same for collection-list/collections-list).

Perhaps switch to just using the url slug to fix both issues? i.e. collection-list -> collections-list, feature-list -> items-list, etc.

The help message for the collection-list and collection-get commands are identical - this is likely to be confusing for users:

Commands:
collection-create  Create a new Features API collection.
collection-get     List Features API collections
collection-list    List Features API collections
feature-add        Add features from a geojson file to a Features API...
feature-list       List features in a Features API collection

This last one probably isn't worth solving right now, but extracting a collection id from the output of collection-list to use in feature-list is a little unwieldy, especially if you've got a lot of collections - this is basically the same problem we've got going from search results to orders, and the same solution of an --ids-only option, or maybe a -h (human-readable) option that also included names, would probably solve it.

stephenhillier · 2025-01-29T19:20:37Z

Thanks @tbarsballe!

I had the same issue as you- I regularly typed collections-list instead of of collection-list while working on this and testing. I was trying to follow the pattern that planet data search-list used, but perhaps we should just go with what feels most natural. Does collections-list sound good (and the same deal for the other actions)?

For your 3rd point, is the use case to list collections, then pick a collection and list those features? --pretty could really help there; would a --compact option help as well to just show title/description/id? Or would you prefer just formatted text output?

tbarsballe · 2025-01-29T19:24:22Z

Does collections-list sound good (and the same deal for the other actions)?

Yeah, collections-list sounds good to me (and maybe items-list instead of feature-list to avoid the double "features", and match the API endpoint)

For your 3rd point, is the use case to list collections, then pick a collection and list those features? --pretty could really help there; would a --compact option help as well to just show title/description/id? Or would you prefer just formatted text output?

Yes, that's exactly the use case. I think a --compact option like you describe would most cleanly solve it.

stephenhillier · 2025-01-29T22:21:24Z

Here are some options for the CLI:

For listing items, we could go with either of the following (both are equivalent):

planet features items-list  <collection_id>
planet features collection-items <collection_id>

For creating an item/feature, I think we would stick with item-add unless you have another idea:

planet features item-add <collection_id> feature.geojson

Alternately we could use another layer of sub-commands to represent collections and items/features:

planet features collections list
planet features collections create
planet features items list <collection_id>
planet features items add

Something like this is also possible, but may make it harder to present the CLI in terms of logical groups in the docs:

planet features collections list
planet features collections create
planet features list <collection_id>
planet features add <collection_id> feature.geojson

I'm not sure about an extra layer sub-commands myself, I personally would rather have one flat list of 6-7 commands in planet features --help than two extra sub-commands to figure out. But open to opinions.

tbarsballe · 2025-01-30T00:24:55Z

I like your first option. And yes, sticking item-add seems fine to me.

I had also considered another layer of commands, but it just seems like added complexity for not much gain.

stephenhillier · 2025-01-31T20:17:40Z

Added --compact mode to planet features collections-list and updated some of the CLI command names.

Output of --help showing available commands:

> planet features --help
Usage: planet features [OPTIONS] COMMAND [ARGS]...

  Commands for interacting with Features API

Options:
  -u, --base-url TEXT  Assign custom base Features API URL.
  --help               Show this message and exit.

Commands:
  collection-create  Create a new Features API collection.
  collection-get     Get a collection by ID
  collections-list   List Features API collections
  item-add           Add features from a geojson file to a collection
  item-get           Get a feature in a collection.
  items-list         List features in a Features API collection

stephenhillier · 2025-01-31T20:19:59Z

Also, I figure we should probably update the SDK methods to say get_item instead of get_feature for consistency... if everyone agrees I'll push another commit. Otherwise this should be it for the initial Features API SDK release.

tbarsballe · 2025-01-31T20:27:19Z

I tried out the compact mode - looks good, much more legible.
Changing get_feature to get_item seems sensible to me.

…es, create_features

planet/sync/data.py

planet/sync/features.py

stephenhillier · 2025-02-06T22:42:14Z

I split everything out into planet features collections and planet features items

I wasn't able to get the flat command list --help output like I hoped (click does have MultiCommand but they don't seem to fit well into in the middle of a chain of groups), but to be honest it seems like a relatively minimal issue and we can always adjust the help text later.

✦ ❯ planet features collections
Usage: planet features collections [OPTIONS] COMMAND [ARGS]...

  commands for interacting with Features API collections

Options:
  --help  Show this message and exit.

Commands:
  create  Create a new Features API collection.
  get     Get a collection by ID
  list    List Features API collections

✦ ❯ planet features items
Usage: planet features items [OPTIONS] COMMAND [ARGS]...

  commands for interacting with Features API items (features within a
  collection)

Options:
  --help  Show this message and exit.

Commands:
  add   Add features from a geojson file to a collection
  get   Get a feature in a collection.
  list  List features in a Features API collection

planet/sync/features.py

stephenhillier · 2025-02-12T18:09:09Z

@tbarsballe @asonnenschein @ischneider anyone able to take another look? I believe all comments should now be addressed but if I missed one let me know.

tbarsballe

Pulled it down for a test today and ran into a new issue - if you're using an expired API key, the features CLI commands will dump a large stacktrace on you:

(.venv) torben.barsballe@torben planet-client-python % planet features collections list
Traceback (most recent call last):
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/http.py", line 296, in _raise_for_status
    response.raise_for_status()
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/httpx/_models.py", line 829, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://api.planet.com/features/v1/ogc/my/collections'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/bin/planet", line 8, in <module>
    sys.exit(main())
             ~~~~^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/cli/cmds.py", line 78, in wrapper
    return cmd(*args, **kwargs)
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/cli/cmds.py", line 102, in wrapper
    return asyncio.run(func(*args, **kwargs))
           ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 194, in run
    return runner.run(main)
           ~~~~~~~~~~^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 721, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/cli/features.py", line 82, in collections_list
    output = [c async for c in results]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/clients/features.py", line 100, in list_collections
    response = await self._session.request(method='GET', url=url)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/http.py", line 404, in request
    http_response = await self._retry(self._send, request, stream=False)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/http.py", line 347, in _retry
    raise e
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/http.py", line 332, in _retry
    resp = await func(*a, **kw)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/http.py", line 410, in _send
    http_resp = await self._client.send(request, stream=stream)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/httpx/_client.py", line 1629, in send
    response = await self._send_handling_auth(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<4 lines>...
    )
    ^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/httpx/_client.py", line 1657, in _send_handling_auth
    response = await self._send_handling_redirects(
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ...<3 lines>...
    )
    ^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/httpx/_client.py", line 1715, in _send_handling_redirects
    raise exc
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/.venv/lib/python3.13/site-packages/httpx/_client.py", line 1697, in _send_handling_redirects
    await hook(response)
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/http.py", line 299, in _raise_for_status
    cls._convert_and_raise(e)
    ~~~~~~~~~~~~~~~~~~~~~~^^^
  File "/Users/torben.barsballe/repos/Planet/planet-client-python/planet/http.py", line 98, in _convert_and_raise
    raise error_type(response.text)
planet.exceptions.InvalidAPIKey: {"detail":"Invalid username/password."}

Other CLI subcommands (orders, subscriptions, etc.) don't have this issue, so I assume there's some error handing missing from the features CLI part.

Aside from that, everything seems to be working fine.

I also skimmed over the last batch of changes - looks good to me, though I did add a minor comment on the doc content.

docs/python/async-sdk-guide.md

asonnenschein · 2025-02-20T19:43:11Z

docs/python/async-sdk-guide.md

@@ -458,6 +458,59 @@ async def download_and_validate():
        cl.validate_checksum(asset, path)
 ```

+### Features API Collections and Features
+
+The Python SDK now supports Features API Collections and Features (note: in the SDK and API, Features are often referred to as items in a collection).


Should we add a link to the platform docs on Features API?

asonnenschein · 2025-02-20T19:44:25Z

docs/python/async-sdk-guide.md

+
+#### Creating a collection
+
+You can use the Python SDK to create Features API collections. 


nit: "You can use the Python SDK to create feature collections in the Features API."

docs/python/async-sdk-guide.md

asonnenschein · 2025-02-20T19:54:44Z

planet/cli/cmds.py

+from planet.cli.options import pretty
+
+
+def command(group: click.Group,


IMO this is a clever solution to the 'too many decorators' problem, however this shouldn't block shipping the Features API client. This can be it's own task that ships as a fast follow enhancement after we ship the Features API client.

planet/cli/features.py

asonnenschein · 2025-02-20T20:04:26Z

planet/cli/features.py

+
+    Example:
+
+    planet features items add my-collection-123 ./my_geom.geojson


Is there support, now or planned, to give the GeoJSON here as a JSON string?

do you mean like the following:

planet features items add my-collection-123 '{"type": "Feature, "geometry": {...}}'

That's a good idea and I hadn't thought about that. If we get feedback that a user would like to do this, we could try to test whether the argument is a filename or valid json. I would probably wait to see if it's a common thing though.

Yup, your example is exactly what I am thinking about. That way a user could simply copy and paste a GeoJSON geometry instead of having to put it in a file.

planet/clients/features.py

asonnenschein

LGTM! I left a few comments, and I see there are some other open comments, but nothing that I think should block this MR. The scope of this MR is already pretty huge, I am happy to see this merged as is and some of the lingering feedback can be fast follows.

stephenhillier reviewed Dec 18, 2024

View reviewed changes

planet/clients/features.py Outdated Show resolved Hide resolved

planet/clients/features.py Outdated Show resolved Hide resolved

planet/clients/features.py Outdated Show resolved Hide resolved

planet/clients/features.py Outdated Show resolved Hide resolved

stephenhillier force-pushed the features-client branch from c7623aa to 005dbdd Compare January 20, 2025 23:57

stephenhillier marked this pull request as ready for review January 27, 2025 17:57

stephenhillier changed the title ~~WIP Features client~~ Features client Jan 31, 2025

tbarsballe and others added 11 commits January 31, 2025 16:00

WIP Features client: list_collections, create_collection, list_featur…

8d0d3c9

…es, create_features

Apply suggestions from code review

66cad5c

added docstrings for features methods

5999a7d

updates to methods, tests, docs and sync client

2d2ef2d

fix typo in features client docstring

649abd4

features cli and tests

98b8173

make sure create tests are post requests

8c6e43a

update cli commands

db97757

add compact mode to collections list

06709f9

add get_feature functionality

12ca718

use items instead of "features"

6d37884

stephenhillier force-pushed the features-client branch from 234be19 to 6d37884 Compare February 1, 2025 00:00

ischneider reviewed Feb 4, 2025

View reviewed changes

planet/sync/data.py Outdated Show resolved Hide resolved

ischneider reviewed Feb 4, 2025

View reviewed changes

planet/sync/features.py Outdated Show resolved Hide resolved

tbarsballe commented Feb 7, 2025

View reviewed changes

planet/sync/features.py Outdated Show resolved Hide resolved

stephenhillier added 7 commits February 7, 2025 15:23

add limit to list functions

70e411b

remove geojson from docs

c07d18c

add limit to sync features client

2a207f0

format

a24d98d

update help text

bd8073b

add Features API to sdk guides

e49b9f8

update actions/cache gh action

9126929

tbarsballe commented Feb 19, 2025

View reviewed changes

docs/python/async-sdk-guide.md Show resolved Hide resolved

asonnenschein reviewed Feb 20, 2025

View reviewed changes

docs/python/async-sdk-guide.md Show resolved Hide resolved

asonnenschein reviewed Feb 20, 2025

View reviewed changes

planet/cli/features.py Show resolved Hide resolved

asonnenschein reviewed Feb 20, 2025

View reviewed changes

planet/clients/features.py Outdated Show resolved Hide resolved

asonnenschein approved these changes Feb 20, 2025

View reviewed changes

stephenhillier added 9 commits February 20, 2025 15:03

refactor decorator stack

b36dd64

remove geom type handling, additions to sdk guide

ad9209d

lint fix

02f0608

add note about collection_id

6395931

add note about reservations

9d3d2b0

link to features API docs

1ff3cb8

assertion no longer always true

f74f966

put back Geometry > Feature upgrade

74b00cf

lint

d5a8b16

stephenhillier merged commit ca33d85 into main Feb 21, 2025
9 checks passed

stephenhillier deleted the features-client branch February 21, 2025 01:16


		#### Creating a collection

		You can use the Python SDK to create Features API collections.

		from planet.cli.options import pretty


		def command(group: click.Group,


		Example:

		planet features items add my-collection-123 ./my_geom.geojson

Features client #1073

Features client #1073

Uh oh!

Conversation

tbarsballe commented Oct 29, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

stephenhillier commented Dec 18, 2024

Uh oh!

tbarsballe commented Dec 19, 2024

Uh oh!

stephenhillier commented Jan 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephenhillier commented Jan 27, 2025

Uh oh!

tbarsballe commented Jan 29, 2025

Uh oh!

stephenhillier commented Jan 29, 2025

Uh oh!

tbarsballe commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

stephenhillier commented Jan 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tbarsballe commented Jan 30, 2025

Uh oh!

stephenhillier commented Jan 31, 2025

Uh oh!

stephenhillier commented Jan 31, 2025

Uh oh!

tbarsballe commented Jan 31, 2025

Uh oh!

Uh oh!

Uh oh!

stephenhillier commented Feb 6, 2025

Uh oh!

Uh oh!

stephenhillier commented Feb 12, 2025

Uh oh!

tbarsballe left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asonnenschein Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

asonnenschein Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asonnenschein Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asonnenschein Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

stephenhillier Feb 20, 2025

Choose a reason for hiding this comment

Uh oh!

asonnenschein Feb 21, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asonnenschein left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

stephenhillier commented Jan 21, 2025 •

edited

Loading

tbarsballe commented Jan 29, 2025 •

edited

Loading

stephenhillier commented Jan 29, 2025 •

edited

Loading

tbarsballe left a comment •

edited

Loading