Add collections

[ygabuev]

Follows the discussion in #73.

Add the concept of a collection with basic CRUD functionality. A collection is an ordered list of items - songs, artist, albums, playlists (other types can be added in the future). Users can create, modify and delete collections.

The proposed endpoints and schemas closely follow those for playlists, however, items don't have to be just songs. Additionally, the updateCollection endpoint has a more elaborate mechanism for adding, removing and moving items around via the add, remove, and move parameters respectively.

New methods:

• getCollections
• getCollection
• createCollection
• updateCollection
• deleteCollection

The existing methods and responses all stay the same.

There are no clear contribution guidelines, so sorry if I opened the PR prematurely without clarifying the details in the discussion. I figured that it's easier to discuss the proposal against a more concrete OpenAPI implementation.

Open questions:

• do we need more item types (podcast, internet radio, ...)?
• should users be able to share collections?
• cover art for collections?

[netlify]

✅ Deploy Preview for opensubsonic ready!

Name	Link
🔨 Latest commit	3889dd7
🔍 Latest deploy log	https://app.netlify.com/projects/opensubsonic/deploys/6a285209fa27910008a68a9d
😎 Deploy Preview	https://deploy-preview-237--opensubsonic.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.
🤖 Make changes	Run an agent on this branch

---

To edit notification comments on pull requests, go to your Netlify project configuration.

[Tolriq]

It must be an extension, markdown first before openAPI.

With that said your API does not work for the ordered part, arrays of content per type does not allow mixed content ordering, the solution also does not properly handle duplicates and moving stuff.

Should be an array of items with mediaType mediaId probably.

[ygabuev]

@Tolriq thanks for the comment. I will provide the markdown docs soon.

You are right, the per-type arrays are wrong, I was confused by the existing semantics for parameters in the GET versions of createX endpoints. I've added a dedicated CollectionItemId schema with type and id fields (the word "media" is a bit out of place here IMO, e.g. artists are not media). Those should be used for creating and updating collections.

I've also added the extension comments and errors for the new endpoints

[ygabuev]

Hey @Tolriq. I am reworking the proposal and would appreciate some input.

I see that many of the older endpoints that manipulate data have a very particular way of using HTTP methods, e.g. createPlaylist allows GET, and deleteUser allows GET and POST (but not DELETE). Also, many body payloads require application/x-www-form-urlencoded MIME type. This quickly becomes unwieldy for complex payloads, particularly when arrays of objects are supplied. For example, creating a collection with a payload similar to

{
  "items": [
    {"type": "song", "id": "1"},
    {"type": "album", "id": "5"}
  ]
}

would require a pretty weird encoding that is - as far as I have checked - not well specified, or at least very convoluted.

I wonder if it makes sense to

• use POST for createCollection, PUT for updateCollection, DELETE for deleteCollection; not use GET
• use application/json in creation and update requests?

I see that you went with this approach in a recent change, so at least there is a precedent of not following the legacy Subsonic approaches.

[Tolriq]

New endpoints should use json it was added for that and move away from the limitation of original API.

[ygabuev]

I've modified the proposal a bit and also added the markdown pages, now it's application/json all the way. Would appreciate another look and more comments.

[lachlan-00]

collection should support everything which i think would be

• album
• artist
• bookmark bookarks are just entries so skip them
• genre
• internetradiostation
• playlist
• podcastepisode
• podcast
• share
• song
• user
• video

if you're going to enforce PUT and DELETE why add separate endpoints? it would only make sense if you were going to allow GET on all endpoints as well.

everything could just be on a collection endpoint if we are moving extensions to this style

[Tolriq]

There's no PUT and DELETE in the rest of the API there's no reason to introduce that now.

For collections, I do not see why it should contain bookmark or users or share.

[lachlan-00]

okay then take it out of the delete requirement for the endpoint https://github.com/opensubsonic/open-subsonic-api/pull/237/changes#diff-431be3340a9769c0f1c11bf62185826558cf6b1eb4de14bb6103e0f0db672539R14

bookmarks are just entries so they can be skipped but shares and users are unique enough to stay for me.

they're not as useful but it would be good to support everything possible

[Tolriq]

Shares can expire so does not make sense. Users are on most servers not visible from other users so also does not make sense.

[lachlan-00]

okay then don't add them

[ygabuev]

I've added the following types:

• genre
• internet radio station
• podcast episode
• podcast

Didn't add videos because they are not well defined in the API, would rather leave this as future work.

As a necessity, updated the collectionItemId type to support either id or name field because not all types support IDs, e.g. genre.

Also updated the HTTP method usage to only use GET and POST, in line with the rest of the API.

[aweathe]

My biggest issue is the idea that a collection can be anything, people keep saying it but it just doesn't make sense to me. As an ex-API developer, I always followed the guiding principal of separation of concerns. Songs, albums, genres and artists all have intrinsically different data. Combining any API call would cause chaos from a frontend perspective and likely hurt the legibility of the schema - everything now needs to handle IF/ELSE/ELSEIF/SWITCH scenarios. The best scenario is almost always separation.

As an end goal I'd like album collections to be a first class citizen. eg.

Albums - All albums
Collections - Collated lists of albums
Favourites - Favourite Tracks/Albums
Playlists - Mix-tapes of favourite tracks

• A collection of songs is just a playlist. Artists and genres would be far better and easier implemented as tags.
• A collection of albums is the only example that has a clear purpose outside of the default usages and a clear separation of concerns.
• A mix of songs and albums would be a headache for any frontend developer, which is why I err on the side of separation.

Tags aren't really the answer for album collections as they will always just end up in some generic tags section of UI (worked on many APIs and this usually happens) - album collections need to be a first class citizen like Albums, Artists, Favourites and Playlists.

Think about the frontend developer and how much of a pain that if/then/else/switch will be if it combines data types.

[Tolriq]

What about thinking about user's need ? ;) A collection is a group of things. If you collect pokemons your collection can have cards, lego, books, .... And it still completely make sens ;)

[aweathe]

What about thinking about user's need ? ;) A collection is a group of things. If you collect pokemons your collection can have cards, lego, books, .... And it still completely make sens ;)

If you want a feature to have uptake by a wider number of clients and devices, the ease of implementation will dictate that. An endpoint that returns 5 or 6 types is asking to be pushed to the bottom of a list. A feature with a single datatype is easier to implement and creates less spagetti in the code.

Your pokemon is a great example of how the API should be laid out - an API should separate concerns, just like code and a contract should return a single, actionable datatype. Trying to pickup "my whole collection" is asking for trouble haha.

What it SHOULD return while still making everyone happy is:

/collections/cards
/collections/books
/collections/lego
/collection/sstuffed

/collections should at MOST return a list of available types, e.g card, book, lego, stuffed so the app can know what endpoints it can call for what features. Then developers can do what they want and implement each feature as they see fit.

A single endpoint returning multiple types is asking for trouble

[Tolriq]

Again not really, specially if you want ordering of the collection data ...

We are not talking about adding multiple tags on items and filtering on that tag, we are talking about ordering a collection.

For the Pokemon example, I want my shelf to show my collection ordered by year, so it will mix cards and lego and books in a specific order.

We define an API to fit a need, not the opposite way.

[aweathe]

Again not really, specially if you want ordering of the collection data ...

We are not talking about adding multiple tags on items and filtering on that tag, we are talking about ordering a collection.

For the Pokemon example, I want my shelf to show my collection ordered by year, so it will mix cards and lego and books in a specific order.

We define an API to fit a need, not the opposite way.

But in what way is a collection of songs, genres, albums, radio stations and artists filling a need? What's the use-case for that? I'm genuinely curious and the pokemon example doesn't work anymore, we're talking about music. What's the need? How does it play out? What does it look like to a user?

[aweathe]

Being clear, I want this feature, I just want it to make sense and actually be added to clients so I can use it (yours as awell @Tolriq ).

[Tolriq]

This is actually a frequent request on Symfonium to have collections of pinned different ordered items in rows.

[epoupon]

Again not really, specially if you want ordering of the collection data ...
We are not talking about adding multiple tags on items and filtering on that tag, we are talking about ordering a collection.
For the Pokemon example, I want my shelf to show my collection ordered by year, so it will mix cards and lego and books in a specific order.
We define an API to fit a need, not the opposite way.

But in what way is a collection of songs, genres, albums, radio stations and artists filling a need? What's the use-case for that? I'm genuinely curious and the pokemon example doesn't work anymore, we're talking about music. What's the need? How does it play out? What does it look like to a user?

Same feeling here, I would be very curious about the actual use cases for mixing all this heterogeneous stuff

[aweathe]

The GetStarred endpoint already does this with multiple object types, so I'm bowing out lol. There's no point in arguing API semantics when there's already an endpoint that does it. I hope to see it in the clients soon and not just buried in a "tags" or "collections" screen. My use case stands that I want to see collections of albums, as in Billboard Top 50 1982 followed by a list of album covers. As long as it does that, I'm fine with it.

https://opensubsonic.netlify.app/docs/endpoints/getstarred/

[Tolriq]

You can downvote facts, this won't change that many actual users request that ... It's the pinned items of Spotify and other apps but not limited to one pinned list.

And whatever solution is chosen, most clients will probably have a collection screen that list collection and you can enter, what other way do you expect ? Else it's actually tags applied and smart filters but you loose ordering.

[aweathe]

You can downvote facts, this won't change that many actual users request that ... It's the pinned items of Spotify and other apps but not limited to one pinned list.

And whatever solution is chosen, most clients will probably have a collection screen that list collection and you can enter, what other way do you expect ? Else it's actually tags applied and smart filters but you loose ordering.

I didn't downvote facts, I downvoted that it's a "frequent request" - if there were supported facts in your statement, and a legitimate recurring use-case I would have given it a thumbs up. I asked a question and received a "because" instead.

[aweathe]

Spotify is a great example, I can follow an artist, favourite an album (or add it to a folder), I can favourite a song.. where is this screen where I can do everything?

[Tolriq]

I didn't downvote facts, I downvoted that it's a "frequent request" - if there were supported facts in your statement, and a legitimate recurring use-case I would have given it a thumbs up. I asked a question and received a "because" instead.

So you have access to my mails and searched the forum for the opened feature requests to know that I invent things and that it's not already prepared inside the app because I obviously love to add things that no user wants ;) ?

As I said facts ....

And for spotify as said it's the pin stuff https://community.spotify.com/t5/Live-Ideas/Increase-the-amount-of-pins-available-in-Your-Library/idc-p/6667151 so you can see that many people use and want more of that ;)

[Tolriq]

Tracks, playlists, artists, radio, ... have images right ?

So you can absolutely display a table of mixed content or a grid of mixed content, I have no idea why you find that impossible ?

Clients can add a small overlay for the type of each entry if mixed, or even different image handling like round for artists, or whatever they want. But the fact that it's mixed or not have 0 impact on what's possible client side.

Edit: Anyway you have clarified the important part that was not clear from your comments, so everything is already ready and planned at least in one client ;)

[aweathe]

Edit: Anyway you have clarified the important part that was not clear from your comments, so everything is already ready and planned at least in one client ;)

A client that I already paid for LOL

[Tolriq]

with that said @ygabuev can you add a coverArt to the collection ?

[ygabuev]

with that said @ygabuev can you add a coverArt to the collection ?

I think users should be able to add their own cover art to collections, but I don't see any examples in API of how this should be done. If there were an approach in the API, it should also cover playlists, internet radio stations, podcasts, possibly avatars for users etc. How is this done for most clients and servers now?

Added the covertArt field for now, I assume this comes out-of-band.

Also added the itemCount and itemOffset parameters in getCollection for pagination.

[Tolriq]

coverArt for now comes from the server, client side metadata edition is a larger more complex issue with security impacts via malformed images, too large images, ... That's another complex discussion for another time :)

[kgarner7]

Only reviewed the markdown so far.

[kgarner7]

Would it make sense to give the option to not return the collection body on create?

[ygabuev]

Not sure about that. This follows the createPlaylist example - we can deviate from it, but I don't see a big point. An alternative could be to return the id of the new collection.

[kgarner7]

The big point is, if you are creating large collections but don't care about the results, you are then forcing a large serialization step not requested by the client.

[ygabuev]

So wdyt about returning an id or an object with an id?

[kgarner7]

Thanks for the point! The simplest would be something like includeItems=false, which has every field except items (makes the types easy).

[kgarner7]

An aside for this point, but you could also add that for getCollection as well (for example, a client only wishes to get collection metadata and not the items).

[ygabuev]

If we do that, we can also combine the collection and collectionWithItems types with the items field optional.

[kgarner7]

What determines whether id or name is appropriate? Would id globally be better (and interpreting the genre value as the id)?

[ygabuev]

There is a small description above. If id exists, it should be id. If not, then name. Not sure if using id for a genre would be clear, but I agree the proposed approach is a bit more complex.

[kgarner7]

I think having consistent id is simpler (especially for generators); for genre, the name is its identifier after all.

[ygabuev]

Another thing that's missing atm is something like getCollectionsByItemID, which might be helpful during the add-to-collection dialogs.

Also not clear if duplicate items should be allowed.

[kgarner7]

Another thing that's missing atm is something like getCollectionsByItemID, which might be helpful during the add-to-collection dialogs.

Also not clear if duplicate items should be allowed.

For your consideration, playlists allow duplicate items. If you don't allow duplicates, then you should consider whether duplicate IDs should be an error, or silently ignored.

[lachlan-00]

as a list of items a collection should allow duplicates unless explicitly set

probably as a true/false property of a collection (allowDuplicates / unique)

[AlexGustafsson]

It's a bit difficult to track the full conversation, but I think I agree with @aweathe.

The main feature I see for this is to be able to group albums, playlists and artists (think discoverability through "daily mixtapes", "artists you like" and things like that).

Mixing playlists and songs is just a bigger playlist. Mixing albums and songs is mostly a playlist with all the albums' songs and the other songs? A collection of songs really is just a playlist. Other combination sounds more of a concern for an application, not the API? What would a portion of an app look like using a collection of songs and cover art, is some sort of precedence?

An "everything API" always has the risk to make client support more difficult and it can lead to differences in implementations (as in, some apps might end up having to ignore one or more of the types of the collection).

As always, it can be a bit hard to convey the intended tone in text, so let me be clear that this mostly sounds like a great feature that I very much look forward to, but I have some concerns and I'm trying to understand the different use cases.

[Tolriq]

I don't need this so other don't ? Either we enable mixed content for all and it's generic collections usable for many purpose or we are back to playlist of albums and playlist of artists.

There's no need to add a concept of collection to extend playlists ... Just extend playlists.

[jatin-gurjar-enjoy]

i have nothing to add other than the fact that i check this discussion every week because i really do want this feature

[Tolriq]

I'd love that too :) But I'm only client side we need reactions from the servers for such features.

[jatin-gurjar-enjoy]

I don't need this so other don't ? Either we enable mixed content for all and it's generic collections usable for many purpose or we are back to playlist of albums and playlist of artists.

There's no need to add a concept of collection to extend playlists ... Just extend playlists.

I have one question about this which is that but playlist list every song of the albums, while collections don't, allowing for easier management of albums,artists etc by grouping them together.

not that your concept is wrong, yours already allows what i speak of here, but was just wondering really.

[Tolriq]

As said a couple of times if we want playlists of albums and artists we expand the already working concept of playlists to support albums and artists to have that.

Collections are another concept of MIXED content, hence the proposal. The added benefits of collections is that they can also work as playlist if you want to use them as that. Filling all the need instead of building on the currently limited playlist api.