Deprecation of the Google Realtime API

[jochym]

This is what I have got a minute ago. Putting an issue out as a starting point for discussion.
What now? - the google-drive backed notebooks in the Lab are really fantastic collaboration tool.
Is there any similar technology we can migrate to?

---

Hello Google Realtime API developer,

We’re writing to let you know that after careful consideration, we are deprecating the Google Realtime API as of November 28, 2017.

Your Realtime API client applications will continue to work normally until December 11, 2018. To ensure continued availability after this date, please migrate your applications using Realtime API to another data store before December 11, 2018. You can read more about the deprecation in our Realtime API deprecation documentation.

We know developers have come to rely on Realtime API and that migration may be a significant effort. We are grateful to our developers, and we hope that the deprecation plan summarized below allows a smooth transition for you and your product(s).

[ian-r-rose]

Yes, I am as surprised as you are! This is unfortunate news, and we will have to deal with it.

Collaborative notebooks are still a priority for us. The good news is that we have already been working on our own solution for real-time collaboration that does not depend on the Google Realtime API. Unfortunately, it is still uncertain how long that will take, and what it will ultimately look like.

It is important to note that the Google Drive API is still supported, so we will still be able to upload, share, and access notebooks using Google Drive, it's just the realtime capabilities that will go away (until we get our own solution running).

[jochym]

Without real-time features this will be just a shadow of the current tool. Well, it is better to know a year in advance, I guess. We (developers/community) have our goal for the next year picked for us :-/
As a new user of the lab-with-realtime-colab I can just assure the developers that this is a fantastic tool and it is really appreciated by my fellow users with particular emphasis to the realtime collaboration part.

[ian-r-rose]

Glad to hear you are finding it useful, hopefully the transition to the new approach is not too bumpy.

[cedricyau]

Is Firebase an option? Real-time JSON datastore with access to any level of the tree. Also notifies and broadcasts updates.

https://firebase.google.com/

[ian-r-rose]

The main problem with Firebase, as far as I know, is that it does not have the sophisticated conflict resolution algorithms (like operational transforms or CRDTs) that are required for collaborative text editing. At least, that has been my reading of the documentation. If I am mistaken in that, please do let me know!

[cedricyau]

Coincidentally, I followed the links on Realtime API warning to use Cloud Firestore and it went to:

https://firebase.google.com/products/firestore/

Will need to read up more on Firebase and Firestore. Seems like the latter is the newer service and may be more applicable.

[cedricyau]

It seems that unlike Google Drive/Realtime, Cloud Firestore requires a Google API account for billing.

What would be required to at least get real-time running on the same jupyterlab server?

Suppose you and I connected to the same server and opened the same notebook. What would it take for the server to broadcast changes between the two clients?

JupyterLab already allows multiple views into a notebook from within a browser window--so it seem possible for a server hosted file to have it's changes broadcast.

Also, this would allow for the sharing of a kernel.

[ian-r-rose]

So far as I have been able to tell, neither Cloud Firestore nor Firebase provide the conflict-resolution capabilities that we need (in addition to being paid products).

We are currently working on a solution that is hosted with the notebook server. At the moment, the server has no APIs for broadcasting changes and resolving conflicts, which is why it is a tricky problem. The case with multiple views in the same browser window works because they are both stored in memory, and the user is not able to make concurrent, conflicting edits.

The intention is that with the self-hosted solution we are working on kernels could be shared.

[eyadsibai]

Firepad is using firebase for the same purpose. https://github.com/firebase/firepad