Suggestion: circuit-relay-like module, but for negotiating WebRTC

[guysv]

So, the best part of circuit-relay, is the fact that it allows to connect nodes where neither can listen for incoming connections.

Unfortunately, running a circuit-relay HOP node is expensive, and by default HOP is turned off.

tl;dr: I hacked js-libp2p-circuit into negotiating WebRTC signals using newly relayed connections, and establish a direct connection between unconnected peers using a helping 3rd party.

Such connection scheme is more considerate of "relay" nodes, because they only need to relay the signaling data, and not the entire connection.
Another plus, is that now browsers should be able to connect to one another directly.

I uploaded the modded js-libp2p-circuit repo and a demo to github:
You can view the diff to the original circuit-relay module in this PR: guysv/js-libp2p-webrtc-poc#1

To run the demo:

1. npm install (note: I included a custom package-lock.json in the demo repo so that js-libp2p-switch will use my modded circuit module, if you feel extra paranoid, regenerate the package-lock.json and fill in guysv/js-libp2p-webrtc-poc#webrtc-poc
2. run the hop node DEBUG=libp2p:circuit:* node src/hop.js
3. run listener node DEBUG=libp2p:circuit:* node src/listener.js
4. run dialer node DEBUG=libp2p:circuit:* node src/dialer.js
5. look for those logs I added, which indicate that WebRTC connection was established with listener node

# dialer node
  libp2p:circuit:dialer Writing signal to circuit connection +21ms
  libp2p:circuit:dialer Writing signal to circuit connection +1ms
  libp2p:circuit:dialer Read signal from circuit connection +46ms
  libp2p:circuit:dialer Connected! +21ms

# listener node
  libp2p:circuit:stop Read signal from circuit connection +0ms
  libp2p:circuit:stop Read signal from circuit connection +3ms
  libp2p:circuit:stop Writing signal to circuit connection +11ms
  libp2p:circuit:stop Writing signal to circuit connection +2ms
  libp2p:circuit:stop Connected! +22ms

The connection is not really usable, but those logs indicate that a WebRTC channel was established, good enough for POC :)

Anyhow, That was a cool experiment. I wrote this issue to hear what you guys think about it. I tried to find material about WebRTC in Libp2p but the only modules I found were webrtc-star, which is a centralized solution, and webrtc-direct, which doesn't really offer interesting features that websockets lack. I think this could be a cool addition to the stack!

[jacobheun]

There is a WIP by @mkg20001 to convert webrtc-star to a distributed architecture at libp2p/js-libp2p-webrtc-star#148. It's currently blocked by some needed work on libp2p-crypto libp2p/js-libp2p-crypto#125.

[guysv]

@mkg20001 , I see you are already one year deep in this.
I crawled through your work and slides, but I did not figure out what extra benefits do we get from randevouz-exchange if we can tunnel direct-exchange over libp2p-circuit as seen in slide 6. (and demo'ed by my PoC, sort of).

[mkg20001]

@guysv The extra benefit isn't really for upgrading but for esablisihng webrtc connections between browsers. If two browsers are connected to the same bootstrap node that also runs the rendezvous-exchange, then they can connect with each other over webrtc using this method. It intents to be a better replacement for the current socket.io based signaling server, since server selection and discovery is fully dynamic.

[guysv]

@mkg20001 But we already have a way to communicate over mutual peer, circuit relay. Why not reuse it instead? Especially concidering the fact the the implementation of a request-response API as you are trying to create with randevouz-exchange is facing a blocking issue involving crypto incompetability?

Regarding WebRTC integration, the only benefit I see randevouz-exchange has over circuit relay is that randevouz-exchange offers an API for a limited data exchange, where circuit-relay as it is now is unlimited and hence prone to abuse.

The unlimited nature of circuit relay is solvable, of course. Whatever the solution, it won't be as generic as having a data exchange API, but it is sufficient for WebRTC.

[mkg20001]

@guysv

But we already have a way to communicate over mutual peer, circuit relay. Why not reuse it instead?

Establishing a p2p-circuit connection takes resources and isn't of much use considering we are going to toss that circuit connection once we connect via webrtc anyways.
Rendezvous exchange prevents high resource usage by not establishing a new connection but instead using the PeerIDs to securely pass data back and forth.
Basically it takes the destination peers public key (it can ask for it at the exchange server if it doesn't have it) to encrypt the request, the exchange forwards it and then the other side sends back an encrypted ACK/NACK with additional data that the other side can use to finalize the connection.

Edit: I understand that this (simple asymmetric crypto) isn't the worlds most secure way of request->response exchanging (in comparison to SECIO), but the other side's identity is going to be verified using SECIO anyways. Most attacks would be of no use (except for the usual "send invalid data, mess with memory, and cause a RCE" kind of vulnerability which would apply to the other, current, webrtc transport as well)

[guysv]

Establishing a p2p-circuit connection takes resources

If the connection has low traffic, why aren't the needed resources low too?

[mkg20001]

If the connection has low traffic, why aren't the needed resources low too?

SECIO takes a few network roundtrips to establish a connection, and also it seems that the browser uses a pretty inefficient way of encrypting the stream (the WebCrypto api has no way to encrypt a stream directly, every buffer must be encrypted separately)

Rendezvous exchange reuses the existing connection with the bootstrap server and sends just a single request and a single response packet (unless the discovery mechanism doesn't provide the peer's public key alongside the peers id)

[guysv]

I see your point, thats indeed a better idea. I saw that you plan on using native JS crypto, how does that compare to WebCrypto in terms of performence?

[mkg20001]

@guysv Well, I sort of "had no other choice", as webcrypto PKCS-v1 encryption support seems broken, so I had to go with node-forge, though only for the encryption part. Feel free, though, to contribute here libp2p/js-libp2p-crypto#125 if you have a good solution in mind

[guysv]

You know, I gave my solution another thought
We could modify circuit into transporting arbitrary data prior to SECIO handshake, and hence being more lightweight. We then use that to move around WebRTC signal, and cut on the said double connection handshake.

Which brings me to the other question, you could cut the crypto, and release randevouz-exchange in no time, this will be sufficient for WebRTC. Why shouldn't you?

[mkg20001]

Why shouldn't you?

Modularity. My solution focused on having both an rendezvous exchange transport and a direct exchange transport, with the ability to swap them for one another or pick the best dynamically.

In the roadmap (see slide) support for a tiny "router" module that would pick the "best" based on path cost (direct is faster then rendezvous) was mentioned.

[guysv]

Well i get that polymorphic approach, but I think that if you ran into troubles doing the crypto, why not postponing to a future iteration? (For the record, I tried to dive in, but I don't see myself qualified to take crypto decisions) Decentralized WebRTC transport is a game changer for libp2p web apps and should be ready ASAP.

On another topic, On the roadmap you mentioned:

• Support passing the swarm as an argument to a transport

Is there an effort underway?

[mkg20001]

Well i get that polymorphic approach, but I think that if you ran into troubles doing the crypto, why not postponing to a future iteration?

Because I thought I could solve it. Then, when I realized I couldn't, I asked for some help. But no one took on the task libp2p/js-libp2p-crypto#125. Also when I asked for a re-review after giving some clarifications on the PR, nobody looked at it. So I deemed the whole endeavor unnecessary. Well, until now libp2p/js-libp2p-webrtc-star#148 (comment). So here we are... And I just happen to no longer have time for that, because I'm currently trying to move out and "get a life", like people always said, while also having an internship with a 99% chance of becoming an apprenticeship. So, sorry but no demos and refactoring for now, unless I somehow get some time freed up in my scheudle.

Is there an effort underway?

There is always this "libp2p-thing needs swarm, but swarm needs libp2p-thing initialized" discussion that keeps popping up and then gets "solved" for a specific part of the swarm. Example would be wsStar: It required a peerID for crypto, so discovery modules now have access to just the peerID.
AFAIK, no. Or not something I'd consider a proper solution to those problems.
What we really need is a separation of low-level stuff (TCP, UDP, Multicast, things that DON'T depend on swarm) and high-level stuff (wsStar, wrtcStar, stardust, things that utilize low-level swarm modules to do stuff like dialing, etc). But it should be done properly

[guysv]

@jacobheun @mkg20001 So in conclusion, libp2p/js-libp2p-webrtc-star#148 is waiting for the integration of mkg20001/interface-data-exchange into libp2p, but that won't happen until libp2p/js-libp2p-crypto#125 is resolved even though libp2p/js-libp2p-webrtc-star#148 does not need mkg20001/interface-data-exchange to be secure? Bummer.

[guysv]

Just found out i'm not even the first one to try my approach: dryajov/js-libp2p-webrtc-circuit I really think we should work out interface-data-exchange integration into libp2p before figuring out how to secure it.