feat: replace websocket-star with stardust

README.md

@@ -537,6 +537,10 @@ The libp2p option allows you to build your libp2p node by configuration, or via
 
 You can see the bundle in action in the [custom libp2p example](examples/custom-libp2p).
 
+- `overrideFunction` (object):
+    - A function that allows overriding parts of libp2p's config, using dynamic arguments, without fully replacing the bundle as a custom libp2p bundle would do
+    - It is called with the arguments `{ datastore, peerInfo, peerBook, options, config }`
+- `extend` (boolean): Whether to fully replace the transport/discover arrays (`false`) or to concat them (`true`)
 - `modules` (object):
     - `transport` (Array<[libp2p.Transport](https://github.com/libp2p/interface-transport)>): An array of Libp2p transport classes/instances to use _instead_ of the defaults. See [libp2p/interface-transport](https://github.com/libp2p/interface-transport) for details.
     - `peerDiscovery` (Array<[libp2p.PeerDiscovery](https://github.com/libp2p/interface-peer-discovery)>): An array of Libp2p peer discovery classes/instances to use _instead_ of the defaults. See [libp2p/peer-discovery](https://github.com/libp2p/interface-peer-discovery) for details. If passing a class, configuration can be passed using the config section below under the key corresponding to you module's unique `tag` (a static property on the class)
@@ -1003,22 +1007,27 @@ The code above assumes you are running a local `signaling server` on port `9090`
 
 #### Is there a more stable alternative to webrtc-star that offers a similar functionality?
 
-Yes, websocket-star! A WebSockets based transport that uses a Relay to route the messages. To enable it, just do:
+Yes, stardust! A WebSockets based transport that uses a Relay to route the messages. To enable it, just do:
 
 ```JavaScript
 const node = await IPFS.create({
+  libp2p: {
+    overrideFunction: require('stardust4ipfs')
+  },
   config: {
     Addresses: {
       Swarm: [
-        '/dns4/ws-star.discovery.libp2p.io/tcp/443/wss/p2p-websocket-star'
+        '/dns4/stardust.mkg20001.io/tcp/443/wss/p2p-stardust'
       ]
     }
   }
 })
 
-// your instance with websocket-star is ready
+// your instance with stardust is ready
 ```
 
+(You may need to install the stardust4ipfs module as well, with `npm install stardust4ipfs`)
+
 #### I see some slowness when hopping between tabs Chrome with IPFS nodes, is there a reason why?
 
 Yes, unfortunately, due to [Chrome aggressive resource throttling policy](https://github.com/ipfs/js-ipfs/issues/611), it cuts freezes the execution of any background tab, turning an IPFS node that was running on that webpage into a vegetable state.

examples/circuit-relaying/README.md

@@ -155,7 +155,7 @@ In order to enable the relay functionality in `go-ipfs` we need to edit it's con
 
 The two options we're looking for are `DisableRelay` and `EnableRelayHop`. We want the former (`DisableRelay`) set to `false` and the latter (`EnableRelayHop`) to `true`, just like in the example above. That should set our go node as a relay.
 
-We also need to make sure our go node can be dialed from the browser. For that, we need to enable a transport that both the browser and the go node can communicate over. We will use the web sockets transport, although there are others that can be used, such as `webrtc-star` and `websocket-star`. To enable the transport and set the interface and port we need to edit the `~/.ipfs/config` one more time. Let's find the `Swarm` array and add our desired address there. I picked `/ip4/0.0.0.0/tcp/4004/ws` because it is a port I know is not being used by anything on my machine, but we can also use port `0` so that the OS chooses a random available port for us — either one should work.
+We also need to make sure our go node can be dialed from the browser. For that, we need to enable a transport that both the browser and the go node can communicate over. We will use the web sockets transport, although there are others that can be used, such as `webrtc-star` and `stardust`. To enable the transport and set the interface and port we need to edit the `~/.ipfs/config` one more time. Let's find the `Swarm` array and add our desired address there. I picked `/ip4/0.0.0.0/tcp/4004/ws` because it is a port I know is not being used by anything on my machine, but we can also use port `0` so that the OS chooses a random available port for us — either one should work.
 
 ```json
   "Swarm": [

examples/custom-libp2p/index.js

@@ -4,7 +4,7 @@ const Libp2p = require('libp2p')
 const IPFS = require('ipfs')
 const TCP = require('libp2p-tcp')
 const MulticastDNS = require('libp2p-mdns')
-const WebSocketStar = require('libp2p-websocket-star')
+const Stardust = require('libp2p-stardust')
 const Bootstrap = require('libp2p-bootstrap')
 const SPDY = require('libp2p-spdy')
 const KadDHT = require('libp2p-kad-dht')
@@ -32,8 +32,8 @@ const libp2pBundle = (opts) => {
   const peerBook = opts.peerBook
   const bootstrapList = opts.config.Bootstrap
 
-  // Create our WebSocketStar transport and give it our PeerId, straight from the ipfs node
-  const wsstar = new WebSocketStar({
+  // Create our Stardust transport and give it our PeerId, straight from the ipfs node
+  const stardust = new Stardust({
     id: peerInfo.id
   })
 
@@ -50,7 +50,7 @@ const libp2pBundle = (opts) => {
     modules: {
       transport: [
         TCP,
-        wsstar
+        stardust
       ],
       streamMuxer: [
         MPLEX,
@@ -62,7 +62,7 @@ const libp2pBundle = (opts) => {
       peerDiscovery: [
         MulticastDNS,
         Bootstrap,
-        wsstar.discovery
+        stardust.discovery
       ],
       dht: KadDHT
     },

examples/custom-libp2p/package.json

@@ -17,7 +17,7 @@
     "libp2p-secio": "~0.11.1",
     "libp2p-spdy": "~0.13.3",
     "libp2p-tcp": "~0.13.0",
-    "libp2p-websocket-star": "~0.10.2",
+    "libp2p-stardust": "~0.1.2",
     "pull-mplex": "~0.1.0"
   }
 }

examples/exchange-files-in-browser/package.json

@@ -9,7 +9,8 @@
   "license": "MIT",
   "devDependencies": {
     "browserify": "^16.2.3",
-    "http-server": "~0.11.1"
+    "http-server": "~0.11.1",
+    "stardust4ipfs": "~0.1.3"
   },
   "dependencies": {
     "ipfs": "file:../../"

examples/exchange-files-in-browser/public/app.js

@@ -46,9 +46,12 @@ async function start () {
   if (!node) {
     const options = {
       repo: 'ipfs-' + Math.random(),
+      libp2p: {
+        overrideFunction: require('stardust4ipfs')
+      },
       config: {
         Addresses: {
-          Swarm: ['/dns4/ws-star.discovery.libp2p.io/tcp/443/wss/p2p-websocket-star']
+          Swarm: ['/dns4/stardust.mkg20001.io/tcp/443/wss/p2p-stardust']
         }
       }
     }
@@ -154,7 +157,7 @@ const publishHash = (hash) => {
 const sendFileList = () => Promise.all(FILES.map(publishHash))
 
 const updateProgress = (bytesLoaded) => {
-  let percent = 100 - ((bytesLoaded / fileSize) * 100)
+  const percent = 100 - ((bytesLoaded / fileSize) * 100)
 
   $progressBar.style.transform = `translateX(${-percent}%)`
 }
@@ -261,7 +264,7 @@ async function connectToPeer (event) {
 
   await node.swarm.connect(multiaddr)
 
-  onSuccess(`Successfully connected to peer.`)
+  onSuccess('Successfully connected to peer.')
   $multiaddrInput.value = ''
 }
 

package-list.json

@@ -49,8 +49,8 @@
     ["libp2p/js-libp2p-bootstrap", "libp2p-bootstrap"],
     ["libp2p/js-libp2p-secio", "libp2p-secio"],
     ["libp2p/js-libp2p-tcp", "libp2p-tcp"],
+    ["libp2p/js-libp2p-stardust", "libp2p-stardust"],
     ["libp2p/js-libp2p-webrtc-star", "libp2p-webrtc-star"],
-    ["libp2p/js-libp2p-websocket-star", "libp2p-websocket-star"],
     ["libp2p/js-libp2p-websockets", "libp2p-websockets"],
     ["libp2p/pull-mplex", "pull-mplex"],
 

package.json

@@ -144,7 +144,6 @@
     "libp2p-secio": "~0.11.0",
     "libp2p-tcp": "^0.13.0",
     "libp2p-webrtc-star": "~0.16.0",
-    "libp2p-websocket-star-multi": "~0.4.3",
     "libp2p-websockets": "~0.12.3",
     "lodash.flatten": "^4.4.0",
     "mafmt": "^6.0.10",
@@ -207,13 +206,14 @@
     "interface-ipfs-core": "^0.118.0",
     "ipfs-interop": "^0.1.1",
     "ipfsd-ctl": "^0.47.2",
-    "libp2p-websocket-star": "~0.10.2",
+    "libp2p-stardust": "~0.1.2",
     "lodash": "^4.17.15",
     "ncp": "^2.0.0",
     "p-event": "^4.1.0",
     "qs": "^6.5.2",
     "rimraf": "^3.0.0",
     "sinon": "^7.4.2",
+    "stardust4ipfs": "^0.1.3",
     "stream-to-promise": "^2.2.0",
     "temp-write": "^4.0.0"
   },
@@ -359,3 +359,4 @@
     "Максим Ильин <[email protected]>"
   ]
 }
+

src/core/components/libp2p.js

@@ -8,20 +8,63 @@ const multiaddr = require('multiaddr')
 const DelegatedPeerRouter = require('libp2p-delegated-peer-routing')
 const DelegatedContentRouter = require('libp2p-delegated-content-routing')
 const PubsubRouters = require('../runtime/libp2p-pubsub-routers-nodejs')
+const Libp2p = require('libp2p')
 
 module.exports = function libp2p (self, config) {
   const options = self._options || {}
   config = config || {}
 
   // Always create libp2p via a bundle function
-  const createBundle = typeof options.libp2p === 'function'
+  const createBundle = typeof options.libp2p !== 'undefined' ? (Array.isArray(options.libp2p) ? options.libp2p : [options.libp2p]) : []
+  createBundle.unshift(defaultBundle)
+
+  /* typeof options.libp2p === 'function'
     ? options.libp2p
-    : defaultBundle
+    : defaultBundle */
 
   const { datastore } = self._repo
   const peerInfo = self._peerInfo
   const peerBook = self._peerInfoBook
-  const libp2p = createBundle({ options, config, datastore, peerInfo, peerBook })
+
+  const end = createBundle.length - 1
+  let libp2p
+  let libp2pOpts
+
+  options.libp2p = libp2pOpts = null
+
+  createBundle.forEach((fncOrObj, i) => {
+    if (typeof fncOrObj === 'function') {
+      const r = fncOrObj({ options, config, datastore, peerInfo, peerBook })
+      if (r instanceof Libp2p) {
+        if (i === end) {
+          libp2p = r
+        } else {
+          throw new Error('Using chained, but non-last function returned instance')
+        }
+      } else if (typeof r === 'object') {
+        if (r.extend) {
+          libp2pOpts = options.libp2p = mergeOptions.call({ concatArrays: true }, libp2pOpts, r) // extend
+        } else {
+          libp2pOpts = options.libp2p = r // override
+        }
+      } else if (typeof r === 'undefined') {
+        // ignore, go on
+      } else {
+        // maybe print a warning?
+      }
+    } else if (typeof fncOrObj === 'object') {
+      libp2pOpts = options.libp2p = mergeOptions.call({ concatArrays: fncOrObj.extend }, libp2pOpts, fncOrObj)
+    } else {
+      throw new TypeError('Option .libp2p has invalid type ' + typeof fncOrObj)
+    }
+  })
+
+  if (!libp2p) {
+    // Required inline to reduce startup time
+    // Note: libp2p-nodejs gets replaced by libp2p-browser when webpacked/browserified
+    const Node = require('../runtime/libp2p-nodejs')
+    libp2p = new Node(libp2pOpts)
+  }
 
   libp2p.on('stop', () => {
     // Clear our addresses so we can start clean
@@ -133,9 +176,5 @@ function defaultBundle ({ datastore, peerInfo, peerBook, options, config }) {
       })
   }
 
-  const libp2pOptions = mergeOptions(libp2pDefaults, get(options, 'libp2p', {}))
-  // Required inline to reduce startup time
-  // Note: libp2p-nodejs gets replaced by libp2p-browser when webpacked/browserified
-  const Node = require('../runtime/libp2p-nodejs')
-  return new Node(libp2pOptions)
+  return libp2pDefaults
 }

src/core/runtime/libp2p-browser.js

@@ -2,25 +2,20 @@
 
 const WS = require('libp2p-websockets')
 const WebRTCStar = require('libp2p-webrtc-star')
-const WebSocketStarMulti = require('libp2p-websocket-star-multi')
 const Multiplex = require('pull-mplex')
 const SECIO = require('libp2p-secio')
 const Bootstrap = require('libp2p-bootstrap')
 const KadDHT = require('libp2p-kad-dht')
 const GossipSub = require('libp2p-gossipsub')
 const libp2p = require('libp2p')
 const mergeOptions = require('merge-options')
-const multiaddr = require('multiaddr')
 
 class Node extends libp2p {
   constructor (_options) {
     const wrtcstar = new WebRTCStar({ id: _options.peerInfo.id })
 
-    // this can be replaced once optional listening is supported with the below code. ref: https://github.com/libp2p/interface-transport/issues/41
-    // const wsstar = new WebSocketStar({ id: _options.peerInfo.id })
-    const wsstarServers = _options.peerInfo.multiaddrs.toArray().map(String).filter(addr => addr.includes('p2p-websocket-star'))
-    _options.peerInfo.multiaddrs.replace(wsstarServers.map(multiaddr), '/p2p-websocket-star') // the ws-star-multi module will replace this with the chosen ws-star servers
-    const wsstar = new WebSocketStarMulti({ servers: wsstarServers, id: _options.peerInfo.id, ignore_no_online: !wsstarServers.length || _options.wsStarIgnoreErrors })
+    const { extend } = _options
+    delete _options.extend
 
     const defaults = {
       switch: {
@@ -33,8 +28,7 @@ class Node extends libp2p {
       modules: {
         transport: [
           WS,
-          wrtcstar,
-          wsstar
+          wrtcstar
         ],
         streamMuxer: [
           Multiplex
@@ -44,7 +38,6 @@ class Node extends libp2p {
         ],
         peerDiscovery: [
           wrtcstar.discovery,
-          wsstar.discovery,
           Bootstrap
         ],
         dht: KadDHT,
@@ -73,7 +66,7 @@ class Node extends libp2p {
       }
     }
 
-    super(mergeOptions(defaults, _options))
+    super(mergeOptions.call({ concatArrays: extend }, defaults, _options))
   }
 }
 

src/core/runtime/libp2p-nodejs.js

@@ -3,23 +3,18 @@
 const TCP = require('libp2p-tcp')
 const MulticastDNS = require('libp2p-mdns')
 const WS = require('libp2p-websockets')
-const WebSocketStarMulti = require('libp2p-websocket-star-multi')
 const Bootstrap = require('libp2p-bootstrap')
 const KadDHT = require('libp2p-kad-dht')
 const GossipSub = require('libp2p-gossipsub')
 const Multiplex = require('pull-mplex')
 const SECIO = require('libp2p-secio')
 const libp2p = require('libp2p')
 const mergeOptions = require('merge-options')
-const multiaddr = require('multiaddr')
 
 class Node extends libp2p {
   constructor (_options) {
-    // this can be replaced once optional listening is supported with the below code. ref: https://github.com/libp2p/interface-transport/issues/41
-    // const wsstar = new WebSocketStar({ id: _options.peerInfo.id })
-    const wsstarServers = _options.peerInfo.multiaddrs.toArray().map(String).filter(addr => addr.includes('p2p-websocket-star'))
-    _options.peerInfo.multiaddrs.replace(wsstarServers.map(multiaddr), '/p2p-websocket-star') // the ws-star-multi module will replace this with the chosen ws-star servers
-    const wsstar = new WebSocketStarMulti({ servers: wsstarServers, id: _options.peerInfo.id, ignore_no_online: !wsstarServers.length || _options.wsStarIgnoreErrors })
+    const { extend } = _options
+    delete _options.extend
 
     const defaults = {
       switch: {
@@ -32,8 +27,7 @@ class Node extends libp2p {
       modules: {
         transport: [
           TCP,
-          WS,
-          wsstar
+          WS
         ],
         streamMuxer: [
           Multiplex
@@ -43,8 +37,7 @@ class Node extends libp2p {
         ],
         peerDiscovery: [
           MulticastDNS,
-          Bootstrap,
-          wsstar.discovery
+          Bootstrap
         ],
         dht: KadDHT,
         pubsub: GossipSub
@@ -76,7 +69,7 @@ class Node extends libp2p {
       }
     }
 
-    super(mergeOptions(defaults, _options))
+    super(mergeOptions.call({ concatArrays: extend }, defaults, _options))
   }
 }