Transient and Unison

(From https://gitter.im/Transient-Transient-Universe-HPlay/Lobby?at=5d0dea29d1aaa16964e9f3e3)

Unison: https://github.com/unisonweb/unison

In the other side, as far as I know, unison is not reactive. this means that a remote node can not stream events back to the calling node:

     event <- sendFromRemote
     processLocal event
     where
     sendFromRemote nod = atNode nod $ choose [1..]

That is very important because most distributed applications stream events (data bursts, messages, communications, events, name them as you like) instead of single results.

Unison uses remote procedure calls + code movement, while transient transport closures. the difference is:

      -- in transient
          x <- localProc
          y <- localProc'
          z <- runAt remoteNode  $ do
                                         process with all the variables
                                         instantiated in the  closure: x, y ....
                                             

     -- in unison
         x <- localProc
         y <- localProc'
         z <- runAt remoteNode  $ \x -> only can access to x.     --  should be \ x y -> 

The former gives more freedom and composability. I have to make this claim more concrete.

Also, I don't know if Unison implement a composable multithreading (threads), concurrency and parallelism using applicatives and alternatives as well as injection of events (react and EVars)

A problem of RPC versus transporting closures is that all remote calls should address static procedures. It is not possible to use lambdas so even the above example should not work in unison, since it uses a lambda.

In the last line, instead of:

        z <- runAt remoteNode  $ \x -> only can access to x

It should bee

        z <- runAt remoteNode  remoteProc(x)

        ....
        remoteProc x= .....

It is quite possible that the compiler could rewrite that lambda as a static function and make it work. But there are functions defined locally in the closure that can not be rewritten that way since they have "free" variables:

proc x = do
    let   localProc y =  expression depending on x
    runAt remoteNode   localProc wathever

This localProc can not be rewritten as static, since there are one static localProcs possible for each value of x.

Anything can be overcome. Since Unison is dynamic, maybe the compiler could detect free variables and compile the appropriate static in the remote node, but this is not easy doing it in an efficient way. In practice it is equivalent to transporting closures.

However, the idea of Unison for managing and distributing software is amazing

Transient uses a program the monitorService running in each node which receives request for compiling/installing other services or programs. they are identified by his name and referenced by a URL to a git repo or a docker image, or a unix executable. It has to install whole executables. Closure data transporting needs it anyway since a remote function which is localy defined in the body of anothe function ad so on needs the whole context. Unison RPC does not need to install whole programs. It identifies particular chunks of code by a hash. They are retrieved, compiled and cached when the code execution is requested in the node.

In Transient, since only the data (local variables) of the closure are transported, not the closure code, the calling and the callee should share the same stack structure. Roughly. In práctical terms, this means both should run the same program, although nodes may have different architectures (A Javascript node in a browser can communicate with the server). To communicate programs with completely different codebases, transient has an RPC mechanism, but this comes for free, since it uses the closure transport using something like a pseudo-closure.

closure transport allows more possibilities: while in Unison the basic distributed primitive is a remote function invocation similar to runAt , in Transient that primitive is a composition of two more basic ones:

runAt node f = wormhole node $ do teleport; r <- f ; teleport ; return f 

wormhole opens a connection and teleport transport back and forth the computation between these two nodes connected.

This is the key for being reactive and streaming; if f ,running at the remote node, generate events, the continuation would execute the second teleport, which will transport the data to the node that was connected by wormhole. This is the reason why transient can stream events. By default, each one of these responses run in his own thread.

Moving the whole execution state and continuing in other node is not possible using RPC. In transient it is indeed possible for a program to distribute his execution among many nodes with a complex lifecycle. The f above has access to all the previous execution context that ran in the calling node. it is even possible to store the execution state and continue it at a later time. A system based on RPC can not do so "natively".

execution states are lightweight because they include only the serialization of the variables that are actually on scope (an not the "dead" ones which are no longer in scope). Additionally, only the new variables are transported. In the above program, only the r value would be sent back, even if that is invoked locally in a complex context.

Since the TransIOcomputation can return zero, one or more responses and these responses can be managed by different threads, the f above could either never return anything, be conventional and return a single result, but also can send back results (events, stream of results, call it as you like) to the calling node and the programmer can decide how many thread would handle these responses.