Status of Async/AFF implementation

[ngenovese11]

Hello and thanks for this awesome library. I see in the history some attempts to implement Async and AFF actors that appear to have been reverted. I have a need for some projects and am happy to take up the effort, but would love some insight into why the changes were abandoned.

[StanJav]

Hi @ngenovese11, we use echo-processes in a huge environment. We run thousands of actors in parallel. The original version used a pausable-blocking-queue implementation which caused two real threads (i.e. System.Threading.Thread) to be held for every Process.

The Async + Aff rewrite did not use real threads anymore. Instead, it used tasks from TPL. However TPL uses System.Threading.ThreadPool which does not give you thousands of threads automatically. It gives you a few threads immediately but once you reach a maximum (see SetMinThreads) you need to wait.

Because of this limit the Async + Aff rewrite was deadlocking if we tried to run more processes. A fix was not trivial so the decision was made to revert the version back to use real threads until we find a fix. So far I haven't got time to look at it again.

[louthy]

@StanJav's answer is correct. The full implementation of the effect system into echo requires an almost full rewrite which is painful. The approach taken lead to unintended consequences and so was rolled back. I have some ideas for a less extreme approach, but not the time to do it currently.

You can however roll your own wrappers for your inbox and state functions (that are passed to spawn):

The following (untested!) utility functions allow you to wrap Task yielding functions for setup and inbox into synchronous functions. These will simply put the per-actor blocking-queue (that manages the synchronous yielding of messages) to sleep whilst the asynchronous inbox function does its thing.

    public static class Async
    {
        public static Func<S> Setup<S>(Func<Task<S>> setup) =>
            () =>
            {
                S state = default;
                Exception err = null;
                using (var wait = new AutoResetEvent(false))
                {
                    setup().ContinueWith(t =>
                    {
                        if (t.IsFaulted)
                        {
                            err = t.Exception;
                        }
                        else
                        {
                            state = t.Result;
                        }
                        wait.Set();
                    });
                    wait.WaitOne();
                }

                if (err != null)
                {
                    ExceptionDispatchInfo.Capture(err).Throw();
                }

                return state;
            };

        public static Func<S, A, S> Inbox<S, A>(Func<S, A, Task<S>> inbox) =>
            (s, m) =>
            {
                Exception err = null;
                using (var wait = new AutoResetEvent(false))
                {
                    inbox(s, m).ContinueWith(t =>
                    {
                        if (t.IsFaulted)
                        {
                            err = t.Exception;
                        }
                        else
                        {
                            s = t.Result;
                        }
                        wait.Set();
                    });
                    wait.WaitOne();
                }

                if (err != null)
                {
                    ExceptionDispatchInfo.Capture(err).Throw();
                }

                return s;
            };
    }

[ngenovese11]

Yes full re-write sounds painful; thanks for the insight. I have some tests going with some TPL refactors (specifically changes in the BlockingQueues), but not at the scale noted above. In the meantime everything is functional and I can work around it with various versions of the wrappers noted above. Thanks a lot for the responses!

[louthy]

Yes full re-write sounds painful

It would have been cathartic if it had worked! ;) The underlying actor-system is a bit ugly as it grew out of a smaller prototype, so it was an opportunity to clean some of that up too. But, in the end the approach wasn't quite right. And although the underlying code is ugly, it works and is battle hardened, and that's always the most valuable thing!

[louthy]

Support is now in v2.2.0. Testing is limited, so any feedback is welcome.

https://github.com/louthy/echo-process/releases/tag/v2.2.0-beta