Breaking change: Seq1 to Seq

This is the backstory: When language-ext was still in version 1, there was no Seq<A> collection type. There were however Seq(...) functions in the Prelude. Those functions coerced various types into being IEnumerable, and in the process protected against any of them being null.

These are they:

	Seq <A> (A? value)
	Seq <A> (IEnumerable<A> value)
	Seq <A> (A[] value)
	Seq <A> (Arr<A> value)
	Seq <A> (IList<A> value)
	Seq <A> (ICollection<A> value)
	Seq <L, R> (Either<L, R> value)
	Seq <L, R> (EitherUnsafe<L, R> value)
	Seq <A> (Try<A> value)
	Seq <A> (TryOption<A> value)
	Seq <T> (TryAsync<T> value)
	Seq <T> (TryOptionAsync<T> value)
	Seq <A> (Tuple<A> tup)
	Seq <A> (Tuple<A, A> tup)
	Seq <A> (Tuple<A, A, A> tup)
	Seq <A> (Tuple<A, A, A, A> tup)
	Seq <A> (Tuple<A, A, A, A, A> tup)
	Seq <A> (Tuple<A, A, A, A, A, A> tup)
	Seq <A> (Tuple<A, A, A, A, A, A, A> tup)
	Seq <A> (ValueTuple<A> tup)
	Seq <A> (ValueTuple<A, A> tup)
	Seq <A> (ValueTuple<A, A, A> tup)
	Seq <A> (ValueTuple<A, A, A, A> tup)
	Seq <A> (ValueTuple<A, A, A, A, A> tup)
	Seq <A> (ValueTuple<A, A, A, A, A, A> tup)
	Seq <A> (ValueTuple<A, A, A, A, A, A, A> tup)

When Seq<A> the type was added, I needed constructor functions, which are these:

    	Seq <A> ()
	Seq1 <A> (A value)
	Seq <A> (A a, A b)
	Seq <A> (A a, A b, A c, params A[] ds)

The singleton constructor needed to be called Seq1 because it was clashing with the original single-argument Seq functions from v1 of language-ext.

This has been bothering me for a long time. So, it's time to take the hit.

All of the legacy coercing functions (the first list) are now renamed to toSeq. And Seq1 has been renamed to Seq (well, Seq1 still exists, but it's marked [Obsolete].

The breaking change is that if you use any of those legacy Seq functions, you'll either need to change, this:

    Seq(blah);

Into,

    toSeq(blah);

Or,

    blah.ToSeq();

There were quite a lot of the coercion functions used in language-ext, and it took me about 10 minutes to fix it up. The good thing is that they all turn into compilation errors, no hidden behaviour changes.

Apologies for any inconvenience with this. I figure that as we've switched into v4 that this would be a good time for a clean break, and to tidy up something that I have actively seen cause confusion for devs. Seq is the most performance immutable list in the .NET sphere, so it shouldn't really be confusing to use!

Language-Ext Version 4.0.0 (finally out of beta!)

Here it is, version 4.0 of language-ext - we're finally out of beta. I had hoped to get some more documentation done before releasing, but I have ever decreasing free time these days, and I didn't want to delay the release any further. A lot has happened since going into beta about a year or so ago:

Effects

How to handle side-effects with pure functional code has been a the perennial question. It is always coming up in the issues and it's time for this library to be opinionated on that. Language-ext already has Try and TryAsync which are partial solutions to this problem, but they only really talk to the exception catching part of IO. Not, "How do we inject mocked behaviours?", "How do we deal with configuration?", "How do we describe the effects that are allowed?", "How do we make async and sync code play well together?" etc.

Language-Ext 4.0 has added four new monadic types:

• Eff<A> - for synchronous side-effects. This is a natural replacement for Try<A>
• Aff<A> - for asynchronous side-effects. This is a natural replacement for TryAsync<A>
• Eff<RT, A> - for synchronous side-effects with an injectable runtime. This is like the Reader<Env, A> monad on steroids!
• Aff<RT, A> - for asynchronous side-effects with an injectable runtime. This is like the ReaderAsync<Env, A> monad that never existed!

These four monads have been inspired by the more bespoke monads I'd been building personally to deal with IO in my private projects, as well as the Aff and Eff monads from PureScript, and also a nod to ZIO from the Scala eco-system. Together they form the Effects System of language-ext, and will be the key area of investment going forward.

These four monads all play well together, they can be used in the same LINQ expression and everything just works.

There is now a whole section of the wiki dedicated to dealing with side-effects from first principles. I ran out of steam to complete it, but I hope to get back to it soon. It does go into depth on the Aff and Eff monads, and how to build runtimes.

On top of that, there's a project called EffectsExamples in the Samples folder. It is a whole application, with an effects driven menu system, for showing what's possible.

Finally, there's a new library called LanguageExt.Sys that wraps up the core side-effect producing methods in .NET and turns them into Aff or Eff monads. So far it covers:

• Time
• Environment
• Encoding
• Console
• IO.Directory
• IO.File
• IO.Stream
• IO.TextRead

This library will grow over time to cover more of .NET. You are not limited to using these: the runtime system is fully extensible, and allows you to add your own side-effecting behaviors.

Pipes

Pipes is a C# interpretation of the amazing Haskell Pipes library by Gabriella Gonzalez. It is a streaming library that has one type: Proxy. This one type can represent:

• Producers
• Pipes
• Consumers
• Client
• Server

Each Proxy variant can be composed with the others, and when you've got it correct they all fuse into an Effect that can be run. For example, a Producer composed with a Pipe, and then composed with a Consumer will fuse into a runnable effect. Each component is standalone, which means it's possible to build reusable effects 'components'

It also supports either uni or bi-directional streaming and can work seamlessly with the effects-system.

It's actually been in language-ext for a long time, but it was borderline unusable because of the amount of generic arguments that were needed to make it work. Since then I've augmented it with additional monads and 100s of SelectMany overrides that make it easier to use. I have also removed all recursion from it, which was a problem before.

The original Haskell version is a monad transformer, meaning that it can wrap other monads. Usually, you'd wrap up the IO monad, but it wasn't limited to that. However, the C# version can't be as general as that, so it only wraps the Aff and Eff monads. However, I have managed to augment the original to also support:

• IEnumerable
• IObservable
• IAsyncEnumerable

This means we can handle asynchronous and synchronous streams. All of which play nice with the effects system. Here's a small example:

    public class TextFileChunkStreamExample<RT> where RT : 
        struct, 
        HasCancel<RT>,
        HasConsole<RT>,
        HasFile<RT>,
        HasTextRead<RT>
    {
        public static Aff<RT, Unit> main =>
            from _ in Console<RT>.writeLine("Please type in a path to a text file and press enter")
            from p in Console<RT>.readLine
            from e in mainEffect(p)
            select unit;
        
        static Effect<RT, Unit> mainEffect(string path) =>
            File<RT>.openRead(path) 
               | Stream<RT>.read(80) 
               | decodeUtf8 
               | writeLine;

        static Pipe<RT, SeqLoan<byte>, string, Unit> decodeUtf8 =>
            from c in awaiting<SeqLoan<byte>>()         
            from _ in yield(Encoding.UTF8.GetString(c.ToReadOnlySpan()))
            select unit;
        
        static Consumer<RT, string, Unit> writeLine =>
            from l in awaiting<string>()
            from _ in Console<RT>.writeLine(l)
            select unit;
    }

• This example asks the user to enter a filename, then reads the file in 80 byte chunks, converts those chunks into strings, and then writes them to the console
• mainEffect is where the Effect is fused by composing the File.openRead producer, the Stream.read pipe, the decodeUtf8 pipe, and the writeLine consumer.
• You may notice the new type SeqLoan. This is a version of Seq that has a borrowed backing array from an ArrayPool. The Pipes system automatically cleans up resources for you. And so, this stream doesn't thrash the GC

NOTE: The client/server aspect of this still has the generics problem, and is difficult to use. I will be doing the same work I did for Producers, Pipes, and Consumers to make that more elegant over time.

Concurrency

Although this library is all about immutability and pure functions, it is impossible (at least in C#) to get away from needing to mutate something. Usually it's a shared cache of some sort. And that's what the concurrency part of language-ext is all about, dealing with that messy stuff. We already had Atom for lock-free atomic mutation of values and Ref for use with the STM system.

In this release there have been some major improvements to Atom and Ref (no more deadlocks). As well as the addition of three new lock-free atomic collections:

• AtomHashMap
• AtomSeq
• VersionHashMap

And two other data-structures:

• VersionVector
• VectorClock

All but VectorClock, VersionVector, and VersionHashMap were detailed in previous release notes, so I won't cover those again. VectorClock, VersionVector, and VersionHashMap allow you to easily work with distributed versioned changes. Vector-clocks comes from the genius work of Leslie Lamport amongst others, and allow for tracking of causal or conflicting events; when combined with some data in VersionVector it's possible to resolve many clients wanting to change the same piece of data, knowing who did what first. When VersionVector is combined with an atomic-hash-map, you get VersionHashMap. This manages multiple keys of VersionVectors, making it a short hop to a distributed database.

There's some good blog posts on the Riak site that explain vector clocks.

when and unless

These two functions make working with alternative values easier in LINQ expressions.

    from txt1 in File<RT>.readAllText(path)
    from txt2 in when(txt1 == "", SuccessEff("There wasn't any text in the file, so use this instead!"))
    select txt;

The dual of when is unless, it simply reverses the predicate.

guard and guardnot

These are similar to when and unless in that they take a predicate, but instead they return the monad's alternative value (Left for Either, Error for Fin, and Aff, for example). This makes it easy to catch problems and generate contextual error messages.

Pretty

Another Haskell library I've taken inspiration from is Pretty. It's synopsis is:

Pretty is a pretty-printing library, a set of API's that provides a way to easily print out text in a consistent format of your choosing. This is useful for compilers and related tools.
It is based on the pretty-printer outlined in the paper 'The Design of a Pretty-printing Library' by John Hughes in Advanced Functional Programming, 1995

It's a fully composable text document building system that can understand layouts and can do lovely reformatting functions depending on context (like the desired width of text, the tabs, etc.). If you've ever had to manage printing of text with indented tabs, this is the feature for you.

It's pretty new in the library, I needed it for some compiler work I was working on, and it seemed generally useful - so I've added it. However, there's no documentation [other tha...

Concurrency improvements (AtomHashMap, Atom, Ref)

This release features improvements to the concurrency elements of language-ext:

• Atom - atomic references
• STM / Ref - software-transactional memory system
• AtomHashMap - new atomic data-structure

Atom and STM

The atomic references system (Atom) that wraps a value and allows for atomic updates; and the STM system that allows many Ref values (that also wrap a value, but instead work within an atomic sync transaction), have both been updated to never give up trying to resolve a conflict.

Previously they would fail after 500 attempts, but now any conflicts will cause the conflicting threads to back-off and eventually yield control so that the other thread(s) they were in conflict with eventually win and can update the atomic reference. This removes the potential time-bomb buried deep within the atomic references system, and creates a robust transactional system.

AtomHashMap<K, V>

One pattern I noted I was doing quite a lot was wrapping HashMap in an atom, usually for shared cached values:

   var atomData = Atom(HashMap<string, int>());

   atomData.Swap(static hm => hm.AddOrUpdate("foo", 123));

It struck me that it would be very useful to have atomic versions of all of the collection data-structures of language-ext. The first one is AtomHashMap. And so, instead of the code above, we can now write:

    var atomData = AtomHashMap<string, int>();

    atomData.AddOrUpdate("foo", 123);

All operations on AtomHashMap are atomic and lock-free. The underling data structure is still an immutable HashMap. It's simply the reference to the HashMap that gets protected by AtomHashMap, preventing two threads updating the data structure with stale data.

The main thing to understand with AtomHashMap is that if a conflict occurs on update, then any transformational operation is re-run with the new state of the data-structure. Obviously conflicts are rare on high-performance CPUs, and so we save processing time from not taking locks on every operation, at the expense of occasional re-running of operations when conflicts arise.

Swap

AtomHashMap also supports Swap, which allows for more complex atomic operations on the underlying HashMap. For example, if your update operation relies on data within the AtomHashMap, then you might want to consider wrapping everything within a Swap call to allow for fully idempotent transformations:

    atomData.Swap(data => data.Find("foo").Case switch
                          {
                              int x => data.SetItem("foo", x + 1),
                              _     => data.Add("foo", 1)
                          });

NOTE: The longer you spend inside a Swap function, the higher the risk of conflicts, and so try to make sure you do the bare minimum within swap that will facilitate your idempotent operation.

In Place Operations

Most operations on AtomHashMap are in-place, i.e. they update the underlying HashMap atomically. However, some functions like Map, Filter, Select, Where are expected to process the data-structure into a new data-structure. This is usually wanted, but we also want in-place filtering and mapping:

    // Only keeps items with a value > 10 in the AtomHashMap
    atomData.FilterInPlace(x => x > 10);

    // Maps all items in the AtomHashMap 
    atomData.MapInPlace(x => x + 10);

The standard Map, Filter, etc. all still exist and work in the 'classic' way of generating a new data structure.

ToHashMap

At any point if you need to take a snapshot of what's in the AtomHashMap you can all:

    HashMap<string, int> snapshot = atomData.ToHashMap();

This is a zero allocation, zero time (well in the order of nanoseconds), operation. And so we can easily take snapshots and work on those whilst the atomic data structure can carry on being mutated without consequence.

The Rest

As well as AtomHashMap<K, V> there's also AtomHashMap<EqK, K, V> which maps to HashMap<EqK, K, V>.

This is just the beginning of the new Atom based data-structures. So watch this space!

Code-gen improvements - discriminated unions

This release features updates to the [Union] feature of language-ext (discriminated union generator) as well as support for iOS Xamarin runtime code-gen

• Generates a Match function for each union
• Generates a Map function for each union.
• Unions ignore ToString and GetHashCode - allowing for your own bespoke implementations
• iOS Xamarin doesn't allow usage of IL, and so for platforms that don't support IL the runtime code-gen (for Record<A>, Range, and other types) now falls back to building implementations with Expression.

Match

Unions can already be pattern-matched using the C# switch statement and switch expression. Now you can use the generated Match function like other built-in types (such as Option, Either, etc.). This match function enforces completeness checking, which the C# switch can't do.

There are two strategies to generating the Match function:

1. If the [Union] type is an interface, then the generated Match function will be an extension-method. This requires your union-type to be a top-level declaration (i.e. not nested within another class). This may be a breaking change for you.

It looks like this:

public static RETURN Match<A, RETURN>(this Maybe<A> ma, Func<Just<A>, RETURN> Just, Func<Nothing<A>, RETURN> Nothing) =>
    ma switch
    {
        Just<A> value => Just(value), 
        Nothing<A> value => Nothing(value), 
        _ => throw new LanguageExt.ValueIsNullException()
    };

2. If the [Union] type is an abstract partial class, then the generated Match function will be an instance-method. This doesn't have the limitation of the interface approach.

Then the Match instance methods look like this:

// For Just<A>
public override RETURN Match<RETURN>(Func<Just<A>, RETURN> Just, Func<Nothing<A>, RETURN> Nothing) => 
    Just(this);

// For Nothing<A>
public override RETURN Match<RETURN>(Func<Just<A>, RETURN> Just, Func<Nothing<A>, RETURN> Nothing) => 
    Nothing(this);

And so, if you still need unions nested within other classes, switch the interface to an abstract class:

This is an example of an interface based union:

    [Union]
    internal interface Maybe<A>
    {
        Maybe<A> Just(A value);
        Maybe<A> Nothing();
    }

This is the equivalent as an abstract class:

    [Union]
    public abstract partial class Maybe<A>
    {
        public abstract Maybe<A> Just(A value);
        public abstract Maybe<A> Nothing();
    }

Map

One thing that is true of all algebraic-data-types (of which category discriminated-unions fall into); is that there is exactly one way to generate a functor Map function (Select in LINQ) it's known as a theorem for free. And so the implementation for Map and Select are now provided by default by the code-gen.

So, for the example below:

    [Union]
    public abstract partial class Maybe<A>
    {
        public abstract Maybe<A> Just(A value);
        public abstract Maybe<A> Nothing();
    }

This code is auto-generated:

public static Maybe<B> Map<A, B>(this Maybe<A> ma, System.Func<A, B> f) =>
    ma switch
    {
        Just<A> _v => new Just<B>(f(_v.Value)), 
        Nothing<B> _v => new Nothing<B>(), 
        _ => throw new System.NotSupportedException()
    };

public static Maybe<B> Select<A, B>(this Maybe<A> ma, System.Func<A, B> f) =>
    ma switch
    {
        Just<A> _v => new Just<B>(f(_v.Value)), 
        Nothing<B> _v => new Nothing<B>(), 
        _ => throw new System.NotSupportedException()
    };

This will currently only support single generic argument unions, it will be expanded later to provide a Map function for each generic argument. The other limitation is that if any of the cases have their own generic arguments, then the Map function won't be generated. I expect this to cover a large number of use-cases though.

Any problems please report in the repo Issues.

Paul

Aff and Eff monad updates (breaking changes)

I've done some refactoring of the Aff and Eff monads as I slowly progress toward a v4.0 release of language-ext:

• RunIO has been renamed to Run
• HasCancel has been rationalised to only use non-Eff and non-Aff properties
  • It was a mistake to use them, I should have just used regular properties and then made access via static functions
  • This has now been done and should make it a bit more obvious how to use when building your own runtimes
• Support for MapAsync on the Eff monads
  • That means we don't need both a Aff and Eff property in the Has* traits, as an Eff can be MapAsyncd into an Aff efficiently
  • So traits now should only return something like Eff<RT, FileIO> FileEff (obviously different types based on your trait)
• The wrappers for the .NET System namespace have now been factored out into a new library LanguageExt.Sys
  • The naming wasn't great, and it was clear it was going to get pretty messy as I wrapped more of the System IO operations
  • Also, there's a chance you wouldn't want to use the implementations I have built, so having the main namespace cluttered with IO traits and types that you might not need was a bit ugly.
• The default 'live' runtime has also been factored out
• A new test runtime has been added - for unit testing
  • A new built-in mocked in-memory file-system - this allows unit tests to work with files without having to physically create them on a disk
  • A new built-in mocked in-memory console - this allows unit tests to mock key-presses in a console, as well as the option the look at the in-memory console buffer to assert correct values being written
  • A time provider - allows the system clock to appear to run from any date, or even be frozen in time
  • An in-memory mocked 'System.Environment'

Finally, I'm starting to document the Aff and Eff usage. This will be fleshed out more over the next few weeks.

All deployed to nu-get now.

Breaking change: Case matching

The Case feature of the collection and union-types has changed, previously it would wrap up the state of the collection or union type into something that could be pattern-matched with C#'s new switch expression. i.e.

var result = option.Case switch
             {
                  SomeCase<A> (var x) => x,
                  NoneCase<A> _       => 0
             }

The case wrappers have now gone, and the raw underlying value is returned:

var result = option.Case switch
             {
                 int x => x,
                 _     => 0
             };

The first form has an allocation overhead, because the case-types, like SomeCase needed allocating each time. The new version has an allocation overhead only for value-types, as they are boxed. The classic way of matching, with Match(Some: x => ..., None: () => ...) also has to allocate the lambdas, so there's a potential saving here by using this form of matching.

This also plays nice with the is expression:

var result = option.Case is string name 
                ? $"Hello, {name}"
                : "Hello stranger";

There are a couple of downsides, but but I think they're worth it:

• object is the top-type for all types in C#, so you won't get compiler errors if you match with something completely incompatible with the bound value
• For types like Either you lose the discriminator of Left and Right, and so if both cases are the same type, it's impossible to discriminate. If you need this, then the classic Match method should be used.

Collection types all have 3 case states:

• Empty - will return null
• Count == 1 will return A
• Count > 1 will return (A Head, Seq<A> Tail)

For example:

static int Sum(Seq<int> values) =>
    values.Case switch
    {
        null                 => 0,
        int x                => x,
        (int x, Seq<int> xs) => x + Sum(xs),
    };

NOTE: The tail of all collection types becomes Seq<A>, this is because Seq is much more efficient at walking collections, and so all collection types are wrapped in a lazy-Seq. Without this, the tail would be rebuilt (reallocated) on every match; for recursive functions like the one above, that would be very expensive.

Massive improvements to Traverse and Sequence

An ongoing thorn in my side has been the behaviour of Traverse and Sequence for certain pairs of monadic types (when nested). These issues document some of the problems:

#552
#651

The Traverse and Sequence functions were previously auto-generated by a T4 template, because for 25 monads that's 25 * 25 * 2 = 1250 functions to write. In practice it's a bit less than that, because not all nested monads should have a Traverse and Sequence function, but it is in the many hundreds of functions.

Because the same issue kept popping up I decided to bite the bullet and write them all by hand. This has a number of benefits:

• The odd rules of various monads when paired can have bespoke code that makes more sense than any auto-generated T4 template could ever build. This fixes the bugs that keep being reported and removes the surprising nature of Traverse and Sequence working most of the time, but not in all cases.
• I'm able to hand-optimise each function based on what's most efficient for the monad pairing. This is especially powerful for working with Traverse and Sequence on list/sequence types. The generic T4 code-gen had to create singleton sequences and the concat them, which was super inefficient and could cause stack overflows. Often now I can pre-allocate an array and use a much faster imperative implementation with sequential memory access. Where possible I've tried to avoid nesting lambdas, again in the quest for performance but also to reduce the amount of GC objects created. I expect a major performance boost from these changes.
• The lazy stream types Seq and IEnumerable when paired with async types like Task, OptionAsync, etc. can now have bespoke behaviour to better handle the concurrency requirements (These types now have TraverseSerial and SequenceSerial which process tasks in a sequence one-at-a-time, and TraverseParallel and SequenceParallel which processes tasks in a sequence concurrently with a window of running tasks - that means it's possible to stop the Traverse or Sequence operation from thrashing the scheduler.

Help

Those are all lovely things, but the problem with writing several hundred functions manually is that there's gonna be bugs in there, especially as I've implemented them in the most imperative way I can to get the max performance out of them.

I have just spent the past three days writing these functions, and frankly, it was pretty soul destroying experience - the idea of writing several thousand unit tests fills me with dread; and so if any of you lovely people would like to jump in and help build some unit tests then I would be eternally grateful.

Sharing the load on this one would make sense. If you've never contributed to an open-source project before then this is a really good place to start!

I have...

• Released the updates in 3.4.14-beta - so if you have unit tests that use Traverse and Sequence then any feedback on the stability of your tests would be really helpful.
• Created a github project for managing the cards of each file that needs unit tests. It's the first time using this, so not sure of its capabilities yet, but it would be great to assign a card to someone so work doesn't end up being doubled up.
• The code is in the hand-written-traverse branch.
• The folder with all the functions is transformers/traverse

Things to know

• Traverse and Sequence take a nested monadic type of the form MonadOuter<MonadInner<A>> and flips it so the result is MonadInner<MonadOuter<A>>
• If the outer-type is in a fail state then usually the inner value's fail state is returned. i.e. Try<Option<A>> would return Option<Try<A>>.None if the outer Try was in a Fail state.
• If the inner-type is in a fail state then usually that short-cuts any operation. For example Seq<Option<A>> would return an Option<Seq<A>>.None if any of the Options in the Seq were None.
• Where possible I've tried to rescue a fail value where the old system returned Bottom. For example: Either<Error, Try<A>>. The new system now knows that the language-ext Error type contains an Exception and can therefore be used when constructing Try<Either<Error, A>>
• All async pairings are eagerly consumed, even when using Seq or IEnumerable. Seq and IEnumerable do have windows for throttling the consumption though.
• Option combined with other types that have an error value (like Option<Try<A>>, Option<Either<L, R>>, etc.) will put None into the resulting type (Try<Option<A>>(None), Either<L, Option<A>>(None) if the outer type is None - this is because there is no error value to construct an Exception or L value - and so the only option is to either return Bottom or a success value with None in it, which I think is slightly more useful. This behaviour is different from the old system. This decision is up for debate, and I'm happy to have it - the choices are: remove the pairing altogether (so there is no Traverse or Sequence for those types) or return None as described above

Obviously, it helps if you understand this code, what it does and how it should work. I'll make some initial tests over the next few days as guidance.

Discussion will be here

Free monads come to C#

Free monads allow the programmer to take a functor and turn it into a monad for free.

The [Free] code-gen attribute provides this functionality in C#.

Below, is a the classic example of a Maybe type (also known as Option, here we're using the Haskell naming parlance to avoid confusion with the language-ext type).

[Free]
public interface Maybe<A>
{
    [Pure] A Just(A value);
    [Pure] A Nothing();

    public static Maybe<B> Map<B>(Maybe<A> ma, Func<A, B> f) => ma switch
    {
        Just<A>(var x) => Maybe.Just(f(x)),
        _              => Maybe.Nothing<B>()
    };
}

Click here to see the generated code

The Maybe<A> type can then be used as a monad:

var ma = Maybe.Just(10);
var mb = Maybe.Just(20);
var mn = Maybe.Nothing<int>();

var r1 = from a in ma
         from b in mb
         select a + b;  // Just(30)

var r2 = from a in ma
         from b in mb
         from _ in mn
         select a + b;  // Nothing

And so, in 11 lines of code, we have created a Maybe monad that captures the short-cutting behaviour of Nothing.

But, actually, it's possible to do this in fewer lines of code:

[Free]
public interface Maybe<A>
{
    [Pure] A Just(A value);
    [Pure] A Nothing();
}

If you don't need to capture bespoke rules in the Map function, the code-gen will build it for you.

A monad, a functor, and a discriminated union in 6 lines of code. Nice.

As with the discriminated-unions, [Free] types allow for deconstructing the values when pattern-maching:

var txt = ma switch
{
    Just<int> (var x) => $"Value is {x}",
    _                 => "No value"
};

The type 'behind' a free monad (in Haskell or Scala for example) usually has one of two cases:

• Pure
• Free

Pure is what we've used so far, and that's why Just and Nothing had the Pure attribute before them:

[Pure] A Just(A value);
[Pure] A Nothing();

They can be considered terminal values. i.e. just raw data, nothing else. The code generated works in exactly the same way as the common types in language-ext, like Option, Either, etc. However, if the [Pure] attribute is left off the method-declaration then we gain an extra field in the generated case type: Next.

Next is a Func<*, M<A>> - the * will be the return type of the method-declaration.

For example:

[Free]
public interface FreeIO<T>
{
    [Pure] T Pure(T value);
    [Pure] T Fail(Error error);
    string ReadAllText(string path);
    Unit WriteAllText(string path, string text);
}

Click here to see the generated code

If we look at the generated code for the ReadAllText case (which doesn't have a [Pure] attribute), then we see that the return type of string has now been injected into this additional Next function which is provided as the last argument.

    public sealed class ReadAllText<T> : FreeIO<T>, System.IEquata...
    {
        public readonly string Path;
        public readonly System.Func<string, FreeIO<T>> Next;
        public ReadAllText(string Path, System.Func<string, FreeIO<T>> Next)
        {
            this.Path = Path;
            this.Next = Next;
        }

Why is all this important? Well, it allows for actions to be chained together into a continuations style structure. This is useful for building a sequence of actions, very handy for building DSLs.

var dsl = new ReadAllText<Unit>("I:\\temp\\test.txt", 
              txt => new WriteAllText<Unit>("I:\\temp\\test2.txt", txt, 
                  _ => new Pure<Unit>(unit)));

You should be able to see now why the [Pure] types are terminal values. They are used at the end of the chain of continuations to signify a result.

But that's all quite ugly, so we can leverage the monadic aspect of the type:

var dsl = from t in FreeIO.ReadAllText("I:\\temp\\test.txt")
          from _ in FreeIO.WriteAllText("I:\\temp\\test2.txt", t)
          select unit;

The continuation itself doesn't do anything, it's just a pure data-structure representing the actions of the DSL. And so, we need an interpreter to run it (which you write). This is a simple example:

public static Either<Error, A> Interpret<A>(FreeIO<A> ma) => ma switch
{
    Pure<A> (var value)                            => value,
    Fail<A> (var error)                            => error,  
    ReadAllText<A> (var path, var next)            => Interpret(next(Read(path))),
    WriteAllText<A> (var path, var text, var next) => Interpret(next(Write(path, text))),
};

static string Read(string path) => 
    File.ReadAllText(path);

static Unit Write(string path, string text)
{
    File.WriteAllText(path, text);
    return unit;
}

We can then run it by passing it the FreeIO<A> value:

var result = Interpret(dsl);

Notice how the result type of the interpreter is Either. We can use any result type we like, for example we could make the interpreter asynchronous:

public static async Task<A> InterpretAsync<A>(FreeIO<A> ma) => ma switch
{
    Pure<A> (var value)                            => value,
    Fail<A> (var error)                            => await Task.FromException<A>(error),  
    ReadAllText<A> (var path, var next)            => await InterpretAsync(next(await File.ReadAllTextAsync(path))),
    WriteAllText<A> (var path, var text, var next) => await InterpretAsync(next(await File.WriteAllTextAsync(path, text).ToUnit())),
};

Which can be run in a similar way, but asynchronously:

var res = await InterpretAsync(dsl);

And so, the implementation of the interpreter is up to you. It can also take extra arguments so that state can be carried through the operations. In fact it's very easy to use the interpreter to bury all the messy stuff of your application (the IO, maybe some ugly state management, etc.) in one place. This then allows the code itself (that works with the free-monad) to be referentialy transparent.

Another trick is to create a mock interpreter for unit-testing code that uses IO without having to ever do real IO. The logic gets tested, which is what is often the most important aspect of unit testing, but not real IO occurs. The arguments to the interpreter can be the mocked state.

Some caveats though:

• The recursive nature of the interpreter means large operations could blow the stack. This can be dealt with using a functional co-routines/trampolining trick, but that's beyond the scope of this doc.
• Although it's the perfect abstraction for IO, it does come with some additional performance costs. Generating the DSL before interpreting it is obviously not as efficient as directly calling the IO functions.

Caveats aside, the free-monad allows for complete abstraction from side-effects, and makes all operations pure. This is incredibly powerful.

Full code-gen documentation

Rollback to netstandard2.0 and CodeGeneration.Roslyn 0.6.1

Unfortunately, the previous release with the latest CodeGeneration.Roslyn build caused problems due to possible bugs in the CodeGeneration.Roslyn plugin system. These issues only manifested in the nuget package version of the LanguageExt.CodeGen and not in my project-to-project tests, giving a false sense of security.

After a lot of head-scratching, and attempts at making it work, it seems right to roll it back.

This also means rolling back to netstandard2.0 so that the old code-gen can work. And so, I have had to also remove the support for IAsyncEnumerable with OptionAsync and EitherAsync until this is resolved.

Apologies for anyone who wasted time on the last release and who might be inconvenienced by the removal of IAsyncEnumerable support. I tried so many different approaches and none seemed to be working.

Issues resolved:

• HEADS UP: New code-gen issue
• Set Equals

Improvements:

• Performance improvements for Map and Lst
• Performance improvements for all hashing of collections

Any further issues, please feel free to shout on the issues page or gitter.

Migrate to `net461` and `netstandard2.1`

NOTE: I am just investigating some issues with this release relating to the code-gen, keep an eye out for 3.4.3 tonight or tomorrow (12/Feb/2020)

In an effort to slowly get language-ext to the point where .NET Core 3 can be fully supported (with all of the benefits of new C# functionality) I have taken some baby steps towards that world:

Updated the references for CodeGeneration.Roslyn to 0.7.5-alpha

This might seem crazy, but the CodeGeneration.Roslyn DLL doesn't end up in your final build (if you set it up correctly), and doesn't get used live even if you do. So, if the code generates correctly at build-time, it works. Therefore, including an alpha is low risk.

I have been testing this with my TestBed and unit-tests and working with the CodeGeneration.Roslyn team and the alpha seems stable.

A release of CodeGeneration.Roslyn is apparently imminent, so, if you're not happy with this, then please wait for subsequent releases of language-ext when I've upgraded to the full CodeGeneration.Roslyn release. I just couldn't justify the code-gen holding back the development of the rest of language-ext any more.

Updated the minimum .NET Framework and .NET Standard versions

Ecosystem	Old	New
.NET Framework	net46	net461
.NET Standard	netstandard2.0	netstandard2.1

OptionAsync<A> and EitherAsync<A> support IAsyncEnumerable<A>

The netstandard2.1 release supports IAsyncEnumerable<A> for OptionAsync<A> and EitherAsync<A>. This is the first baby-step towards leveraging some of the newer features of C# and .NET Core.

pipe prelude function

Allow composition of single argument functions which are then applied to the initial argument.

var split   = fun((string s) => s.Split(' '));
var reverse = fun((string[] words) => words.Rev().ToArray());
var join    = fun((string[] words) => string.Join(" ", words));

var r = pipe("April is the cruellest month", split, reverse, join); //"month cruellest this is April"

Added Hashable<A> and HashableAsync<A> type-classes

Hashable<A> and HashableAsync<A> provide the methods GetHashCode(A x) and GetHashCodeAsync(A x). There are lots of Hashable*<A> class-instances that provide default implementations for common types.

Updates to the [Record] and [Union] code-gen

The GetHashCode() code-gen now uses Hashable*<A> for default field hashing. Previously this looked for Eq*<A> where the * was the type of the field to hash, now it looks for Hashable*<A>.

By default Equals, CompareTo, and GetHashCode use:

    // * == the type-name of the field/property
    default(Eq*).Equals(x, y);
    default(Ord*).CompareTo(x, y);
    default(Hashable*).GetHashCode(x);

To provide the default structural functionality for the fields/properties. Those can now be overridden with The Eq, Ord, and Hashable attributes:

    [Record]
    public partial struct Person
    {
        [Eq(typeof(EqStringOrdinalIgnoreCase))]
        [Ord(typeof(OrdStringOrdinalIgnoreCase))]
        [Hashable(typeof(HashableStringOrdinalIgnoreCase))]
        public readonly string Forename;

        [Eq(typeof(EqStringOrdinalIgnoreCase))]
        [Ord(typeof(OrdStringOrdinalIgnoreCase))]
        [Hashable(typeof(HashableStringOrdinalIgnoreCase))]
        public readonly string Surname;
    }

The code above will generate a record where the fields Forename and Surname are all structurally part of the equality, ordering, and hashing. However, the case of the strings is ignored, so:

    { Forename: "Paul", Surname: "Louth" } == { Forename: "paul", Surname: "louth" }

NOTE: Generic arguments aren't allowed in attributes, so this technique is limited to concrete-types only. A future system for choosing the structural behaviour of generic fields/properties is yet to be designed/defined.

Bug fixes

1. Fix for: 'Seq.ToFullString' and 'Seq.ToFullArrayString' ignore the 'separator' argument
2. Fix for: Lst<T> serialization broken for web forms pages
3. Fix for: Bug in ToString implementation for unions
4. Fix for: Set is missing RemoveRange