Merge `Effect` and `ST`

[garyb]

This is a idea for the future, floated by @natefaubion when we were talking about #30.

ST and Effect have the same implementation, and it should be possible to coerce safely in either direction (at first I thought runST would essentially become unsafePerformEffect when applied to an Effect converted to ST Global, but actually that won't type check). Given that, Effect could just be what is currently ST Global.

This would mean we have fewer cases to deal with in the optimizer, no need to implement two sets of essentially identical instances and functions, and avoid the need to explicitly coerce or lift ST things to work in Effect (useful for STArray, etc.).

With this unified thing, the somewhat obvious name for the type is Effect, but we'd probably want to preserve the Type -> Type version of Effect we currently have so we don't have to perform an Eff-to-Effect-worthy migration again, so we'd need to come up with something else.

Nate also suggested that we'd want to wait until synonyms are preserved in error messages, since we don't want to reveal the underlying type for Effect to newbies unless we need to, which I agree with.

[hdgarrood]

I dislike this for more or less the same reason that I dislike #30, although this one bothers me quite a bit more. My objection to this is that I don’t think it makes a lot of sense unless you’re intimately familiar with how these things are all implemented. Mutable variables are just one of the many things you can do with Effect - why privilege that specific thing over all the others? It seems plausible that there is at least one other effectful concept which makes sense to model in a similar way to ST, but if we implement this, Effect will already be tightly welded to ST (and exclusively ST) so that won’t be an option. Also, what will a beginner who finds type Effect = ST Global in Pursuit think? I think that’s likely to be very misleading, ie imply that Effect is only about mutable variables specifically, rather than as the general type for all synchronous effectful actions.

For me, an API surface that makes sense should really come before implementation concerns; I don’t think this proposal follows that principle.

[natefaubion]

It's not really about mutation per se. It's more that it's tracking the token for observable effects. If IO/Effect is pseudo-semantically RealWorld token passing, then this is parameterizing the effect by the world token for which you can observe the effects. If we have a rank-2 effect here then it's effects can't be observed and it can safely be eliminated. I understand it's rehashing the old Eff a bit, but it's far more limited in scope.

I think there's quite a bit of value in sharing as much as possible and I think there's something semantically that unites them. For example you can't use EffectFns in ST even though it's the one place that would benefit the most from it, and I'm not confident a proposal would be accepted to introduce them (due to complexity concerns or proliferation of types).

[hdgarrood]

Can you give an example of where you might want to make use of this idea outside of what's currently covered by ST? I worry that if we do this then we will spend the next year or so having to explain to people what Effect :: Region -> Type -> Type means, and why they need to replace all of their uses of Effect with Effect Global. I barely ever use ST anyway, and I just can't see this functionality being used anywhere near enough to justify reconfiguring one of the most basic types in the whole ecosystem. To me, a separate set of STFn types and optimizations for them is significantly more appealing.

[garyb]

Not that this was part of the original goal, but actually it could be Eff :: k -> Type -> Type, as then it can work like the original Eff, or be used as type Effect = Eff Global or be type ST r = Eff (r :: Region) 😄.

The point of this is to be able to freely mix Effect and ST code (like, use STArray for a mutable buffer in general, not stuck in a private region). That is possible with liftST, but it would happen "for free" this way.

It seems to me that the cons are:

1. It might be hard to explain to newcomers if they see the underlying Eff (or whatever) type.

vs pros:

1. No need to duplicate FFI implementations for Effect and ST, the looping operations, and the related things like Ref
2. No need to duplicate optimisation code in the compiler
3. Make use of ST-things in Effect without casting/coercion
4. People could use effect rows and such again if they really wanted to

The con seems like even less of a serious concern given one of the conditions of implementing this is that synonyms would appear in error messages, so the only place to see type Effect = Eff Global would be when tracking down the implementation or something.

[hdgarrood]

The point of this is to be able to freely mix Effect and ST code (like, use STArray for a mutable buffer in general, not stuck in a private region). That is possible with liftST, but it would happen "for free" this way.

But surely it won't be possible to do this, or indeed to reap any of the userland benefits of this proposal, without exposing the guts of how Effect and Eff relate to each other? This is why I still think the "confusing API" issue is a serious concern.

I think I might be able to get on board with this if we were to define

data Eff :: k -> Type -> Type
newtype Effect a = Effect (Eff Global a)

and set things up so that the Effect constructor isn't normally visible, i.e. not exported by the Effect module. The optimisation code in the compiler might need to be made smarter so that it works for newtypes too, but that's probably something which would be useful to have anyway. Now that we have Coercible I think having people use coerce to go between Effect and ST isn't too much of a burden.

[garyb]

I guess I just don't see what would be confusing about it, it doesn't seem much different than the likes of type State = StateT Identity, etc. to me.

[hdgarrood]

Ah right yeah, that is a good point.

[hdgarrood]

I guess we ought to make a decision here. I'm still reluctant; I think my problem with this is really that Effect is used quite heavily in basically every single PureScript app, while in my mind ST is, to be frank, mostly a curiosity and an example for showcasing rank-n types. Given this, I don't think the design of Effect should be influenced by what would be handy for users of ST; I don't think we should complicate the API for using mutable variables in Effect so that the same functions can be used in ST without coercions, even if it is in the relatively minor way being proposed here. Just because Effect is so much more important than ST is.

Having type Effect = Effect' Global might not be so bad, but when you are dealing with things which are supposed to work in both Effect and ST, the API necessarily becomes a little more complicated; Ref needs to get an extra type parameter, and the type of Ref.read becomes forall r a. Ref r a -> Effect' r a; at that point, the reader probably does need to get their head around the fact that Effect is now defined as type Effect = Effect' Global.

As for the other pros you mentioned:

• No need to duplicate FFI implementations for Effect and ST, the looping operations, and the related things like Ref
• No need to duplicate optimisation code in the compiler

For me, having these things influence the API design is wrong. These are implementation concerns which I still don't think should be a factor here. I'm sure there are other ways of removing this duplication (e.g. making ST a newtype around Effect and making the compiler optimizations appropriately smart).

• People could use effect rows and such again if they really wanted to

This is arguably a con rather than a pro, I think 😛

[ajnsit]

I just want to say that the migration from Eff to Effect was really painful, and we shouldn't have to go through anything like that again without good reason.

For me, having these things influence the API design is wrong. These are implementation concerns which I still don't think should be a factor here.

Totally agreed.

[garyb]

I listed those pros as like, "bonus things we get if we do this", not reasons to do it, the implementations are already identical.

I like it for the API as conceptually I think it makes sense too - everything you can do in ST is reasonable to do in Effect also. It's just whether it's allowed to happen locally or globally.

[JordanMartinez]

For what it's worth, I think explaining Effect to people should be handled via documentation. People are already choosing a niche language that already has a learning curve. Why should "but learners will get confused" be a reason to not implement something that might otherwise be useful? Isn't this problem better solved by clear tutorials and documentation?

People could use effect rows and such again if they really wanted to

We've already got run for people who want to have that. So, this argument doesn't appeal to me. However, I get that those who want that can now have it again.

making ST a newtype around Effect and making the compiler optimizations appropriately smart

This sounds like a better tradeoff for deduplicating code, merging ST and Effect, and ensuring Effect is still easily understandable by new learners. @garyb Would this approach be a better way to do that?

For example you can't use EffectFns in ST even though it's the one place that would benefit the most from it, and I'm not confident a proposal would be accepted to introduce them (due to complexity concerns or proliferation of types).

Based on Nate's comment above, perhaps this is why making ST a newtype around Effect wouldn't work?

Make use of ST-things in Effect without casting/coercion

If we did not implement this change and used Coercible, what would be the pros/cons of that?

[JordanMartinez]

I like it for the API as conceptually I think it makes sense too - everything you can do in ST is reasonable to do in Effect also. It's just whether it's allowed to happen locally or globally.

Couldn't we also just add API that enables ST in a global context? Or is this the "make ST a newtype around Effect" idea that Harry proposed?

[hdgarrood]

Why should "but learners will get confused" be a reason to not implement something that might otherwise be useful?

I think there's definitely a tradeoff in how general you make things. When you make things really general, the benefit is that you only need to make sense of the function once and you can apply that knowledge in lots of different places. However, when you make things more targeted towards particular use cases, it means that the APIs require less effort to understand, you're able to write documentation which is easier to make sense of (since you can use more concrete examples without being misleading), and I think the APIs are also less likely to behave in surprising ways if they end up being applied in a setting where they don't really make sense.

In this particular case, my opinion is that the difference between Ref.read :: forall r a. Ref r a -> Effect' r a and Ref.read :: forall a. Ref a -> Effect a is small but nontrivial, and not that easy to justify based on how often I think mutable variables are used in Effect versus in ST.

I don't actually think this issue is specific to learners (although it is likely to affect learners more than other people); I think it's potentially an issue for anyone who uses this API. Ergonomics really do matter; it shouldn't just be a case of saying "approach A allows people to do X in more situations than approach B therefore A is better." I should note that this proposal could improve the ergonomics quite significantly for people who are interested in using mutable variables in both Effect and ST, so I guess my objection here is dependent on my impression that people don't use ST very often, which could be wrong!

This sounds like a better tradeoff for deduplicating code, merging ST and Effect, and ensuring Effect is still easily understandable by new learners. @garyb Would this approach be a better way to do that?

Not necessarily: I think the argument in favour of doing this is (more or less) that if we do what's proposed here, we'll end up with less API surface overall, and we won't run into cases where there's some abstraction which is usable only in Effect or only in ST even though it makes sense for both (such as EffectFns). Right now, you have to explicitly coerce to go between Effect and ST, which means that you have to go out of your way to make something which works in both settings, by doing something like the MutableBuffer class in node-buffers.

If we did not implement this change and used Coercible, what would be the pros/cons of that?

I think using Coercible is basically the same as what I am advocating; the central question is "should there be explicit coercions or not".

Couldn't we also just add API that enables ST in a global context?

We have that already; see #26