Skip to content

Latest commit

 

History

History
868 lines (616 loc) · 23.8 KB

File metadata and controls

868 lines (616 loc) · 23.8 KB

Advanced Topics

Transactions, grain directory, OpenTelemetry, shutdown, state migration, and serialization.

What you'll learn

  • How to use Orleans transactions from F#
  • How to configure the grain directory
  • How to integrate with OpenTelemetry
  • How to perform graceful shutdown
  • How to migrate grain state between versions
  • How to configure F# type serialization

Transactions

The Orleans.FSharp.Transactions module provides F# wrappers for Orleans transactional state.

Transaction options

open Orleans.FSharp.Transactions

// These map to Orleans [Transaction(option)] attributes in CodeGen
type TransactionOption =
    | Create           // Always creates a new transaction
    | Join             // Must run within an existing transaction
    | CreateOrJoin     // Joins if exists, creates otherwise
    | Supported        // Not transactional but can be called within one
    | NotAllowed       // Cannot be called within a transaction
    | Suppress         // Suppresses any ambient transaction

Reading transactional state

let! currentBalance = TransactionalState.read accountState

Updating transactional state

do! TransactionalState.update
    (fun state -> { state with Balance = state.Balance + amount })
    accountState

Performing a read with projection

let! balance =
    TransactionalState.performRead
        (fun state -> state.Balance)
        accountState

Converting options

let orleansOption = TransactionOption.toOrleans CreateOrJoin
// Returns Orleans.TransactionOption.CreateOrJoin

Higher-Level Transactional Grains

Orleans.FSharp.Runtime provides FSharpTransactionalGrain<'State> and FSharpAtmGrain<'TAccountGrain> — generic base classes that wire pure F# functions to Orleans ACID transactions. Compared to using ITransactionalState<'T> directly, they remove all boilerplate: you supply just the business logic as a record of functions.

Why use these classes?

Raw ITransactionalState<'T> FSharpTransactionalGrain<'State>
Write PerformRead / PerformUpdate in every grain method Provide functions once as a TransactionalGrainDefinition
Repeat CopyState logic in every PerformUpdate CopyState is centralised in the definition
Boilerplate C# class per grain One C# stub, one F# definition

Define the state type

The state must be a sealed class with a parameterless constructor (Orleans requirement for ITransactionalState<'T>):

[<GenerateSerializer; Sealed>]
type AccountState() =
    [<Id(0u)>] member val Balance: decimal = 0m with get, set

Define the grain behaviour (F#)

open Orleans.FSharp.Runtime

let accountDef: TransactionalGrainDefinition<AccountState> =
    {
        Deposit  = fun state amount ->
            let s = AccountState()
            s.Balance <- state.Balance + amount
            s

        Withdraw = fun state amount ->
            if state.Balance < amount then
                raise (System.InvalidOperationException("Insufficient funds"))
            let s = AccountState()
            s.Balance <- state.Balance - amount
            s

        GetBalance = fun state -> state.Balance

        CopyState  = fun source target ->
            target.Balance <- source.Balance
    }

Deposit and Withdraw must return a new state object — do not mutate the input. CopyState copies field values from source into target in place (Orleans requires this for transactional semantics).

Define the grain interface (F#)

Use TransactionOption.CreateOrJoin on methods that should work both standalone and inside an ATM transaction:

open Orleans
open Orleans.Transactions
open Orleans.FSharp.Transactions

type IAccountGrain =
    inherit IGrainWithStringKey

    [<Transaction(TransactionOption.CreateOrJoin)>]
    abstract Deposit:    amount: decimal -> Task

    [<Transaction(TransactionOption.CreateOrJoin)>]
    abstract Withdraw:   amount: decimal -> Task

    [<Transaction(TransactionOption.CreateOrJoin)>]
    abstract GetBalance: unit   -> Task<decimal>

Create the C# stub

Orleans source generators only work with C#. Add one stub per concrete grain:

using Orleans.Transactions.Abstractions;
using Orleans.FSharp.Runtime;
using Orleans.FSharp.Sample;

public sealed class AccountGrainImpl
    : FSharpTransactionalGrain<AccountState>,
      IAccountGrain
{
    public AccountGrainImpl(
        [TransactionalState("state", "TransactionStore")]
        ITransactionalState<AccountState> state,
        TransactionalGrainDefinition<AccountState> definition,
        ILogger<FSharpTransactionalGrain<AccountState>> logger)
        : base(state, definition, logger) { }
}

The "TransactionStore" string must match the storage provider name registered on the silo.

Register the definition and silo providers

open Orleans.FSharp.Runtime

// In your silo configuration function:
siloBuilder
    .UseTransactions()
    .AddMemoryGrainStorage("TransactionStore")  // or real storage in production
|> ignore

// In DI:
services.AddFSharpTransactionalGrain<AccountState>(accountDef) |> ignore

Call the grain

let account = grainFactory.GetGrain<IAccountGrain>("alice")

do! account.Deposit(500m)
do! account.Withdraw(200m)
let! balance = account.GetBalance()
// balance = 300m

ATM Grain (cross-account transfers)

FSharpAtmGrain<'TAccountGrain> orchestrates atomic transfers across multiple account grains in a single Orleans transaction. It wraps two grain calls (Withdraw + Deposit) in a [Transaction(Create)] method.

Define the ATM behaviour

open Orleans.FSharp.Runtime

let atmDef: AtmGrainDefinition<IAccountGrain> =
    {
        Transfer = fun from to' amount ->
            task {
                do! from.Withdraw(amount)
                do! to'.Deposit(amount)
            }
    }

Define the ATM interface

type IAtmGrain =
    inherit IGrainWithStringKey

    [<Transaction(TransactionOption.Create)>]
    abstract Transfer: fromKey: string * toKey: string * amount: decimal -> Task

Create the C# ATM stub

public sealed class AtmGrainImpl
    : FSharpAtmGrain<IAccountGrain>,
      IAtmGrain
{
    public AtmGrainImpl(
        AtmGrainDefinition<IAccountGrain> definition,
        ILogger<FSharpAtmGrain<IAccountGrain>> logger)
        : base(definition, logger) { }
}

Register the ATM definition

services.AddFSharpAtmGrain<IAccountGrain>(atmDef) |> ignore

Execute an atomic transfer

let atm = grainFactory.GetGrain<IAtmGrain>("atm-1")
do! atm.Transfer("alice", "bob", 100m)
// Both the withdraw from alice and deposit to bob commit atomically,
// or both roll back if either throws.

Testing transactional grains

Because TransactionalGrainDefinition is a plain F# record of functions, you can unit-test the business logic without a silo:

open FsCheck.Xunit

[<Property>]
let ``Deposit then Withdraw round-trips the balance`` (initial: decimal) (amount: decimal) =
    let safeInitial = abs initial % 1_000_000m
    let safeAmount  = abs amount  % 1_000_000m + 0.01m
    let state = AccountState()
    state.Balance <- safeInitial
    let afterDeposit  = accountDef.Deposit  state  safeAmount
    let afterWithdraw = accountDef.Withdraw afterDeposit safeAmount
    afterWithdraw.Balance = safeInitial

[<Fact>]
let ``Withdraw throws when balance is insufficient`` () =
    let state = AccountState()
    state.Balance <- 10m
    Assert.Throws<exn>(fun () -> accountDef.Withdraw state 50m |> ignore)
    |> ignore

Grain Directory

The grain directory maps grain identities to their physical silo locations. You can configure it with different backing stores.

open Orleans.FSharp.GrainDirectory

// Default in-memory distributed directory
let configureFn = GrainDirectory.configure Default

// Redis-backed directory
let configureFn = GrainDirectory.configure (Redis redisConnStr)

// Azure Table-backed directory
let configureFn = GrainDirectory.configure (AzureStorage azureConnStr)

// Custom directory
let configureFn = GrainDirectory.configure (Custom myConfigurator)

// Apply to silo builder
configureFn siloBuilder |> ignore

OpenTelemetry

Orleans publishes traces and metrics under well-known activity-source and meter names. Wire them into OpenTelemetry directly with the standard .NET builder — no Orleans.FSharp helper is required.

Activity source and meter names

// Orleans tracing activity sources
"Microsoft.Orleans.Runtime"        // runtime traces
"Microsoft.Orleans.Application"    // application traces

// Orleans metrics meter
"Microsoft.Orleans"

Full OpenTelemetry setup

open OpenTelemetry.Trace
open OpenTelemetry.Metrics

builder.Services
    .AddOpenTelemetry()
    .WithTracing(fun tracing ->
        tracing
            .AddSource("Microsoft.Orleans.Runtime")
            .AddSource("Microsoft.Orleans.Application")
            .AddOtlpExporter()
        |> ignore)
    .WithMetrics(fun metrics ->
        metrics
            .AddMeter("Microsoft.Orleans")
            .AddOtlpExporter()
        |> ignore)
|> ignore

Graceful Shutdown

The Shutdown module provides helpers for clean silo shutdown.

Configure drain timeout

open Orleans.FSharp

Shutdown.configureGracefulShutdown (TimeSpan.FromSeconds 30.) hostBuilder
|> ignore

Stop the host

do! Shutdown.stopHost host

Register a shutdown handler

Shutdown.onShutdown (fun ct ->
    task {
        printfn "Silo is shutting down..."
        // Clean up resources
    }) hostBuilder
|> ignore

Multiple shutdown handlers can be registered; they run in registration order.


State Migration

The StateMigration module enables upgrading grain state schemas across deployments.

Define migrations

open Orleans.FSharp

// Migration from v1 to v2: add a new field with a default value
let v1ToV2 =
    StateMigration.migration<CounterStateV1, CounterStateV2> 1 2
        (fun v1 -> { Count = v1.Count; CreatedAt = DateTime.MinValue })

// Migration from v2 to v3: rename a field
let v2ToV3 =
    StateMigration.migration<CounterStateV2, CounterStateV3> 2 3
        (fun v2 -> { Value = v2.Count; CreatedAt = v2.CreatedAt })

Apply migrations

let migrations = [ v1ToV2; v2ToV3 ]

// Upgrade from v1 to latest (throws if chain is invalid)
let currentState : CounterStateV3 =
    StateMigration.applyMigrations<CounterStateV3> migrations 1 (box oldV1State)

// Safe version — validate and apply in one call, returns Result
match StateMigration.tryApplyMigrations<CounterStateV3> migrations 1 (box oldV1State) with
| Ok newState -> // use newState
| Error errs  -> for e in errs do log.LogError("Migration error: {Error}", e)

Migrations are sorted by FromVersion and applied sequentially. tryApplyMigrations runs validate first; if the chain has gaps or duplicates it returns Error (string list) without touching the state.

Validate migration chain

let errors = StateMigration.validate migrations

match errors with
| [] -> printfn "Migration chain is valid"
| errs -> for e in errs do printfn "Error: %s" e

Validates:

  • No duplicate FromVersion values
  • Contiguous chain (each migration's ToVersion matches the next migration's FromVersion)

Serialization

FSharp.SystemTextJson

Orleans.FSharp ships with pre-configured JSON serialization options for F# types:

open Orleans.FSharp

// Pre-configured options with DU support
let options = Serialization.fsharpJsonOptions

// Or the raw FSharpJson module
let options = FSharpJson.serializerOptions

These options support:

  • Discriminated unions (adjacent tag encoding)
  • Records
  • Options / ValueOptions
  • Lists, Sets, Maps
  • Tuples

Register F# converters

let myOptions = JsonSerializerOptions()
Serialization.addFSharpConverters myOptions |> ignore

Create options with extra converters

let options = Serialization.withConverters [ myCustomConverter ]

Orleans native F# serialization

For native Orleans serializer support (not JSON), use the FSharpSerialization module:

open Orleans.FSharp.FSharpSerialization

FSharpSerialization.addFSharpSerialization siloBuilder |> ignore

Requires Microsoft.Orleans.Serialization.FSharp.


Immutable Values

Use Immutable<'T> for zero-copy grain argument passing when the value will not be modified:

open Orleans.FSharp

let data = immutable [1; 2; 3; 4; 5]
// Pass 'data' to a grain method -- Orleans skips serialization copies

let values = unwrapImmutable data
// values = [1; 2; 3; 4; 5]

Grain State Operations

The GrainState module wraps IPersistentState<'T> for idiomatic F# access:

open Orleans.FSharp

// Read from storage
let! value = GrainState.read persistentState

// Write to storage
do! GrainState.write persistentState newValue

// Clear storage
do! GrainState.clear persistentState

// Get in-memory value (no I/O)
let current = GrainState.current persistentState

Observers

The Observer module manages grain observer lifecycle:

open Orleans.FSharp

// Create a grain object reference for a local observer
let observerRef = Observer.createRef<IMyObserver> grainFactory myObserver

// Delete when done (prevents memory leaks)
Observer.deleteRef<IMyObserver> grainFactory observerRef

// Or use subscribe for automatic cleanup via IDisposable
use subscription = Observer.subscribe<IMyObserver> grainFactory myObserver
// observerRef is automatically deleted when subscription is disposed

Grain Services

Register background services that run on every silo with the addGrainService operation in siloConfig { }:

open Orleans.FSharp.Runtime

let config = siloConfig {
    useLocalhostClustering
    addGrainService typeof<MyBackgroundService>
}

Parallel Grain Calls

Orleans grains are independent actors — many queries can be issued concurrently. Orleans.FSharp provides two complementary approaches.

and! — fixed parallel bindings (F# applicative CE syntax)

For a small, fixed set of concurrent grain calls use the and! keyword inside a task {} expression. All bound tasks start simultaneously and the CE collects their results:

task {
    let! balance1 = account1.GetBalance()
    and! balance2 = account2.GetBalance()
    and! balance3 = account3.GetBalance()
    // All three calls ran in parallel
    return balance1 + balance2 + balance3
}

and! compiles to Task.WhenAll under the hood and is the idiomatic F# way to express parallel fan-out for a known set of grain references.

GrainBatch — dynamic collections of grains

When the number of grains is determined at runtime (e.g., retrieved from a roster or config), use GrainBatch:

open Orleans.FSharp

// Collect balances from N account grains
let keys = ["alice"; "bob"; "carol"; "dave"]
let accounts = keys |> List.map (fun k -> factory.GetGrain<IAccountGrain>(k))

// Fan-out: all calls run concurrently
let! balances = GrainBatch.map accounts (fun a -> a.GetBalance())
// balances: decimal list — same order as accounts

// Aggregate
let! total = GrainBatch.aggregate accounts (fun a -> a.GetBalance()) List.sum

// Fault-tolerant: capture individual failures instead of failing the whole batch
let! results = GrainBatch.tryMap accounts (fun a -> a.GetBalance())
// results: Result<decimal, exn> list

// Partition into successes and failures
let! (ok, failed) = GrainBatch.partition accounts (fun a -> a.GetBalance())
// ok: decimal list, failed: exn list

// Filter: only accounts that opted in to notifications
let! opted = GrainBatch.choose accounts (fun a -> a.GetOptInPreference())
// opted: SomeType list — None results are removed

// Fire-and-forget on all grains
do! GrainBatch.iter accounts (fun a -> a.Deposit(0.01m))

// Fire-and-forget with per-grain error capture
let! statuses = GrainBatch.tryIter accounts (fun a -> a.Deposit(0.01m))
// statuses: Result<unit, exn> list

When to use which

Scenario Recommendation
2–4 specific grain calls and! — cleaner syntax, no list overhead
N grains from a collection GrainBatch.map / GrainBatch.aggregate
Tolerating individual failures GrainBatch.tryMap / GrainBatch.partition
Fire-and-forget all GrainBatch.iter
Fire-and-forget, capture errors GrainBatch.tryIter
Filter grain responses GrainBatch.choose

TaskHelpers

Utility functions for composing Task-based operations with Result:

open Orleans.FSharp

let! result = TaskHelpers.taskResult 42            // Task<Result<int, _>> = Ok 42
let! error = TaskHelpers.taskError "failed"        // Task<Result<_, string>> = Error "failed"

let! mapped =
    TaskHelpers.taskResult 42
    |> TaskHelpers.taskMap (fun n -> n * 2)         // Ok 84

let! bound =
    TaskHelpers.taskResult 42
    |> TaskHelpers.taskBind (fun n ->
        if n > 0 then TaskHelpers.taskResult (n * 2)
        else TaskHelpers.taskError "negative")      // Ok 84

Logging

The Log module provides structured logging with automatic correlation ID propagation:

open Orleans.FSharp

// Structured log messages
Log.logInfo logger "Processing order {OrderId}" [| box orderId |]
Log.logWarning logger "Slow response from {Service}" [| box serviceName |]
Log.logError logger exn "Failed to process {Command}" [| box command |]
Log.logDebug logger "Cache hit for {Key}" [| box cacheKey |]

// Correlation scopes
do! Log.withCorrelation requestId (fun () ->
    task {
        // All logs in this scope include {CorrelationId}
        Log.logInfo logger "Step 1" [||]
        Log.logInfo logger "Step 2" [||]
    })

// Get current correlation ID
let corrId = Log.currentCorrelationId()  // Some "abc-123" or None

Reminders (Module API)

For programmatic reminder management (outside the grain { } CE):

open Orleans.FSharp

let! handle = Reminder.register grain "MyReminder" dueTime period
do! Reminder.unregister grain "MyReminder"
let! existing = Reminder.get grain "MyReminder"  // Some handle or None

Timers (Module API)

For programmatic timer management (outside the grain { } CE):

open Orleans.FSharp

let timer = Timers.register grain callback dueTime period
// Dispose to cancel: timer.Dispose()

let timerWithState = Timers.registerWithState grain callback state dueTime period

Request Context

The RequestCtx module wraps Orleans RequestContext for idiomatic F# access. Values placed in the request context are automatically propagated from callers to callees by the Orleans runtime — useful for correlation IDs, tenant IDs, or any per-call metadata.

Set and get a value

open Orleans.FSharp

// Set before making a grain call
RequestCtx.set "tenantId" (box "acme-corp")

// Inside the callee grain — returns Some "acme-corp"
let tenantId = RequestCtx.get<string> "tenantId"

// Or with a fallback
let tenant = RequestCtx.getOrDefault<string> "tenantId" "unknown"

Remove a value

RequestCtx.remove "tenantId"

Scoped context with withValue

withValue sets a key before running a Task and removes it afterwards — even if the Task throws:

let! result =
    RequestCtx.withValue<OrderState> "correlationId" (box requestId) (fun () ->
        task {
            // All grain calls made here propagate correlationId automatically
            let handle = FSharpGrain.ref<OrderState, OrderCommand> factory "order-42"
            return! handle |> FSharpGrain.send PlaceOrder
        })

API summary

Function Signature Description
RequestCtx.set string -> obj -> unit Set a context value
RequestCtx.get<'T> string -> 'T option Read a typed value (None if missing or wrong type)
RequestCtx.getOrDefault<'T> string -> 'T -> 'T Read with a fallback default
RequestCtx.remove string -> unit Remove a key from the context
RequestCtx.withValue<'T> string -> obj -> (unit -> Task<'T>) -> Task<'T> Scoped set/remove around a Task

Scripting / REPL

The Scripting module starts a local in-process silo from an F# script (.fsx file), letting you iterate on grain logic without a full project setup.

Quick start

#r "nuget: Orleans.FSharp"
#r "nuget: Orleans.FSharp.Runtime"

open Orleans.FSharp
open Orleans.FSharp.Runtime

// Define a grain inline
type PingState = { Count: int }
type PingCmd = Ping | GetCount

let pingGrain =
    grain {
        defaultState { Count = 0 }
        handle (fun state cmd -> task {
            match cmd with
            | Ping     -> return { Count = state.Count + 1 }, box (state.Count + 1)
            | GetCount -> return state, box state.Count
        })
    }

// Start an in-process silo on the given ports (localhost clustering + in-memory storage)
let! silo = Scripting.startOnPorts 11111 30000

// Register and call the grain
silo.Host.Services
    .AddFSharpGrain<PingState, PingCmd>(pingGrain)
    |> ignore

let handle = FSharpGrain.ref<PingState, PingCmd> silo.GrainFactory "ping-1"
let! s1 = handle |> FSharpGrain.send Ping
printfn "Count: %d" s1.Count

// Shut down when done
do! Scripting.shutdown silo

API

Function Description
Scripting.startOnPorts siloPort gatewayPort Start a silo on specific ports (useful when running multiple silos)
Scripting.getGrain<'T> handle key Get a grain reference by int64 key
Scripting.shutdown handle Stop the silo and release resources

The SiloHandle returned by startOnPorts exposes .Host, .Client, and .GrainFactory for direct access when you need lower-level control.

The silo is pre-configured with in-memory storage (Default and PubSubStore), an in-memory stream provider (StreamProvider), and in-memory reminders.


Kubernetes

The Kubernetes module (namespace Orleans.FSharp.Kubernetes) configures silo clustering for Kubernetes deployments. It uses reflection to call the Orleans Kubernetes extension method, so the NuGet package Microsoft.Orleans.Hosting.Kubernetes remains an optional runtime dependency — your silo project only needs it when deployed to Kubernetes.

Standard Kubernetes clustering

open Orleans.FSharp.Kubernetes

// Returns ISiloBuilder -> ISiloBuilder; apply during silo configuration
let configure = Kubernetes.useKubernetesClustering

// Wire it into a siloConfig manually when you need more control
builder.UseOrleans(fun siloBuilder ->
    siloBuilder |> configure |> ignore) |> ignore

Multi-tenant: Kubernetes clustering with a custom namespace

// Uses the Kubernetes namespace as the Orleans ServiceId
let configure = Kubernetes.useKubernetesClusteringWithNamespace "my-k8s-namespace"

builder.UseOrleans(fun siloBuilder ->
    siloBuilder |> configure |> ignore) |> ignore

How it works

Both functions search all loaded assemblies for the UseKubernetesHosting extension method at runtime. If the package is not found, an InvalidOperationException is thrown with a clear message naming the missing NuGet package. This keeps the core library free of a hard assembly dependency on the Kubernetes hosting package.

API

Function Signature Description
Kubernetes.useKubernetesClustering ISiloBuilder -> ISiloBuilder Configure Kubernetes clustering (uses Kubernetes API for silo discovery)
Kubernetes.useKubernetesClusteringWithNamespace string -> ISiloBuilder -> ISiloBuilder Same, but sets ClusterOptions.ServiceId to the given namespace

Next steps