wip 0.1.4 remove arity confusion

CHANGELOG.md

@@ -1,3 +1,7 @@
+## [0.1.4] - Unreleased
+- Remove the arity check, it's not very intuitive
+- Adds a `@context` that gets set at runtime and reset after each run. This is a much simpler way for methods to access a shared runtime context that can be set per `run!`.
+
 ## [0.1.3] - 2024-04-17
 - `Experiment` now enforces arity at runtime for the `#enabled?`, `control`, and `candidate` methods.
 

Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    lab_coat (0.1.3)
+    lab_coat (0.1.4)
 
 GEM
   remote: https://rubygems.org/

README.md

@@ -57,7 +57,7 @@ See the [`Experiment`](lib/lab_coat/experiment.rb) class for more details.
 |`publish!`|This is not _technically_ required, but `Experiments` are not useful unless you can analyze the results. Override this method to record the `Result` however you wish.|
 
 > [!IMPORTANT]
-> The `#run!` method accepts arbitrary arguments and forwards them to `enabled?`, `control`, and `candidate` in case you need to provide data at runtime. This means the [arity](https://en.wikipedia.org/wiki/Arity) of the three methods needs to be the same. This is enforced by `LabCoat` at runtime.
+> The `#run!` method accepts arbitrary key word arguments and stores them in an instance variable called `@context` in case you need to provide data at runtime. You can access the runtime context via `@context` or `context`. The runtime context **is reset** after each run.
 
 #### Additional methods
 
@@ -88,13 +88,16 @@ You might want to `publish!` all experiments in a consistent way so that you can
 # application_experiment.rb
 class ApplicationExperiment < LabCoat::Experiment
   def publish!(result)
-    payload = result.to_h.merge(user_id: @user.id)
+    payload = result.to_h.merge(
+      user_id: @user.id, # e.g. something from the `Experiment` state
+      build_number: context.version # e.g. something from the runtime context
+    )
     YourO11yService.track_experiment_result(payload)
   end
 end
 ```
 
-You might have a common way to enable experiments such as a feature flag system and/or common guards you want to enforce application wide. These might come from a mix of services and the `Experiment`'s state.
+You might have a common way to enable experiments such as a feature flag system and/or common guards you want to enforce application wide. These might come from a mix of services, the `Experiment`'s state, or the runtime `context`.
 
 ```ruby
 # application_experiment.rb
@@ -123,6 +126,8 @@ end
 
 You don't have to create an `Observation` yourself; that happens automatically when you call `Experiment#run!`. The control and candidate `Observations` are packaged into a `Result` and [passed to `Experiment#publish!`](#publish-the-result).
 
+The `run!` method accepts arbitrary keyword arguments, to allow you to set runtime context for the specific run of the experiment. You can access this `Hash` via the `context` reader method, or directly via the `@context` instance variable.
+
 |Attribute|Description|
 |---|---|
 |`duration`|The duration of the run represented as a `Benchmark::Tms` object.|
@@ -180,14 +185,14 @@ A `Result` represents a single run of an `Experiment`.
 |`matched?`|Whether or not the `control` and `candidate` match, as defined by `Experiment#compare`|
 |`to_h`|A hash representation of the `Result`. Useful for publishing and/or reporting.|
 
-The `Result` is passed to your implementation of `#publish!` when an `Experiment` is finished running. The `to_h` method on a Result is a good place to start and might be sufficient for most experiments.
+The `Result` is passed to your implementation of `#publish!` when an `Experiment` is finished running. The `to_h` method on a Result is a good place to start and might be sufficient for most experiments. You might want to `merge` additional data such as the runtime `context` or other state if you find that relevant for analysis.
 
 ```ruby
 # your_experiment.rb
 def publish!(result)
   return if result.ignored?
 
-  puts result.to_h
+  puts result.to_h.merge(run_context: context)
 end
 ```
 
@@ -276,7 +281,7 @@ end
 ```
 
 > [!WARNING]
-> Be careful when using `Observation` instances without an `Experiment` set. Some methods like `#publishable_value` and `#slug` depend on an `experiment` and may raise an error when called.
+> Be careful when using `Observation` instances without an `Experiment` set. Some methods like `#publishable_value` and `#slug` depend on an `experiment` and may raise an error or return unexpected values when called without one.
 
 ## Development
 

lib/lab_coat/experiment.rb

@@ -3,10 +3,11 @@
 module LabCoat
   # A base experiment class meant to be subclassed to define various experiments.
   class Experiment
-    attr_reader :name
+    attr_reader :name, :context
 
     def initialize(name)
       @name = name
+      @context = {}
     end
 
     # Override this method to control whether or not the experiment runs.
@@ -18,13 +19,13 @@ def enabled?(...)
     # Override this method to define the existing aka "control" behavior. This method is always run, even when
     # `enabled?` is false.
     # @return [Object] Anything.
-    def control(...)
+    def control
       raise InvalidExperimentError, "`#control` must be implemented in your Experiment class."
     end
 
     # Override this method to define the new aka "candidate" behavior. Only run if the experiment is enabled.
     # @return [Object] Anything.
-    def candidate(...)
+    def candidate
       raise InvalidExperimentError, "`#candidate` must be implemented in your Experiment class."
     end
 
@@ -59,25 +60,25 @@ def publish!(result); end
 
     # Runs the control and candidate and publishes the result. Always returns the result of `control`.
     # @param context [Hash] Any data needed at runtime.
-    def run!(...) # rubocop:disable Metrics/MethodLength
-      enforce_arity!
+    def run!(**context)
+      # Set the context for this run.
+      @context = context
 
       # Run the control and exit early if the experiment is not enabled.
-      control_obs = Observation.new("control", self) do
-        control(...)
-      end
+      control_obs = Observation.new("control", self) { control }
       raised(control_obs) if control_obs.raised?
-      return control_obs.value unless enabled?(...)
+      return control_obs.value unless enabled?
 
-      candidate_obs = Observation.new("candidate", self) do
-        candidate(...)
-      end
+      candidate_obs = Observation.new("candidate", self) { candidate }
       raised(candidate_obs) if candidate_obs.raised?
 
       # Compare and publish the results.
       result = Result.new(self, control_obs, candidate_obs)
       publish!(result)
 
+      # Reset the context for this run.
+      @context = {}
+
       # Always return the control.
       control_obs.value
     end

lib/lab_coat/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module LabCoat
-  VERSION = "0.1.3"
+  VERSION = "0.1.4"
 end