scientist

# Scientist!

A Ruby library for carefully refactoring critical paths. [![Build Status](https://github.com/github/scientist/actions/workflows/ci.yml/badge.svg)](https://github.com/github/scientist/actions/workflows/ci.yml)

## How do I science?

Let's pretend you're changing the way you handle permissions in a large web app. Tests can help guide your refactoring, but you really want to compare the current and refactored behaviors under load.

```ruby
require "scientist"

class MyWidget
  def allows?(user)
    experiment = Scientist::Default.new "widget-permissions"
    experiment.use { model.check_user(user).valid? } # old way
    experiment.try { user.can?(:read, model) } # new way

    experiment.run
  end
end
```

Wrap a `use` block around the code's original behavior, and wrap `try` around the new behavior. `experiment.run` will always return whatever the `use` block returns, but it does a bunch of stuff behind the scenes:

* It decides whether or not to run the `try` block,
* Randomizes the order in which `use` and `try` blocks are run,
* Measures the wall time and cpu time of all behaviors in seconds,
* Compares the result of `try` to the result of `use`,
* Swallow and record exceptions raised in the `try` block when overriding `raised`, and
* Publishes all this information.

The `use` block is called the **control**. The `try` block is called the **candidate**.

Creating an experiment is wordy, but when you include the `Scientist` module, the `science` helper will instantiate an experiment and call `run` for you:

```ruby
require "scientist"

class MyWidget
  include Scientist

  def allows?(user)
    science "widget-permissions" do |experiment|
      experiment.use { model.check_user(user).valid? } # old way
      experiment.try { user.can?(:read, model) } # new way
    end # returns the control value
  end
end
```

If you don't declare any `try` blocks, none of the Scientist machinery is invoked and the control value is always returned.

## Making science useful

The examples above will run, but they're not really *doing* anything. The `try` blocks don't run yet and none of the results get published. Replace the default experiment implementation to control execution and reporting:

```ruby
require "scientist/experiment"

class MyExperiment
  include Scientist::Experiment

  attr_accessor :name

  def initialize(name)
    @name = name
  end

  def enabled?
    # see "Ramping up experiments" below
    true
  end

  def raised(operation, error)
    # see "In a Scientist callback" below
    p "Operation '#{operation}' failed with error '#{error.inspect}'"
    super # will re-raise
  end

  def publish(result)
    # see "Publishing results" below
    p result
  end
end
```

When `Scientist::Experiment` is included in a class, it automatically sets it as the default implementation via `Scientist::Experiment.set_default`. This `set_default` call is skipped if you include `Scientist::Experiment` in a module.

Now calls to the `science` helper wi