With numerous methods providing similar solutions, which one is best suited for your specific needs?

This is where benchmarking comes into play.

By gathering data on execution time and resource usage, it allows developers to identify bottlenecks and compare various implementations.

Additionally, it's valuable tool for verifying different claims, as errors can arise. It’s always wise to cross-check with benchmarks rather than relying solely on assertions.

In the .NET ecosystem, one of the most effective tools for benchmarking is BenchmarkDotNet.

BenchmarkDotNet

BenchmarkDotNet is a powerful, open-source library designed for benchmarking .NET applications.

It helps you to transform methods into benchmarks, track their performance and share reproducible measurement experiments.

It also protects you from common mistakes and warns you if something is wrong with your benchmark design or obtained measurements.

Measured data can be exported to different formats (md, html, csv, xml, json, etc.) including plots.

Getting Started with BenchmarkDotNet

To get started with BenchmarkDotNet, you need to install the NuGet package. You can do this via the NuGet Package Manager or by running the following command in the Package Manager Console:

Install-Package BenchmarkDotNet

Basic Usage

Once installed, you can start benchmarking.

First, we need to create a benchmark class. This class must be public and will contain the methods we want to measure. Each method should be annotated with the [Benchmark] attribute.

Here’s a simple example of benchmark class:

public class LoopBenchmark
{
    private readonly List<string> _items = ["www.", "nikolatech", ".net"];

    [Benchmark]
    public string For()
    {
        var response = string.Empty;
        var size = _items.Count;

        for (var i = 0; i < size; i++)
        {
            response = _items[i];
        }

        return response;
    }

    [Benchmark]
    public string Foreach()
    {
        var response = string.Empty;

        foreach (var item in _items)
        {
            response = item;
        }

        return response;
    }
}

Your benchmark class is ready. Now, we need to call BenchmarkRunner.Run() in Program.cs to initiate the benchmarks.

using BenchmarkDotNet.Running;

var summary = BenchmarkRunner.Run<LoopBenchmark>();

Note: To execute the benchmarks, ensure the application is in Release mode and run it without debugging.

Setup And Cleanup

It's common to need to execute logic before or after a benchmark without including it in the benchmark itself.

BenchmarkDotNet offers several attributes for this purpose:
[GlobalSetup], [GlobalCleanup], [IterationSetup] and [IterationCleanup].

A method marked with the [GlobalSetup] attribute runs once for each benchmarked method.

A method marked with the [GlobalCleanup] attribute runs once for each benchmarked method, after all invocations of that method have completed.

public const int Size = 1000;

private readonly List<string> _items = [];

[GlobalSetup]
public void Setup()
{
    var random = new Random(123);
    for (var i = 0; i < Size; i++)
    {
        var randomValue = random.Next();
        _items.Add(randomValue.ToString());
    }
}

NOTE: This is especially useful for managing unmanaged resources. You can allocate them in the GlobalSetup method and release them in the GlobalCleanup method.

Tips & Tricks

BenchmarkDotNet is packed with features to simplify benchmarking for developers. Its extensive configurability options allow you to tailor benchmarks to meet specific needs.

Baseline

You can mark a benchmark method as a baseline, allowing for easy comparison with other benchmarks.

[Benchmark(Baseline = true)]
public string For()
{
    var response = string.Empty;
    var size = _items.Count;

    for (var i = 0; i < size; i++)
    {
        response = _items[i];
    }

    return response;
}

Targeting Multiple .NET Versions

You can benchmark across multiple .NET versions by updating the target frameworks in your .csproj file and adding the [SimpleJob] attribute to your benchmark class.

<PropertyGroup>
  <OutputType>Exe</OutputType>
  <TargetFrameworks>net6.0;net7.0;net8.0</TargetFrameworks>
  <ImplicitUsings>enable</ImplicitUsings>
  <Nullable>enable</Nullable>
</PropertyGroup>

[SimpleJob(RuntimeMoniker.Net60)]
[SimpleJob(RuntimeMoniker.Net70)]
[SimpleJob(RuntimeMoniker.Net80)]
public class LoopBenchmark
{
    // ...
}

Parameterization

You can mark fields and properties in your class with the [Params] attribute to specify a set of values.

public class LoopBenchmark
{
    [Params(100, 10_000, 1_000_000)] public int Size { get; set; }

    // ...
}

Cold Start

The [SimpleJob] attribute can also be used to specify how many times it should launch, warm up and iterate, simulating real-world performance under cold start conditions.

[SimpleJob(RunStrategy.ColdStart, launchCount: 3, warmupCount: 2, iterationCount: 10)]
public class LoopBenchmark
{
    // ...
}

Diagnosers

BenchmarkDotNet provides a variety of diagnosers to analyze different aspects.

For exmaple [MemoryDiagnoser] measures memory usage, tracks allocations and garbage collection.

[MemoryDiagnoser]
public class LoopBenchmark
{
    // ...
}

ManualConfig

You can also utilize ManualConfig to create custom configurations for your benchmarks.

For example you can configure summary output of benchmarks with different ratio formats.

[Config(typeof(Config))]
public class LoopBenchmark
{

    private class Config : ManualConfig
    {
        public Config()
        {
            SummaryStyle =
                SummaryStyle.WithRatioStyle(RatioStyle.Trend);
        }
    }

    // ...
}

Conclusion

BenchmarkDotNet is an invaluable tool with its high precision, customizability and straightforward implementation.

It simplifies the process of measuring performance by providing a range of handy features.

Check out BenchmarkDotNet on GitHub and if you like it, give it a star:

If you want to check out examples I created, you can find the source code here:

I hope you enjoyed it, subscribe and get a notification when new blog is up!