Entity Framework Core 6 vs EF6 Performance - BenchmarkDotNet

Entity Framework Final Countdown

This post is Part 1 of a multi-part series of posts on benchmarking EF Core 6 and EF6 using BenchmarkDotNet

Be on the lookout for future EF Core vs EF6 Benchmarking posts

I am an unabashed fanboy of Entity Framework. EF Core 6 is my current tool of choice but I fondly remember stumbling upon EF4 as a fresh, starry eyed engineer many years ago and being amazed that something as easy as LINQ could serve data from a database. Sure, sometimes it’s necessary to drop into SQL Land and reconnect with your past but like milk and cookies, peanut butter and jelly or even Siegfried and Roy – Entity Framework and databases just go good together.

These days, with EF Core 6 being almost at full parity with EF6, I exclusively use EF Core for database access but I’ve always been interested in how much more performant EF Core is than legacy EF6.

Putting my curiosity into action and utilizing the fantastic BenchmarkDotNet Library I am going to be pitting EF Core against EF6 in the next series of blog posts.

For all code in this post and all subsequent posts on EF Core 6 vs EF6 Benchmarking, see the following project on my github at https://github.com/BrianMikinski/EfPerformance

An Old Friend… .Net Stopwatch Class!

If you’re doing some casual benchmarking and need a rough idea of how much faster one method implementation is vs the other, I would suggest using the tried and true .Net Stopwatch.

Here’s an example from a simple ProfilingService.cs that I wrote to double check the exciting new benchmarking tool we’ll be discussing next.

1
2
3
4
5
6
Stopwatch stopWatch = new();
stopWatch.Start();

_blogContext.Posts.FirstOrDefault();

stopWatch.Stop();

In this example, we simply do the following

1. Create a new .Net Stopwatch object
2. Start() the Stopwatch timer
3. Run our code and then Stop() the Stopwatch timer

That’s it, that’s all there is to it. It’s simple, easy and works great…but there is another .Net profiling tool that we should try out.

Introducing…BenchmarkDotNet!

This is my first time exploring BenchmarkDotNet but my initial impressions are that it is a fantastic library for digging deep into C# code to fully understand performance from both a CPU and Memory perspective. BenchmarkDotNet allows for easy, simple (and advanced) statistical analysis through the use of method attributes on your benchmark functions. There are of course many advanced features of BenchmarkDotNet but in the most barebones use cases, BenchmarkDotNet is nothing more than adding a couple of method attributes.

BenchmarkDotNet Driver Console App

In my limited experience with BenchmarkDotNet, it seems like the correct way to structure your benchmarking is to use dotnet console app as your benchmark driver and then build your benchmarks in separate classes or projects all together. Going forward, for my EF Core vs EF6 examples, this is the structure I will be using.

Here is the Program.cs file that can be found in EFPerformance project. The BenchmarkDotNet driver console app is pretty straight forward.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
using BenchmarkDotNet.Configs;
using BenchmarkDotNet.Jobs;
using BenchmarkDotNet.Running;
using Blog.Benchmarks;

Console.WriteLine("Hello, World! Welcome to EF Core vs Ef6 Performance profiling!");
BenchmarkSwitcher.FromAssembly(typeof(BenchmarkBase).Assembly).Run(args, GetGlobalConfig());

Console.WriteLine("Post profiling statistics: ");
var ef6ProfilingService = new ProfilingService();

var (coreBlogDiagnostics, blogDiagnostics) = ef6ProfilingService.TableDiagnostics();

coreBlogDiagnostics.PrintTableDiagnostics();
blogDiagnostics.PrintTableDiagnostics(false);

/// <summary>
/// Programatic configuration of the jobs.
/// This can be overwridden by passing in arguments
/// </summary>
static IConfig GetGlobalConfig()
    => DefaultConfig.Instance.AddJob(Job.Default
            .WithWarmupCount(1)
            .AsDefault());

Most of the file can be ignored but the main line to focus in on is line 7 or the one that begins with BenchmarkSwitcher

BenchmarkSwitcher.FromAssembly(typeof(BenchmarkBase).Assembly).Run(args, GetGlobalConfig());

The BenchmarkSwitcher is what BenchmarkDotNet utilizes to know it is configuring this console app to run benchmarks. Another part of the BenchmarkSwitcher is to keep track of is the FromAssembly(typeof(BenchmarkBase).Assembly).

FromAssembly allows us to tell BenchmarkDotNet where to look for our benchmark methods. In this case, BenchmarkBase is referencing BenchmarkBase.cs that I have created in a separate project called Blog.Benchmarks.csproj where all of my benchmarks are grouped into classes around functionality.

Now let’s get onto the good stuff. Some actual code using BenchmarkDotNet!

Benchmark Demo Class

Here is an DemoBenchmark.cs class to demonstrate some baseline functionality.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Configs;
using System.Text;

namespace Blog.Benchmarks;

[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]
[CategoriesColumn]
public class DemoBenchmark : BenchmarkBase
{
    [GlobalSetup]
    public void GlobalSetup()
    {
       // code that is run once prior to all BenchmarkDotNet iterations
    }

    [IterationSetup]
    public void IterationSetup()
    {
        // code that is run once prior to each iteration
    }

    [BenchmarkCategory(nameof(DemoBenchmark))]
    [Benchmark(Baseline = true)]
    public void OldFunction()
    {
        string firstName = "John";
        string lastName = "Doe";
        string name = firstName + " " + lastName;
    }

    [BenchmarkCategory(nameof(DemoBenchmark))]
    [Benchmark]
    public void NewFunction()
    {
        string firstName = "John";
        string lastName = "Doe";
        StringBuilder sb = new(firstName);

        string name = sb.Append(" ")
            .Append(lastName)
            .ToString();
    }

    [IterationCleanup]
    public void IterationCleanup()
    {
        // code run once after each iteration
    }

    [GlobalCleanup]
    public void GlobalCleanup()
    {
       // code run once after all iterations have run
    }
}

In the DemoBenchmark.cs file we can see the [Benchmark] attribute for identifying methods that we want to be benchmarked as well as the [GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)] and [BenchmarkCategory(nameof(DemoBenchmark))] for marking the name of the category that these tests fall in to.

Why might we want to do this? The answer is in how we want to run the tests. More often then not, it is useful to run only select tests for specific scenarios rather than a full battery of tests. The benchmark category attributes allow us to do just that.

BenchmarkDotNet Automation with PowerShell

Because this project holds many Benchmarks, and running all of the benchmarks at once would take quite a long time (due to the fact that we are going to be continually hitting a database for our tests) I have my benchmarks automated to run specific categories using PowerShell scripts.

Here is an example of the demo.Benchmark.ps1 file

1
2
# run benchmarking code
dotnet run -c Release --filter *DemoBenchmark*

In the code above, the –filter argument get’s passed to the .Run(args, GetGlobalConfig()). line of the Program.cs console app. This can be traced through the following BenchmarkSwitcher.FromAssembly code

1
BenchmarkSwitcher.FromAssembly(typeof(BenchmarkBase).Assembly).Run(args, GetGlobalConfig());

And, from the previous section above showing the full DemoBenchmark class, we have our benchmark functions marked with attributes that will always match the name of their containing class by using the nameof(DemoBenchmark) in our category definitions. Pretty neat way of keeping your benchmark categories and BenchmarkDotNet method runs in sync!

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
[GroupBenchmarksBy(BenchmarkLogicalGroupRule.ByCategory)]
[CategoriesColumn]
public class DemoBenchmark : BenchmarkBase
{
    // ...
    // setup code
    // ...
    
    [BenchmarkCategory(nameof(DemoBenchmark))]
    [Benchmark(Baseline = true)]
    public void OldFunction()
    {
        string firstName = "John";
        string lastName = "Doe";
        string name = firstName + " " + lastName;
    }
}

So far, this setup for running benchmarks on methods has worked very well. I am able to quickly and repeatedly test specific benchmark methods yet still maintain a clean level of organization across the benchmark’s containing classes.  

Results of the DemoBenchmark Class

Another great thing about BenchmarkDotNet is the ease at which one can share the results of their benchmarks. My blog is written in Markdown and lucky for me, BenchmarkDotNet automatically generates results in multiple formats - one of which is an .md Markdown file that I have included below.

1
2
3
4
5
6
7
8
BenchmarkDotNet=v0.13.1, OS=Windows 10.0.19043.1586 (21H1/May2021Update)
Intel Core i5-6200U CPU 2.30GHz (Skylake), 1 CPU, 4 logical and 2 physical cores
.NET SDK=6.0.201
  [Host]     : .NET 6.0.3 (6.0.322.12309), X64 RyuJIT
  Job-HWYLMR : .NET 6.0.3 (6.0.322.12309), X64 RyuJIT

InvocationCount=1  UnrollFactor=1  WarmupCount=1  
Categories=DemoBenchmark  

As you can see from the results, the NewFunction is about ~1.55 times slower than the OldFunction. If we were rewriting this method in a real world scenario, we would not want to use the new implementation over the older one.

BenchmarkDotNet Conclusion

In conclusion, BenchmarkDotNet is an absolutely incredible tool and one that I will no doubt continue to explore. In the follow up series of posts, I intend to put EF Core 6 and EF6 to the test using BenchmarkDotNet to dig deep and understand under what scenarios which EF version is faster then the other. Until next time, have a Wonderful Day and Happy Coding!  

If you want to view all of the code show in this post, you can view the source at the following github repo https://github.com/BrianMikinski/EfPerformance