IS there a good strategy to benchmark against previous versions of a library?

[paulomorgado]

I'd like to compare the current branch with the latest published NuGet.

Is there a good strategy for that?

[timcassell]

See https://benchmarkdotnet.org/articles/samples/IntroNuGet.html

You can use a Condition to select a package reference or project reference depending on your custom MsBuildArguments.

<ProjectReference Include="MyLib.csproj" Condition="'$(MyLibVersion)' == 'develop'">
<PackageReference Include="MyLib" Version="'$(MyLibVersion)'" Condition="'$(MyLibVersion)' != 'develop'">

[paulomorgado]

Got it! Thanks, @timcassell!

private class Config : ManualConfig
{
    public Config()
    {
        string[] targetVersions = [
            "",
            "0.3.1",
        ];

        Runtime[] targetRuntimes = [CoreRuntime.Core80, CoreRuntime.Core10_0, ClrRuntime.Net481];

        foreach (var targetRuntime in targetRuntimes)
        {
            foreach (var version in targetVersions)
            {
                AddJob(Job.MediumRun
                    .WithRuntime(targetRuntime)
                    .WithMsBuildArguments($"/p:LibVersion={version}")
                    .WithId(string.IsNullOrEmpty(version) ? "local" : $"Nuget-{version}")
                );
            }
        }

        AddDiagnoser(BenchmarkDotNet.Diagnosers.MemoryDiagnoser.Default);
    }
}

Method	Job	Runtime	Arguments	Mean	Error	StdDev	Gen0	Allocated
ProtectUnprotect	0.3.1	.NET 10.0	/p:LibVersion=0.3.1	2.073 us	0.0064 us	0.0096 us	0.0343	688 B
ProtectUnprotect	0.3.1	.NET 8.0	/p:LibVersion=0.3.1	2.509 us	0.0064 us	0.0093 us	0.0343	688 B
ProtectUnprotect	0.3.1	.NET Framework 4.8.1	/p:LibVersion=0.3.1	3.107 us	0.0130 us	0.0191 us	0.1068	690 B
ProtectUnprotect	local	.NET 10.0	/p:LibVersion=	2.079 us	0.0054 us	0.0081 us	0.0343	688 B
ProtectUnprotect	local	.NET 8.0	/p:LibVersion=	2.501 us	0.0201 us	0.0301 us	0.0343	688 B
ProtectUnprotect	local	.NET Framework 4.8.1	/p:LibVersion=	3.115 us	0.0076 us	0.0114 us	0.1068	690 B