Benchmarks and profiles

In addition to functional tests, go test also has built-in benchmarks (BenchmarkXxx) and supports CPU and memory profiles (pprof). All without external dependencies.

Shape of a benchmark

func BenchmarkSum(b *testing.B) {
    for i := 0; i < b.N; i++ {
        _ = Sum(2, 3)
    }
}

Characteristics:

• Exact signature: func BenchmarkXxx(b *testing.B).
• The loop must run b.N iterations: the framework adjusts b.N automatically, starting at 1 and growing until the measurement stabilises (~1 second by default).
• The result is ns/op (nanoseconds per operation) and, with -benchmem, also B/op and allocs/op.

Typical commands

go test -bench=. ./...              # all benchmarks
go test -bench=Sum -benchmem        # only Sum, with memory info
go test -bench=. -benchtime=5s      # each benchmark for 5 seconds
go test -bench=. -cpuprofile=cpu.out # generate a CPU profile

Typical output:

BenchmarkSum-8        1000000000        0.31 ns/op

-8 is the GOMAXPROCS value. 0.31 ns/op is the average time per call.

Expensive setup: b.ResetTimer

If you have a long preparation step before the loop, exclude it from the measurement:

func BenchmarkSearch(b *testing.B) {
    data := buildLargeSlice(1_000_000) // expensive
    b.ResetTimer()
    for i := 0; i < b.N; i++ {
        _ = Search(data, 42)
    }
}

Useful variants:

• b.StopTimer() / b.StartTimer() — pause the measurement around an uninteresting phase (handy inside the loop).
• b.ReportAllocs() — force allocation reporting even without -benchmem.

Benchmark table with b.Run

Just like t.Run for tests, there is b.Run for parametric benchmarks:

func BenchmarkSplit(b *testing.B) {
    for _, size := range []int{10, 1000, 100000} {
        b.Run(fmt.Sprintf("n=%d", size), func(b *testing.B) {
            s := strings.Repeat("a,", size)
            b.ResetTimer()
            for i := 0; i < b.N; i++ {
                _ = strings.Split(s, ",")
            }
        })
    }
}

Output rows like BenchmarkSplit/n=10-8, BenchmarkSplit/n=1000-8, ... ideal for scaling charts.

Comparing two implementations with benchstat

go test -bench=. -count=10 > old.txt
# apply change
go test -bench=. -count=10 > new.txt
benchstat old.txt new.txt

benchstat (in golang.org/x/perf/cmd/benchstat) computes the mean, standard deviation and statistical significance of the difference.

Exercises

$ go test ???