Go Subtest Tips
I recently learned more detail about subtests in Go.
Testing in Go is a first class concept. It’s this easy (example taken from play.golang.org:
package main
import (
"testing"
)
// LastIndex returns the index of the last instance of x in list, or
// -1 if x is not present. The loop condition has a fault that
// causes somes tests to fail. Change it to i >= 0 to see them pass.
func LastIndex(list []int, x int) int {
for i := len(list) - 1; i > 0; i-- {
if list[i] == x {
return i
}
}
return -1
}
func TestLastIndex(t *testing.T) {
tests := []struct {
list []int
x int
want int
}{
{list: []int{1}, x: 1, want: 0},
{list: []int{1, 1}, x: 1, want: 1},
{list: []int{2, 1}, x: 2, want: 0},
{list: []int{1, 2, 1, 1}, x: 2, want: 1},
{list: []int{1, 1, 1, 2, 2, 1}, x: 3, want: -1},
{list: []int{3, 1, 2, 2, 1, 1}, x: 3, want: 0},
}
for _, tt := range tests {
if got := LastIndex(tt.list, tt.x); got != tt.want {
t.Errorf("LastIndex(%v, %v) = %v, want %v", tt.list, tt.x, got, tt.want)
}
}
}The output of the above is:
=== RUN TestLastIndex
TestLastIndex: prog.go:34: LastIndex([1], 1) = -1, want 0
TestLastIndex: prog.go:34: LastIndex([2 1], 2) = -1, want 0
TestLastIndex: prog.go:34: LastIndex([3 1 2 2 1 1], 3) = -1, want 0
--- FAIL: TestLastIndex (0.00s)
FAIL
1 test failed.
Tests of any substance I tend to break up into subtests. I might change the above like this:
package main
import (
"strconv"
"testing"
)
func LastIndex(list []int, x int) int {
for i := len(list) - 1; i > 0; i-- {
if list[i] == x {
return i
}
}
return -1
}
func TestLastIndex(t *testing.T) {
tests := []struct {
list []int
x int
want int
}{
0: {list: []int{1}, x: 1, want: 0},
1: {list: []int{1, 1}, x: 1, want: 1},
2: {list: []int{2, 1}, x: 2, want: 0},
3: {list: []int{1, 2, 1, 1}, x: 2, want: 1},
4: {list: []int{1, 1, 1, 2, 2, 1}, x: 3, want: -1},
5: {list: []int{3, 1, 2, 2, 1, 1}, x: 3, want: 0},
}
for i, tt := range tests {
t.Run(strconv.Itoa(i), func(t *testing.T) {
if got := LastIndex(tt.list, tt.x); got != tt.want {
t.Errorf("LastIndex(%v, %v) = %v, want %v", tt.list, tt.x, got, tt.want)
}
})
}
}The output of this version is:
=== RUN TestLastIndex
=== RUN TestLastIndex/0
TestLastIndex/0: prog.go:33: LastIndex([1], 1) = -1, want 0
=== RUN TestLastIndex/1
=== RUN TestLastIndex/2
TestLastIndex/2: prog.go:33: LastIndex([2 1], 2) = -1, want 0
=== RUN TestLastIndex/3
=== RUN TestLastIndex/4
=== RUN TestLastIndex/5
TestLastIndex/5: prog.go:33: LastIndex([3 1 2 2 1 1], 3) = -1, want 0
--- FAIL: TestLastIndex (0.00s)
--- FAIL: TestLastIndex/0 (0.00s)
--- PASS: TestLastIndex/1 (0.00s)
--- FAIL: TestLastIndex/2 (0.00s)
--- PASS: TestLastIndex/3 (0.00s)
--- PASS: TestLastIndex/4 (0.00s)
--- FAIL: TestLastIndex/5 (0.00s)
FAIL
4 tests failed.
There are two nice thing about subtests:
- built in reporting of which tests failed (0, 2, and 5)
- the ability to only run some subset of tests
The built in test framework allows running tests like this:
$ go test ./...That runs tests for all of the go recursively under the current directory. Sometimes when you are dealing with a failing test you might want to run just the broken ones:
$ go test -v -run 'Last/[025]'
=== RUN TestLastIndex
=== RUN TestLastIndex/0
TestLastIndex/0: x_test.go:33: LastIndex([1], 1) = -1, want 0
=== RUN TestLastIndex/2
TestLastIndex/2: x_test.go:33: LastIndex([2 1], 2) = -1, want 0
=== RUN TestLastIndex/5
TestLastIndex/5: x_test.go:33: LastIndex([3 1 2 2 1 1], 3) = -1, want 0
--- FAIL: TestLastIndex (0.00s)
--- FAIL: TestLastIndex/0 (0.00s)
--- FAIL: TestLastIndex/2 (0.00s)
--- FAIL: TestLastIndex/5 (0.00s)
FAIL
FAIL x 0.003s
FAILThe -run flag to go test is a regex, or actually a bunch of regexen
separated with /s. This is super handy and allows for realy brief
commandlines, but also there are some subtle implications. Consider this
totally contrived code:
package main
import (
"testing"
)
func LastIndex(list []int, x int) int {
for i := len(list) - 1; i > 0; i-- {
if list[i] == x {
return i
}
}
return -1
}
func TestLastIndex(t *testing.T) {
tests := []struct {
name string
list []int
x int
want int
}{
{name: "single", list: []int{1}, x: 1, want: 0},
{name: "pair", list: []int{1, 1}, x: 1, want: 1},
{name: "pair/different", list: []int{2, 1}, x: 2, want: 0},
{name: "four", list: []int{1, 2, 1, 1}, x: 2, want: 1},
{name: "five", list: []int{1, 1, 1, 2, 2, 1}, x: 3, want: -1},
{name: "six", list: []int{3, 1, 2, 2, 1, 1}, x: 3, want: 0},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := LastIndex(tt.list, tt.x); got != tt.want {
t.Errorf("LastIndex(%v, %v) = %v, want %v", tt.list, tt.x, got, tt.want)
}
})
}
}Output is (hopefully unsurprisingly:)
=== RUN TestLastIndex
=== RUN TestLastIndex/single
TestLastIndex/single: prog.go:33: LastIndex([1], 1) = -1, want 0
=== RUN TestLastIndex/pair
=== RUN TestLastIndex/pair/different
TestLastIndex/pair/different: prog.go:33: LastIndex([2 1], 2) = -1, want 0
=== RUN TestLastIndex/four
=== RUN TestLastIndex/five
=== RUN TestLastIndex/six
TestLastIndex/six: prog.go:33: LastIndex([3 1 2 2 1 1], 3) = -1, want 0
--- FAIL: TestLastIndex (0.00s)
--- FAIL: TestLastIndex/single (0.00s)
--- PASS: TestLastIndex/pair (0.00s)
--- FAIL: TestLastIndex/pair/different (0.00s)
--- PASS: TestLastIndex/four (0.00s)
--- PASS: TestLastIndex/five (0.00s)
--- FAIL: TestLastIndex/six (0.00s)
FAIL
4 tests failed.
Now let’s say you wanted to run just pair/different while you debug it. In
this test there’s not a lot of output, but sometimes tests might have tons of
lines of output so it can be useful to only run what you must. Here’s some commands you
might consider trying to run pair/different:
go test -run Index/pair/differentgo test -run Index/pair\/differentgo test -run 'Index/pair\/different'go test -run 'Index/pair[/]different'go test -run 'Index/pair.different'
All of these either run both pair and pair/different or they run no
subtests at all. The answer comes in the documentation for -run:
-run regexp
Run only those tests and examples matching the regular expression.
For tests, the regular expression is split by unbracketed slash (/)
characters into a sequence of regular expressions, and each part
of a test's identifier must match the corresponding element in
the sequence, if any. Note that possible parents of matches are
run too, so that -run=X/Y matches and runs and reports the result
of all tests matching X, even those without sub-tests matching Y,
because it must run them to look for those sub-tests.
This explains both of the outcomes in the commands above. When you run go
test -run X/Y/Z go creates three regexp objects (one for X, one for Y, and one
for Z.) It matches X against the function name of the test; if it matches, it
will then run the test. When the test is running, it tries to run all subtests
and if the subtest matches Y, the subtest gets run. Finally any subtests of Y
named Z will be run, though we happen to not have a third level of nesting
here.
This implies two rules for subtests in go. First, do not put / in subtest
names. One could argue that testing should actually escape these, but just
not using / is simple. If you have a / in a subtest name and the prefix
before the matches other tests, you’ll never be able to run the longer named
test without also running the prefix. At work I used to have tests that had
filenames in the subtest name; because of this I replaced the / with
something like !.
Second, if possible, keep the meat of the test in the leaf tests. Anything in non-leaf tests may get run and may produce distracting output.
(Affiliate links below.)
If you want to learn more
about programming Go, you should check out The
Go Programming Language
Another book to consider learning Go with is
Go Programming Blueprints
If you're interested in being notified when new posts are published, you can subscribe here; you'll get an email once a week at the most.