@matloob @bcmills @jayconrod

This has basically nothing to do with test2json — it is more-or-less correctly parsing the bad output. The problem is that the output is missing a newline, and the testing wrapper doesn't inject one:

example.com$ go version
go version devel +6b6414ca Wed Mar 25 04:06:34 2020 +0000 linux/amd64

example.com$ go mod init example.com
go: creating new go.mod: module example.com

example.com$ cat > testissue_test.go
package testissue

import (
        "fmt"
        "testing"
)

func TestBad(t *testing.T) {
        // No newline below
        fmt.Print("bad output")
}

func TestGood(t *testing.T) {
        // Print a newline below
        fmt.Println("good output")
}

example.com$ go test -v .
=== RUN   TestBad
bad output--- PASS: TestBad (0.00s)
=== RUN   TestGood
good output
--- PASS: TestGood (0.00s)
PASS
ok      example.com     0.021s

This problem appears to have existed for a long time:

example.com$ go1.13.9 test -v .
=== RUN   TestBad
bad output--- PASS: TestBad (0.00s)
=== RUN   TestGood
good output
--- PASS: TestGood (0.00s)
PASS
ok      example.com     (cached)

example.com$ go1.12.17 test -v .
=== RUN   TestBad
bad output--- PASS: TestBad (0.00s)
=== RUN   TestGood
good output
--- PASS: TestGood (0.00s)
PASS
ok      example.com     (cached)

CC @mpvl @changkun

Can we really do anything about this? I could not imagine a way to determine the last character from flushed outputs... In the log package, it is pretty straightforward, because it has an internal buffer, and the check can be done by checking its buffer, but in the testing package, the output is directly executed from the userland.

Unless we print a \n before the --- PASS or PASS anyway.

Related to #27764 #35180 #33419 at least.

Can we really do anything about this? I could not imagine a way to determine the last character from flushed outputs...

Maybe. Test binaries write messages to the go test driver on stdout. Tests may also write to stdout, so the output gets mixed together. Test binaries could write messages on some other channel (pipe, socket, whatever) to avoid this problem.

A dedicated “test metadata” communication channel with JSON or other machine readable format to transport this data around would seem sensible. Then it can be converted to the regular output (or kept as JSON or converted to whatever other output formats that are required) would seem logical to me.

Could it even be kept in-process, to avoid the serialisation overhead?

Could it even be kept in-process, to avoid the serialisation overhead?

Unfortunately not. Each package's test is built into a separate binary and run in a separate process. Global side effects from tests and init functions shouldn't be visible to tests for other packages.

@changkun, testing and os are both in the standard library, so if need be we could even hook os.Stdout and os.Stderr with a side-channel to tell the testing package about output line state.

(#29062 is closely related, in the sense that it could be implemented by hooking the os package.)

Interesting. The print and println builtins just came to my mind and I didn't know them is not guaranteed to stay in Go (said in builtin package). A quick look into runtime implementation indicates that they invoke syscall write directly. The same applies to the syscall package... It seems we cannot easily do hooks about these writes, is it?

Perhaps we could just duplicate meta outputs from testing to a tmp file similar to what @jayconrod suggested, then parse these temp files in test2json?

I'm not worried about hooking the builtins or anything that hard-codes writes to those file descriptors other than via the os package. (Any test relying on their behavior is already, uh, “not guaranteed not to be broken”.)

Change https://golang.org/cl/251597 mentions this issue: testing: inject new line if test ends in an incomplete line

What's the resolution to this issue? It's currently marked closed, but I couldn't find a clear explanation of why it was closed. Should downstream test frameworks be handling this in some way? Or was it fixed in some version of Go?

@nicks The issue remains unsolved, but not really sure if someone has the interest to write a fix that well balances the implementation complexity and the actual achievements on solving the problem (and probably the reason that @pete-woods of this issue closes this). As for now, a simple workaround is as you referenced: add a \n if needed.

I actually thought the issue was solved (for the most part) with changes to the way the logs are emitted.

For me, at least, we moved away from parallel tests at CircleCI after finding extremely marginal to negative benefit to using them.