cmd/cgo: cgo is broken with gcc -fdiagnostics-color #40415

eli-schwartz · 2020-07-26T16:45:51Z

What version of Go are you using (`go version`)?

go version go1.14.6 linux/amd64

Does this issue reproduce with the latest release?

see above

What operating system and processor architecture are you using (`go env`)?

see above

What did you do?

In any project, compile cgo stuff using CGO_CFLAGS='-fdiagnostics-color' go build

What did you expect to see?

A successful compile

What did you see instead?

# net
gcc did not produce error at completed:1
on input:

#line 1 "cgo-builtin-prolog"
#include <stddef.h> /* for ptrdiff_t and size_t below */

/* Define intgo when compiling with GCC.  */
typedef ptrdiff_t intgo;

#define GO_CGO_GOSTRING_TYPEDEF
typedef struct { const char *p; intgo n; } _GoString_;
typedef struct { char *p; intgo n; intgo c; } _GoBytes_;
_GoString_ GoString(char *p);
_GoString_ GoStringN(char *p, int l);
_GoBytes_ GoBytes(void *p, int n);
char *CString(_GoString_);
void *CBytes(_GoBytes_);
void *_CMalloc(size_t);

__attribute__ ((unused))
static size_t _GoStringLen(_GoString_ s) { return (size_t)s.n; }

__attribute__ ((unused))
static const char *_GoStringPtr(_GoString_ s) { return s.p; }
#line 10 "/usr/lib/go/src/net/cgo_resnew.go"

#include <sys/types.h>
#include <sys/socket.h>

#include <netdb.h>

#line 1 "cgo-generated-wrapper"
#line 1 "not-declared"
void __cgo_f_1_1(void) { __typeof__(char) *__cgo_undefined__1; }
#line 1 "not-type"
void __cgo_f_1_2(void) { char *__cgo_undefined__2; }
#line 1 "not-int-const"
void __cgo_f_1_3(void) { enum { __cgo_undefined__3 = (char)*1 }; }
#line 1 "not-num-const"
void __cgo_f_1_4(void) { static const double __cgo_undefined__4 = (char); }
#line 1 "not-str-lit"
void __cgo_f_1_5(void) { static const char __cgo_undefined__5[] = (char); }
#line 2 "not-declared"
void __cgo_f_2_1(void) { __typeof__(getnameinfo) *__cgo_undefined__1; }
#line 2 "not-type"
void __cgo_f_2_2(void) { getnameinfo *__cgo_undefined__2; }
#line 2 "not-int-const"
void __cgo_f_2_3(void) { enum { __cgo_undefined__3 = (getnameinfo)*1 }; }
#line 2 "not-num-const"
void __cgo_f_2_4(void) { static const double __cgo_undefined__4 = (getnameinfo); }
#line 2 "not-str-lit"
void __cgo_f_2_5(void) { static const char __cgo_undefined__5[] = (getnameinfo); }
#line 3 "not-declared"
void __cgo_f_3_1(void) { __typeof__(socklen_t) *__cgo_undefined__1; }
#line 3 "not-type"
void __cgo_f_3_2(void) { socklen_t *__cgo_undefined__2; }
#line 3 "not-int-const"
void __cgo_f_3_3(void) { enum { __cgo_undefined__3 = (socklen_t)*1 }; }
#line 3 "not-num-const"
void __cgo_f_3_4(void) { static const double __cgo_undefined__4 = (socklen_t); }
#line 3 "not-str-lit"
void __cgo_f_3_5(void) { static const char __cgo_undefined__5[] = (socklen_t); }
#line 1 "completed"
int __cgo__1 = __cgo__2;

full error output:
not-int-const: In function '__cgo_f_1_3':
not-int-const:1:60: error: invalid type argument of unary '*' (have 'int')
not-num-const: In function '__cgo_f_1_4':
not-num-const:1:73: error: expected expression before ';' token
not-str-lit: In function '__cgo_f_1_5':
not-str-lit:1:73: error: expected expression before ';' token
not-type: In function '__cgo_f_2_2':
not-type:2:39: error: '__cgo_undefined__2' undeclared (first use in this function)
not-type:2:39: note: each undeclared identifier is reported only once for each function it appears in
not-type:2:38: error: invalid operands to binary * (have 'int (*)(const struct sockaddr * restrict,  socklen_t,  char * restrict,  socklen_t,  char * restrict,  socklen_t,  int)' {aka 'int (*)(const struct sockaddr * restrict,  unsigned int,  char * restrict,  unsigned int,  char * restrict,  unsigned int,  int)'} and 'const char *')
not-int-const: In function '__cgo_f_2_3':
not-int-const:2:67: error: invalid operands to binary * (have 'int (*)(const struct sockaddr * restrict,  socklen_t,  char * restrict,  socklen_t,  char * restrict,  socklen_t,  int)' {aka 'int (*)(const struct sockaddr * restrict,  unsigned int,  char * restrict,  unsigned int,  char * restrict,  unsigned int,  int)'} and 'int')
not-num-const: In function '__cgo_f_2_4':
not-num-const:2:67: error: incompatible types when initializing type 'double' using type 'int (*)(const struct sockaddr * restrict,  socklen_t,  char * restrict,  socklen_t,  char * restrict,  socklen_t,  int)' {aka 'int (*)(const struct sockaddr * restrict,  unsigned int,  char * restrict,  unsigned int,  char * restrict,  unsigned int,  int)'}
not-str-lit: In function '__cgo_f_2_5':
not-str-lit:2:67: error: invalid initializer
not-int-const: In function '__cgo_f_3_3':
not-int-const:3:65: error: invalid type argument of unary '*' (have 'int')
not-int-const:3:33: error: enumerator value for '__cgo_undefined__3' is not an integer constant
not-num-const: In function '__cgo_f_3_4':
not-num-const:3:78: error: expected expression before ';' token
not-num-const:3:67: error: initializer element is not constant
not-str-lit: In function '__cgo_f_3_5':
not-str-lit:3:78: error: expected expression before ';' token
not-str-lit:3:67: error: invalid initializer
completed: At top level:
completed:1:16: error: '__cgo__2' undeclared here (not in a function); did you mean '__cgo__1'?
completed:1:16: error: initializer element is not constant

Further info

go/src/cmd/cgo/gcc.go

Lines 372 to 450 in a5a9a06

    
           stderr := p.gccErrors(b.Bytes()) 
        
           if stderr == "" { 
        
           	fatalf("%s produced no output\non input:\n%s", p.gccBaseCmd()[0], b.Bytes()) 
        
           } 
        
           completed := false 
        
           sniff := make([]int, len(names)) 
        
           const ( 
        
           	notType = 1 << iota 
        
           	notIntConst 
        
           	notNumConst 
        
           	notStrLiteral 
        
           	notDeclared 
        
           ) 
        
           sawUnmatchedErrors := false 
        
           for _, line := range strings.Split(stderr, "\n") { 
        
           	// Ignore warnings and random comments, with one 
        
           	// exception: newer GCC versions will sometimes emit 
        
           	// an error on a macro #define with a note referring 
        
           	// to where the expansion occurs. We care about where 
        
           	// the expansion occurs, so in that case treat the note 
        
           	// as an error. 
        
           	isError := strings.Contains(line, ": error:") 
        
           	isErrorNote := strings.Contains(line, ": note:") && sawUnmatchedErrors 
        
           	if !isError && !isErrorNote { 
        
           		continue 
        
           	} 
        
           	c1 := strings.Index(line, ":") 
        
           	if c1 < 0 { 
        
           		continue 
        
           	} 
        
           	c2 := strings.Index(line[c1+1:], ":") 
        
           	if c2 < 0 { 
        
           		continue 
        
           	} 
        
           	c2 += c1 + 1 
        
           	filename := line[:c1] 
        
           	i, _ := strconv.Atoi(line[c1+1 : c2]) 
        
           	i-- 
        
           	if i < 0 || i >= len(names) { 
        
           		if isError { 
        
           			sawUnmatchedErrors = true 
        
           		} 
        
           		continue 
        
           	} 
        
           	switch filename { 
        
           	case "completed": 
        
           		// Strictly speaking, there is no guarantee that seeing the error at completed:1 
        
           		// (at the end of the file) means we've seen all the errors from earlier in the file, 
        
           		// but usually it does. Certainly if we don't see the completed:1 error, we did 
        
           		// not get all the errors we expected. 
        
           		completed = true 
        
           	case "not-declared": 
        
           		sniff[i] |= notDeclared 
        
           	case "not-type": 
        
           		sniff[i] |= notType 
        
           	case "not-int-const": 
        
           		sniff[i] |= notIntConst 
        
           	case "not-num-const": 
        
           		sniff[i] |= notNumConst 
        
           	case "not-str-lit": 
        
           		sniff[i] |= notStrLiteral 
        
           	default: 
        
           		if isError { 
        
           			sawUnmatchedErrors = true 
        
           		} 
        
           		continue 
        
           	} 
        
           	sawUnmatchedErrors = false 
        
           } 
        
           if !completed { 
        
           	fatalf("%s did not produce error at completed:1\non input:\n%s\nfull error output:\n%s", p.gccBaseCmd()[0], b.Bytes(), stderr) 
        
           }

Compiling code with gcc seems to be doing string analysis of compiler output with "expected errors". It fails if the output contains ANSI escape sequences due to the compiler emitting color output. The error message is incredibly vague and doesn't inform the user that an internal implementation detail of the compiler has dumped its internals all over the screen.

This test should probably do more filtering of options around

go/src/cmd/cgo/gcc.go

Line 1977 in a5a9a06

// Optimization options can confuse the error messages; remove them.

The text was updated successfully, but these errors were encountered:

ianlancetaylor · 2020-07-26T20:16:24Z

The cgo tool doesn't really do string analysis of error messages. It just uses the file name and line number. But it does look for the literal string ": error:". It looks like -fdiagnostics-color is messing that up, because color codes are written out after the first colon.

eli-schwartz · 2020-07-26T21:14:51Z

Well, I merely said "string analysis of compiler output", I'd say it counts if it is using string analysis of compiler output to find the literal string : error:. ;)
(I do not know why such a thing is being done, but given I largely lack knowledge of golang outside of trying to build programs written in it for distro repositories... I assume there must be reasons of some sort.)

�[01m�[Knot-str-lit:3:67:�[m�[K �[01;31m�[Kerror: �[m�[Kinvalid initializer
�[01m�[Kcompleted:�[m�[K At top level:
�[01m�[Kcompleted:1:16:�[m�[K �[01;31m�[Kerror: �[m�[K'�[01m�[K__cgo__2�[m�[K' undeclared here (not in a function); did you mean '�[01m�[K__cgo__1�[m�[K'?
�[01m�[Kcompleted:1:16:�[m�[K �[01;31m�[Kerror: �[m�[Kinitializer element is not constant

The file:line:offset: is in bold white, the string error: in red, which makes sense as the trailing : in the former possesses a different context from the latter so it must cross a color boundary.

howjmay · 2020-07-27T16:20:17Z

How about removing the ANSI escape directly, then cgo can return the error message successfully?

eli-schwartz · 2020-07-27T16:42:04Z

That sounds like more work than extending existing functionality for filtering unwanted options. But I suppose that would equally do the trick.

gopherbot · 2020-08-13T20:13:41Z

Change https://golang.org/cl/248398 mentions this issue: cmd/cgo: ensure GCC does not use ANSI escape sequences in errors

ALTree changed the title ~~cgo is broken with gcc -fdiagnostics-color~~ cmd/cgo: cgo is broken with gcc -fdiagnostics-color Jul 26, 2020

ALTree added the NeedsInvestigation label Jul 26, 2020

ianlancetaylor added this to the Backlog milestone Jul 26, 2020

ianlancetaylor added the help wanted label Jul 26, 2020

gopherbot closed this as completed in 2963443 Aug 27, 2020

golang locked and limited conversation to collaborators Aug 27, 2021

gopherbot added the FrozenDueToAge label Aug 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cmd/cgo: cgo is broken with gcc -fdiagnostics-color #40415

cmd/cgo: cgo is broken with gcc -fdiagnostics-color #40415

eli-schwartz commented Jul 26, 2020

ianlancetaylor commented Jul 26, 2020

eli-schwartz commented Jul 26, 2020

howjmay commented Jul 27, 2020 •

edited

Loading

eli-schwartz commented Jul 27, 2020

gopherbot commented Aug 13, 2020

cmd/cgo: cgo is broken with gcc -fdiagnostics-color #40415

cmd/cgo: cgo is broken with gcc -fdiagnostics-color #40415

Comments

eli-schwartz commented Jul 26, 2020

What version of Go are you using (go version)?

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

What did you do?

What did you expect to see?

What did you see instead?

Further info

ianlancetaylor commented Jul 26, 2020

eli-schwartz commented Jul 26, 2020

howjmay commented Jul 27, 2020 • edited Loading

eli-schwartz commented Jul 27, 2020

gopherbot commented Aug 13, 2020

What version of Go are you using (`go version`)?

What operating system and processor architecture are you using (`go env`)?

howjmay commented Jul 27, 2020 •

edited

Loading