Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/cgo: missing -lmsvcrt causes sprintf misbehaving on Windows #12030

Closed
xicheng opened this issue Aug 5, 2015 · 24 comments
Closed

runtime/cgo: missing -lmsvcrt causes sprintf misbehaving on Windows #12030

xicheng opened this issue Aug 5, 2015 · 24 comments

Comments

@xicheng
Copy link

xicheng commented Aug 5, 2015

print any number aways get 'g' when I use golua,
I found it is caused by sprinf.

in lua source code:

define LUA_NUMBER_SCAN "%lf"

define LUA_NUMBER_FMT "%.14g"

define lua_number2str(s,n) sprintf((s), LUA_NUMBER_FMT, (n))

define LUAI_MAXNUMBER2STR 32 /* 16 digits, sign, point, and \0 */

so ,I wrote a test code:
/*

include <stdio.h>

void my_print(float a)
{
char buff[32];
sprintf(buff,"%.14g",a);
printf("%s\n",buff);
}
*/

C.my_print(20.5)

it output g

i use go1.5 beta3 on win7 64

@ianlancetaylor
Copy link
Contributor

Please show us a complete standalone Go example. Thanks.

I don't understand what Lua has to do with this.

@mikioh mikioh changed the title cgo sprintf runtime/cgo: sprintf using Lua C API via cgo Aug 5, 2015
@xicheng
Copy link
Author

xicheng commented Aug 6, 2015

package main

/*
#include "stdio.h"

void my_print(float a)
{
char buff[32];
sprintf(buff,"%.14g",a);
printf("%s\n",buff);
}
*/
import "C"

func main() {
C.my_print(20.5)
}

@ianlancetaylor
Copy link
Contributor

When I run that program it prints "20.5", which is what I expect.

Are you saying that it prints something different on your system?

@rsc rsc changed the title runtime/cgo: sprintf using Lua C API via cgo runtime/cgo: sprintf misbehaving Aug 6, 2015
@rsc rsc added this to the Go1.5Maybe milestone Aug 6, 2015
@lixin9311
Copy link

same issue with go 1.5 rc1 and mingw64 on Win7-64.

@ianlancetaylor
Copy link
Contributor

@lixin9311 What does the program print on your system?

@lixin9311
Copy link

@ianlancetaylor
g

printf works well. sprintf is misbehaving.

@ianlancetaylor
Copy link
Contributor

@lixin9311 Please provide a complete standalone program, please tell us what you expect it to do, and please tell us what it actually does do. Thanks.

@lixin9311
Copy link

same code as @xicheng provided.
it runs well under linux, but output 'g' in windows.

@ianlancetaylor
Copy link
Contributor

What does the equivalent C program print on Windows?

@lixin9311
Copy link

20.5

On Mon, Aug 10, 2015 at 4:12 PM, Ian Lance Taylor notifications@github.com
wrote:

What does the equivalent C program print on Windows?


Reply to this email directly or view it on GitHub
#12030 (comment).

@rsc
Copy link
Contributor

rsc commented Aug 10, 2015

I have reproduced this on a Windows machine. Nice.

@rsc
Copy link
Contributor

rsc commented Aug 10, 2015

This is kind of amazing. Here is a C program:

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <windows.h>

DWORD WINAPI thread(LPVOID x) {
    char buf[128];
    double d = 1.0;

    printf("printf: d=%g\n", d);
    sprintf(buf, "sprintf: d=%g\n", d);
    printf("%s", buf);

    return 0;
}

int main(void)
{
    HANDLE h;
    h = CreateThread(NULL, 0, &thread, 0, 0, NULL);
    WaitForSingleObject(h, INFINITE);
    CloseHandle(h);
    return 0;
}

It prints:

printf: d=1
sprintf: d=1

Here is a very similar Go program:

package main

/*

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <windows.h>

DWORD WINAPI thread(LPVOID x) {
    char buf[128];
    double d = 1.0;

    printf("printf: d=%g\n", d);
    sprintf(buf, "sprintf: d=%g\n", d);
    printf("%s", buf);

    return 0;
}

int cmain(void)
{
    HANDLE h;
    h = CreateThread(NULL, 0, &thread, 0, 0, NULL);
    WaitForSingleObject(h, INFINITE);
    CloseHandle(h);
    return 0;
}

*/
import "C"
import "runtime"

func init() {
    runtime.LockOSThread()
}

func main() {
    C.cmain()
}

It prints:

printf: d=1
sprintf: d=g

The use of a separate thread to run the code was meant to make sure that Go execution had not somehow poisoned that thread. The thread running the code in this case was created by Windows and has never been used for anything but C. Even so sprintf doesn't work. No matter what you try, %g formats as "g". Similarly, %e formats as "e" and %f formats as "f".

I found that if you try to print 0 with %q (not valid) you get just "q" out, much like %g producing just "g". I wonder if the default case in this printf is to drop the %, as if %% were generalized. Then the question is just why %e, %f, %g are invalid like %q in a Go binary, but valid in a C binary. And why they are invalid in sprintf but valid in printf.

On 32-bit x86 systems Go changes the FPU mode at startup, but there is no such change on 64-bit systems. And again printf works while sprintf does not.

I also tried snprintf (fails the same way) and sprintf_s (returns -1, indicating some kind of error, but we knew that).

It's quite the mystery.

@rsc rsc changed the title runtime/cgo: sprintf misbehaving runtime/cgo: sprintf misbehaving on Windows Aug 10, 2015
@rsc
Copy link
Contributor

rsc commented Aug 10, 2015

This appears to be caused by external linking. Compiling with -ldflags=-linkmode=internal makes sprintf start working again.

It's hard to say how serious this is. It seems like something very fundamental must be wrong; the question is how much it affects. If it really is serious then we could change the default link mode on windows back to internal linking (external linking support is new in this release).

How important is external linking?

/cc @alexbrainman @minux

@rsc rsc changed the title runtime/cgo: sprintf misbehaving on Windows runtime/cgo: sprintf misbehaving on Windows with external linking Aug 10, 2015
@rsc
Copy link
Contributor

rsc commented Aug 10, 2015

Confirmed with git bisect that this program stopped working when external linking was added in CL 7534.

package main

/*

#include <stdio.h>
#include <string.h>
#include <stdlib.h>
#include <windows.h>

int cmain(void)
{
    char buf[128];
    double d = 1.0;

    printf("printf: d=%g\n", d);
    sprintf(buf, "sprintf: d=%g\n", d);
    printf("%s", buf);

    return 0;
}

*/
import "C"
import "runtime"

func init() {
    runtime.LockOSThread()
}

func main() {
    C.cmain()
}

@alexbrainman
Copy link
Member

We have resolved a lot of windows cgo issues with external linking. I think external linking is good default.

I think we need to understand what this problem is before deciding what to do about it. I am fighting with issue #10776 at this moment. I will try to debug this when I have time. I am sure Minux might have some ideas too.

Alex

@rsc rsc modified the milestones: Go1.6Early, Go1.5Maybe Aug 18, 2015
@minux minux modified the milestones: Go1.5.2, Go1.6Early Sep 10, 2015
@minux
Copy link
Member

minux commented Sep 10, 2015

The real cause is actually not that serious.

ntdll.dll also defines a sprintf function, during internal linking,
the sprintf is linked to msvcrt.dll, however, with external linking,
sprintf is linked to ntdll.dll (because we explicitly pass -lntdll to
external linker.)

Try this to reproduce the issue with C:

$ type b.c
#include <stdio.h>
int main() {
char buf[128];
sprintf(buf, "d=%g", 1.0);
puts(buf);
}
$ gcc -o b.exe b.c
$ b.exe
d=1
$ gcc -o b.exe b.c -lntdll
$ b.exe
d=XXX

(XXX is certain random garbage, it probably is system dependent.)

@alexbrainman
Copy link
Member

Good catch @minux . Do you know how to fix that?

Alex

@minux
Copy link
Member

minux commented Sep 10, 2015 via email

@alexbrainman
Copy link
Member

And then something else will break. Can we supply dll names as part of obj file instead somehow?

Alex

@minux
Copy link
Member

minux commented Sep 10, 2015 via email

@alexbrainman
Copy link
Member

I don't know if we can request a symbol from a specific dll in COFF.

How does ldpe function determines which dlls to call? Surely it is there in pe symbol table. Or maybe in import section.

... to remove the dependency on NtWaitForSingleObject. ...

As far as I remember rsc added call to NtWaitForSingleObject in usleep because he didn't have enough stack to call standard runtime.stdcall function (back in the days when runtime was written in C). It is also little bit quicker (because it does not need runtime.cgocall and friends). I wouldn't bother changing it.

Alex

@gopherbot
Copy link

CL https://golang.org/cl/14472 mentions this issue.

@minux minux self-assigned this Sep 10, 2015
@minux minux closed this as completed in 0b5bcf5 Sep 12, 2015
@rsc rsc changed the title runtime/cgo: sprintf misbehaving on Windows with external linking runtime/cgo: missing -lmsvcrt causes sprintf misbehaving on Windows Nov 13, 2015
@gopherbot
Copy link

CL https://golang.org/cl/16966 mentions this issue.

minux added a commit that referenced this issue Nov 17, 2015
It's because runtime links to ntdll, and ntdll exports a couple
incompatible libc functions. We must link to msvcrt first and
then try ntdll.

Fixes #12030.

Change-Id: I0105417bada108da55f5ae4482c2423ac7a92957
Reviewed-on: https://go-review.googlesource.com/14472
Reviewed-by: Alex Brainman <alex.brainman@gmail.com>
Run-TryBot: Minux Ma <minux@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-on: https://go-review.googlesource.com/16966
Run-TryBot: Austin Clements <austin@google.com>
Reviewed-by: Minux Ma <minux@golang.org>
Reviewed-by: Russ Cox <rsc@golang.org>
@gopherbot
Copy link

CL https://golang.org/cl/30737 mentions this issue.

gopherbot pushed a commit that referenced this issue Oct 11, 2016
CL 14472 solved issue #12030 by explicitly linking msvcrt.dll
to every cgo executable we build. This CL achieves the same
by manually loading ntdll.dll during startup.

Updates #12030

Change-Id: I5d9cd925ef65cc34c5d4031c750f0f97794529b2
Reviewed-on: https://go-review.googlesource.com/30737
Run-TryBot: Ian Lance Taylor <iant@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Reviewed-by: Minux Ma <minux@golang.org>
@golang golang locked and limited conversation to collaborators Oct 11, 2017
@rsc rsc unassigned minux Jun 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests

7 participants