Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/cgo: fails to build after updating to Mojave #27921

Closed
Elixworks opened this issue Sep 28, 2018 · 59 comments
Closed

runtime/cgo: fails to build after updating to Mojave #27921

Elixworks opened this issue Sep 28, 2018 · 59 comments
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin
Milestone

Comments

@Elixworks
Copy link

What version of Go are you using (go version)?

go version go1.11 darwin/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/foo/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Volumes/DailySD/SD_Dev"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/fj/yby80m4x1pq1nzvfvd1hplhw0000gn/T/go-build110153853=/tmp/go-build -gno-record-gcc-switches -fno-common"

GCC

Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include/c++/4.2.1
Apple LLVM version 10.0.0 (clang-1000.11.45.2)
Target: x86_64-apple-darwin18.0.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

What did you do?

update macOS to mojave,
install go, vscode for Developement.

An error that did not get offended when go test was done within an existing project occurred. It is below.

# runtime/cgo
_cgo_export.c:3:10: fatal error: 'stdlib.h' file not found

If possible, provide a recipe for reproducing the error.
A complete runnable program is good.
A link on play.golang.org is best.

What is missing, please help me.

@ianlancetaylor
Copy link
Contributor

What, precisely, did you do?

@ianlancetaylor ianlancetaylor changed the title go test: runtime/cgo fatal error by MacOS mojave, vscode runtime/cgo: fails to build after updating to Mojave Sep 28, 2018
@ianlancetaylor ianlancetaylor added OS-Darwin NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Sep 28, 2018
@ianlancetaylor ianlancetaylor added this to the Go1.12 milestone Sep 28, 2018
@ianlancetaylor
Copy link
Contributor

The error certainly looks like a problem with your xcode installation. It looks like your C compiler can not find the standard header files.

@Elixworks
Copy link
Author

$ dlv debug
$ go test 

such errors, thx.

runtime/cgo

_cgo_export.c:3:10: fatal error: 'stdlib.h' file not found
exit status 2

$ sudo rm -rf /Library/Developer/CommandLineTools
$ xcode-select --install

But the error does not go away. TT

@ianlancetaylor
Copy link
Contributor

What happens if you put this code into a file named hello.c and run clang hello.c?

#include <stdlib.h>
int main() { exit(0); }

@Elixworks
Copy link
Author

Elixworks commented Sep 28, 2018

hello.c:1:10: fatal error: 'stdlib.h' file not found
#include <stdlib.h>
         ^~~~~~~~~~
1 error generated.

done.

@ianlancetaylor
Copy link
Contributor

OK, you need to fix that problem somehow. That is a problem with your C compiler. It is not a problem with Go. There is nothing that we can change in the Go tools to fix that. Unfortunately I have no idea what the problem is. I'm going to close this issue since there is nothing we can do. Please comment if you disagree.

@wujunze
Copy link

wujunze commented Nov 13, 2018

Me too @Elixworks

@wujunze
Copy link

wujunze commented Nov 13, 2018

You can try xcode-select --install

@fredcarle
Copy link

I have the same issue and xcode-select --install did not solve the problem

@fredcarle
Copy link

Found the solution with this one

@mimoo
Copy link

mimoo commented Dec 26, 2018

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

@mrtuborg
Copy link

@mimoo, you saved my day, thanks :)

@DrAuYueng
Copy link

You can try xcode-select --install

worked for me

@cnjsstong
Copy link

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

This does the magic.

@kongque
Copy link

kongque commented Feb 26, 2019

xcode-select --install

worked for me

@kamal-github
Copy link

I was seeing the same issue but that was due to GCC, which I install using apk add build-base

@micklove
Copy link

FYI - Had the same issue, tried everything above

However, the file, mentioned in comments above...

/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

seems to be no longer available, with the latest version of xcode, see the New Features section here
https://developer.apple.com/documentation/xcode_release_notes/xcode_10_release_notes


Solution:

Running brew doctor, as per the instructions below...

Warning: Unbrewed header files were found in /usr/local/include.
If you didn't put them there on purpose they could cause problems when
building Homebrew formulae, and may need to be deleted.

Unexpected header files:
  /usr/local/include/abstract.h
  /usr/local/include/aes.h
  /usr/local/include/anchor.h
etc...

So, using the script provided in the link below, which simply moves the files one at a time to another folder, fixed the issue for me
SOHU-Co/kafka-node#881 (comment)

Repeated below...

mkdir /tmp/includes
brew doctor 2>&1 | grep "/usr/local/include" | awk '{$1=$1;print}' | xargs -I _ mv _ /tmp/includes

@wujunze
Copy link

wujunze commented Apr 8, 2019

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

Nice

@cuiwm
Copy link

cuiwm commented Apr 8, 2019

on mojave 10.14.4, it works. command line> open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

@xbeta
Copy link

xbeta commented Apr 11, 2019

open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

This fix it for me!

@raxod502
Copy link

raxod502 commented Jun 7, 2019

Unfortunately, that package appears to no longer exist as of a recent update. What can I do?

% ls -lA /usr/include
ls: /usr/include: No such file or directory
% ls -lA /Library/Developer/CommandLineTools
total 0
drwxr-xr-x  6 root  admin  192 Jun  4 09:08 Library
drwxr-xr-x  6 root  wheel  192 Jun  4 09:08 SDKs
drwxr-xr-x  7 root  admin  224 Apr  5 17:30 usr
% ls -lA /Library/Developer/CommandLineTools/Packages
ls: /Library/Developer/CommandLineTools/Packages: No such file or directory
% xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" to install updates

@ianlancetaylor I totally am on your side here, but unfortunately Apple has made it clear they want every project to update their build tooling to work with the weird new SDK system:

some software may fail to build correctly against the SDK and require macOS headers to be installed in the base system under /usr/include. If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so.

So unfortunately I think this is a bug in Go, unless you want to drop support for macOS.

@raxod502
Copy link

raxod502 commented Jun 7, 2019

Solution: go to https://developer.apple.com/download/more/ (Apple Developer account required) and download "Command Line Tools (macOS 10.14) for Xcode 10.2.1" (or equivalent for your OS version). Extract that package, which installs the macOS_SDK_headers_for_macOS_10.14.pkg package to /Library/Developer/CommandLineTools/Packages, which you can then extract separately. That finally gets you your /usr/include back.

@ianlancetaylor
Copy link
Contributor

@raxod502 Go doesn't care where the header file is. It only cares that it can pass

#include <stdlib.h>

to the C compiler and have the C compiler work. If that doesn't work, there is nothing we can do in Go to fix that.

@raxod502
Copy link

raxod502 commented Jun 8, 2019

@ianlancetaylor I understand what you are saying, but I think Apple's implication is that the Go build system needs to pass an extra -I argument to the C compiler with the path to the macOS SDK. Yes, that's completely ridiculous -- but what is the alternative? Every single user of Go on macOS needs to export C_INCLUDE_PATH, or ...?

It is an unfortunate fact that the C compiler cannot find stdlib.h on macOS anymore, at least by default. So while it makes perfect sense to say "it is your own responsibility to make sure the compiler can find stdlib.h", the sad truth is that this means every single Go developer on macOS would be required to take manual steps to get Go working properly.

An alternative solution would be for Homebrew to do something to fix this problem in general, but I don't know what the options look like for that.

@ianlancetaylor
Copy link
Contributor

Are you saying that every Makefile for a C program needs to add a -I option to find the standard header files on macOS?

@timwangmusic
Copy link

Shortly the way it worked on macOS Catalina in my case:

export CPATH="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/"
export CGO_ENABLED=1; export CC=gcc;

doesn't work in my case

@mutescent
Copy link

Shortly the way it worked on macOS Catalina in my case:

export CPATH="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/"
export CGO_ENABLED=1; export CC=gcc;

This works for me, using Catalina.

@fredcarle
Copy link

Shortly the way it worked on macOS Catalina in my case:

export CPATH="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/"
export CGO_ENABLED=1; export CC=gcc;

Thanks @shalakhin! This works for me in Catalina

@damienstanton
Copy link

Shortly the way it worked on macOS Catalina in my case:

export CPATH="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/usr/include/"
export CGO_ENABLED=1; export CC=gcc;

This does not work for me, using Go 1.12.14.

@urjitbhatia
Copy link

Removing brew llvm helped me. brew remove llvm I think it was not searching for the headers in the right spot.

@ogreface
Copy link

@urjitbhatia Thanks! Same issue here on Catalina. xcode-select --install && brew remove llvm got me going.

@newbieh4cker
Copy link

if someone facing this issue in catalina (10.15)
try this.
export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"

@gezibash
Copy link

xcode-select --install gave me
xcode-select: error: command line tools are already installed, use "Software Update" to install updates.

brew remove llvm worked and solved the issue

@chetanyakan
Copy link

Maybe this can help someone: mattn/go-sqlite3#481 (comment)

@hakkiyagiz
Copy link

Removing brew llvm helped me. brew remove llvm I think it was not searching for the headers in the right spot.

worked like a charm! thanks

@starlightme
Copy link

if someone facing this issue in catalina (10.15)
try this.
export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"

Thanks a lot! This work for me!

@blackbeans
Copy link

it works for me (Catalina 15.6)

brew remove llvm

@malajisi
Copy link

malajisi commented Oct 6, 2020

It's all related to c compiler clang... I found the clue from the discussion above and solved my issue( I install older version clang6 pkg and leave a export in .bash_profile), all these caused by clang. So
FIRST check the environment in terminal with
clang -v
If it's too old. Change to use the newest Xcode installed clang, for macOS Mojave maybe 10. All will done.

@tempcke
Copy link

tempcke commented Nov 2, 2020

So I had the same/very similar issue running linux mint and this resolved the issue for me

sudo apt update
sudo apt-get install build-essential

I'm posting this here because this is the number one search result on google, so others searching will land here ..

@bobbyz3g
Copy link

It works for me (Big Sur 11.1)

xcode-select --install
brew remove llvm

@jaehoonkimm
Copy link

jaehoonkimm commented Jan 9, 2021

It works for me because I've used clang in anaconda. (macOS Big Sur 11.1)

which clang

/Users/username/anaconda3/bin/clang

rm /Users/username/anaconda3/bin/clang
which clang

/usr/bin/clang

@jbarfield
Copy link

jbarfield commented Apr 1, 2021

For me on Mac OS Catalina:

I found that my .zshenv "PATH=" variable, had non-standard bin paths set before the MacOS native bin path. Like so:

export PATH=${GOPATH}/bin:${HOME}/pkg/sbin:${HOME}/pkg/bin:/opt/local/sbin:/opt/local/bin:/opt/pkg/bin:/opt/pkg/sbin:${HOME}/Library/Python/3.8/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/Library/Apple/usr/bin:${HOME}/bin

Simply re-ordering the PATH fixed this:

export PATH=${GOPATH}/bin:${HOME}/pkg/sbin:${HOME}/pkg/bin:/opt/local/sbin:/opt/local/bin:${HOME}/Library/Python/3.8/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/MacGPG2/bin:/Library/Apple/usr/bin:${HOME}/bin:/opt/pkg/bin:/opt/pkg/sbin

As you can see I moved /opt/pkg/{bin,sbin} to the end of the line.

To find out if this is your problem:

Open terminal, run:
which clang
if it returns any bin path other than /usr/bin/clang than you are not invoking Apples clang, you are invoking a clang installed by another package manager. Rather than remove or unlink the package managers version just make their compilers last in the PATH search list.

Apples clang is located in the system path /usr/bin. "bash/zsh", etc, finds commands in the order that the PATH variable is set meaning that the shell takes the first binary matching the word that it finds no matter the path it is sitting in.

After you update .zshenv or .bashrc (.bash_profile), etc, run:
source ~/.zshenv or source ~/.bashrc and then try to build the go source again by running:
make build or make test
command again in the go src folder. I had to run the commands with sudo or it failed with permissions errors on some libraries. likely system lib's.

In my case, I have brew and macports installed, but I rarely use those two because I am an avid pkgsrc user. So /opt/pkg/bin was set by me when I deployed Joyents pkgsrc bootstrap. Therefore changing my path order, In my case at least, can break other pkg's that I have installed that I want prioritised over Apples.

I tried to resolve this by running
export CC=/usr/bin/clang and export CXX=/usr/bin/clang
but GO doesn't seem to care about these variables. I could be setting them wrong but IMO I think this could be a go bug on MacOS.

@ex-tag
Copy link

ex-tag commented May 2, 2021

It works for me because I've used clang in anaconda. (macOS Big Sur 11.1)

which clang

/Users/username/anaconda3/bin/clang

rm /Users/username/anaconda3/bin/clang
which clang

/usr/bin/clang

This fixed my issue. The path for Clang was coming up as /user/local/bin/clang, instead of /user/bin/clang, guessing because I had once used Homebrew to install a version of Clang.

@duval1024
Copy link

reinstall x-code commandline tools :

sudo rm -rf  /Library/Developer/CommandLineTools
sudo xcode-select --install

and then

 open /Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

it work for me !

@KingSun0
Copy link

KingSun0 commented Dec 1, 2021

clang的版本问题,我修改为xcode的clang

@iOSPrincekin
Copy link

if someone facing this issue in catalina (10.15)
try this.
export SDKROOT="$(xcrun --sdk macosx --show-sdk-path)"

Thanks a lot! This work for me!

set Environment Varibles in xcode clang target arguments:

SDKROOT=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX12.3.sdk

worked for me!

@matronator
Copy link

@ianlancetaylor wrote:

OK, you need to fix that problem somehow. That is a problem with your C compiler. It is not a problem with Go. There is nothing that we can change in the Go tools to fix that. Unfortunately I have no idea what the problem is. I'm going to close this issue since there is nothing we can do. Please comment if you disagree.

Apple says:

The Command Line Tools package installs the macOS system headers inside the macOS SDK. Software that compiles with the installed tools will search for headers within the macOS SDK provided by either Xcode at:
/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk
or the Command Line Tools at:
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk
depending on which is selected using xcode-select.

The command line tools will search the SDK for system headers by default. However, some software may fail to build correctly against the SDK and require macOS headers to be installed in the base system under /usr/include. If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so. As a workaround, an extra package is provided which will install the headers to the base system. In a future release, this package will no longer be provided. You can find this package at:
/Library/Developer/CommandLineTools/Packages/macOS_SDK_headers_for_macOS_10.14.pkg

To make sure that you’re using the intended version of the command line tools, run xcode-select -s or xcode select -s /Library/Developer/CommandLineTools after installing.

Notice this part:

If you are the maintainer of such software, we encourage you to update your project to work with the SDK or file a bug report for issues that are preventing you from doing so.

Is there really nothing that can be done in the Go tools to fix this issue?

@ianlancetaylor
Copy link
Contributor

Yes, there is really nothing that can be done in the Go tools. The Go tools never read system headers themselves, and they never read /usr/include. Instead, they invoke the C compiler, and expect the C compiler to read those headers. You can see the C compiler that they will invoke by using go env CC, and you can see precisely what the Go tool does by using go build -x.

@matronator
Copy link

Yes, there is really nothing that can be done in the Go tools. The Go tools never read system headers themselves, and they never read /usr/include. Instead, they invoke the C compiler, and expect the C compiler to read those headers. You can see the C compiler that they will invoke by using go env CC, and you can see precisely what the Go tool does by using go build -x.

Got it. Thanks for the reply :)

@xiliangMa
Copy link

You can try xcode-select --install

Thanks a lot! This work for me!

@BigCatGit
Copy link

thanks

@akuzni2

This comment was marked as off-topic.

@golang golang locked as resolved and limited conversation to collaborators Sep 20, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Darwin
Projects
None yet
Development

No branches or pull requests