This line is the error when mmap failed. Does the machine you are using not support mmap?

We dropped the fallback path when mmap fails. Maybe we should add them back? @jeremyfaller

Seems to be the OS is returning a different error than one we've seen before. We check for "operation not supported", not "operation not permitted". This error seems related to using CONFIG_STRICT_DEVMEM in the kernel. Although this could result in a game of whack-a-mole with error codes, since this is the only currently reported error, I'll send a CL to also fail gracefully with that error. If any more come up, I'll come up with a better pattern.

** Thinking about this more, and digging into the code more. I think I will default to not failing. Reading about that flag makes me hesitant to accept I have all the failure cases in mind.

The old linker uses a fallback path if mmap fails with any reason. Maybe we want to add that back?

Yes, that's what I'm gonna do.

SGTM. Thanks.

Change https://golang.org/cl/254777 mentions this issue: cmd/link: ignore mmap failures in the linker

Thank you!

Wait, backing up a moment, I don't see at all what this would have to do with CONFIG_STRICT_DEVMEM. We're definitely not mmaping /dev/mem.

@fogfish, it would be helpful to see an strace of the failing link. Could you run go build -work -x hw.go (-work will let you rerun build commands by hand), then copy-paste the WORK=... line at the top into your shell, then finally run strace -f -e mmap,open,openat <link command> with the failing link command and paste the output?

Thank you! I'll try. This is CI/CD instance, which is fully managed by AWS. There are no shell access to it.

Yeah, taking another look, it seems this fails with EPERM, which is rather weird. Looking at the manpages, I don't think any of the EPERM cases of mmap or fallocate can happen in our case. Maybe ftruncate?

@fogfish, thanks. You don't happen to know what file system it's running on, do you (or what your test instance from your initial report was running on)?

It runs on top of Linux Kernel 4.14.171-105.231.amzn1.x86_64

Let me try to reproduce it with given kernel on EC2

If you can run mount, we can also see any special mount options that may be in use.

w.r.t. strace, if you can't trace the WORK= command directly, strace -f -e mmap,open,openat,fallocate,ftruncate go build hw.go should work too, it will just give us a much longer trace to comb through.

The environment is hardened enough and I cannot use sudo, etc. I need more time to find a work around my limits.

strace: exit status 1
strace: test_ptrace_get_syscall_info: PTRACE_TRACEME: Operation not permitted
strace: ptrace(PTRACE_TRACEME, ...): Operation not permitted
+++ exited with 1 +++

Various sandbox environments that restrict the set of permitted system calls can cause a result of EPERM for any system call that is not permitted. Those environments often don't bother to figure out the "right" error to return.

@ianlancetaylor , that's a good point. It wouldn't surprise me at all of fallocate in particular simply wasn't on the allow list.

Indeed fallocate failed: Operation not permitted. It also fails in docker for mounted filesystem but linker work in the docker.

Here is the all info, I've managed to collect about the environment

Linux version 4.14.177-104.253.amzn2.x86_64 (gcc version 7.3.1 20180712 (Red Hat 7.3.1-6) (GCC)) #1 SMP Fri May 1 02:01:13 UTC 2020

/dev/vdd /tmp ext4 rw,relatime,data=writeback 0 0

sh-4.2$ fallocate -l 1K a
fallocate: fallocate failed: Operation not permitted
sh-4.2$ fallocate -x -l 1K a
sh-4.2$


strace: exit status 1
strace: test_ptrace_get_syscall_info: PTRACE_TRACEME: Operation not permitted
strace: ptrace(PTRACE_TRACEME, ...): Operation not permitted
+++ exited with 1 +++

I can mock a simple program that uses some of system calls or build excessive debug logging of cmd/link if you just give an advice what direction to look ignorer to isolate and understand linker issue.

I tend to agree with @ianlancetaylor that this looks like a sandboxed environment. Is this AWS CodePipeline? I've been trying to find docs about its execution environment, though I haven't found much.

Maybe the path forward here is to add EPERM to the allowed errors of fallocate (along with ENOTSUP/EOPNOTSUPP which we already do), but not accepting arbitrary errors (for one, we do want to catch "not enough space" error, which is the whole purpose of fallocate), also not accepting errors on Mmap.

That seems fine; the fallocate is just a hint and if there really is a permission error it should come up again on mmap.

the issue is still waiting for info. How I can help you to dig more info?

@fogfish thanks. I think we have enough information and a proposed solution (#41356 (comment)).