Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gollvm: stack splitting support is not detected,on Aarch64 (GCC, Clang). False negatives during mmap detection #43772

Closed
advancedwebdeveloper opened this issue Jan 19, 2021 · 11 comments
Closed

gollvm: stack splitting support is not detected,on Aarch64 (GCC, Clang). False negatives during mmap detection #43772

advancedwebdeveloper opened this issue Jan 19, 2021 · 11 comments
Labels
FrozenDueToAge
Milestone
gollvm

Comments

@advancedwebdeveloper
Copy link

This is the first issue, which was tacked on Aarch64 servers, provided by Equinix .
CC'ing to @vielmetti , since he is helping with servers, on behalf of gollvm project/Aarch64 port.

Edward, this issue is related to
https://llvm.org/docs/SegmentedStacks.html#:~:text=Segmented%20stack%20allows%20stack%20space,into%20a%20doubly%20linked%20list.
https://gcc.gnu.org/wiki/SplitStacks
. So it is missing critical/being a significant blocker.

Actually I was surprised about it - I thought building on 64bit ARM hardware was possible.

Here is my setup, for now:

$ lscpu
Architecture: aarch64
CPU op-mode(s): 64-bit
Byte Order: Little Endian
CPU(s): 96
On-line CPU(s) list: 0-95
Thread(s) per core: 1
Core(s) per socket: 48
Socket(s): 2
NUMA node(s): 2
Vendor ID: Cavium
Model: 1
Model name: ThunderX 88XX
Stepping: 0x1
BogoMIPS: 200.00
NUMA node0 CPU(s): 0-47
NUMA node1 CPU(s): 48-95
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Vulnerable
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Vulnerable
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid

Checking the linker:

$ gold --version
GNU gold (GNU Binutils for Ubuntu 2.34) 1.16
; and another one:
$ ld --version
GNU ld (GNU Binutils for Ubuntu) 2.34
.

Checking the OS:

$ uname -a
Linux ams1-c1.large.arm-01 5.4.0-40-generic #44-Ubuntu SMP Mon Jun 22 23:59:48 UTC 2020 aarch64 aarch64 aarch64
. In detail:

$ cat /etc/release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=20.04
DISTRIB_CODENAME=focal
DISTRIB_DESCRIPTION="Ubuntu 20.04 LTS"
.
Checking compilers:

$ gcc --version
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0

$ clang-11 --version
Ubuntu clang version 11.0.0-2~ubuntu20.04.1
Target: aarch64-unknown-linux-gnu

$ llc-11 --version
LLVM (http://llvm.org/):
LLVM version 11.0.0

Optimized build.
Default target: aarch64-unknown-linux-gnu
Host CPU: thunderxt88

The full list of installed package could be checked here
.

Here is how I am able to track the issue:

$ cmake -DLLVM_TARGETS_TO_BUILD=Aarch64 -DLLVM_USE_LINKER=gold -G Ninja ../llvm-project/llvm
-- The C compiler identification is GNU 9.3.0
-- The CXX compiler identification is GNU 9.3.0
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- No build type selected, default to Debug
-- Looking for dlfcn.h
-- Looking for dlfcn.h - found
-- Looking for errno.h
-- Looking for errno.h - found
-- Looking for fcntl.h
-- Looking for fcntl.h - found
-- Looking for link.h
-- Looking for link.h - found
-- Looking for malloc/malloc.h
-- Looking for malloc/malloc.h - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Looking for signal.h
-- Looking for signal.h - found
-- Looking for sys/ioctl.h
-- Looking for sys/ioctl.h - found
-- Looking for sys/mman.h
-- Looking for sys/mman.h - found
-- Looking for sys/param.h
-- Looking for sys/param.h - found
-- Looking for sys/resource.h
-- Looking for sys/resource.h - found
-- Looking for sys/stat.h
-- Looking for sys/stat.h - found
-- Looking for sys/time.h
-- Looking for sys/time.h - found
-- Looking for sys/types.h
-- Looking for sys/types.h - found
-- Looking for sysexits.h
-- Looking for sysexits.h - found
-- Looking for termios.h
-- Looking for termios.h - found
-- Looking for unistd.h
-- Looking for unistd.h - found
-- Looking for valgrind/valgrind.h
-- Looking for valgrind/valgrind.h - not found
-- Looking for fenv.h
-- Looking for fenv.h - found
-- Looking for FE_ALL_EXCEPT
-- Looking for FE_ALL_EXCEPT - found
-- Looking for FE_INEXACT
-- Looking for FE_INEXACT - found
-- Looking for mach/mach.h
-- Looking for mach/mach.h - not found
-- Looking for histedit.h
-- Looking for histedit.h - not found
-- Looking for CrashReporterClient.h
-- Looking for CrashReporterClient.h - not found
-- Looking for linux/magic.h
-- Looking for linux/magic.h - found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Looking for pthread_getspecific in pthread
-- Looking for pthread_getspecific in pthread - found
-- Looking for pthread_rwlock_init in pthread
-- Looking for pthread_rwlock_init in pthread - found
-- Looking for pthread_mutex_lock in pthread
-- Looking for pthread_mutex_lock in pthread - found
-- Looking for dlopen in dl
-- Looking for dlopen in dl - found
-- Looking for clock_gettime in rt
-- Looking for clock_gettime in rt - found
-- Looking for pfm_initialize in pfm
-- Looking for pfm_initialize in pfm - not found
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Failed
-- Looking for pthread_create in pthreads
-- Looking for pthread_create in pthreads - not found
-- Looking for pthread_create in pthread
-- Looking for pthread_create in pthread - found
-- Found Threads: TRUE
-- Found ZLIB: /usr/lib/aarch64-linux-gnu/libz.so (found version "1.2.11")
-- Looking for compress2
-- Looking for compress2 - found
-- Could NOT find LibXml2 (missing: LIBXML2_LIBRARY LIBXML2_INCLUDE_DIR)
-- Looking for xar_open in xar
-- Looking for xar_open in xar - not found
-- Looking for arc4random
-- Looking for arc4random - not found
-- Looking for backtrace
-- Looking for backtrace - found
-- backtrace facility detected in default set of libraries
-- Found Backtrace: /usr/include
-- Performing Test C_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW
-- Performing Test C_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW - Failed
-- Looking for __register_frame
-- Looking for __register_frame - found
-- Looking for __deregister_frame
-- Looking for __deregister_frame - found
-- Looking for _Unwind_Backtrace
-- Looking for _Unwind_Backtrace - found
-- Looking for getpagesize
-- Looking for getpagesize - found
-- Looking for sysconf
-- Looking for sysconf - found
-- Looking for getrusage
-- Looking for getrusage - found
-- Looking for setrlimit
-- Looking for setrlimit - found
-- Looking for isatty
-- Looking for isatty - found
-- Looking for futimens
-- Looking for futimens - found
-- Looking for futimes
-- Looking for futimes - found
-- Looking for posix_fallocate
-- Looking for posix_fallocate - found
-- Looking for sigaltstack
-- Looking for sigaltstack - found
-- Looking for lseek64
-- Looking for lseek64 - found
-- Looking for mallctl
-- Looking for mallctl - not found
-- Looking for mallinfo
-- Looking for mallinfo - found
-- Looking for malloc_zone_statistics
-- Looking for malloc_zone_statistics - not found
-- Looking for getrlimit
-- Looking for getrlimit - found
-- Looking for posix_spawn
-- Looking for posix_spawn - found
-- Looking for pread
-- Looking for pread - found
-- Looking for sbrk
-- Looking for sbrk - found
-- Looking for strerror
-- Looking for strerror - found
-- Looking for strerror_r
-- Looking for strerror_r - found
-- Looking for strerror_s
-- Looking for strerror_s - not found
-- Looking for setenv
-- Looking for setenv - found
-- Looking for dlopen
-- Looking for dlopen - found
-- Looking for dladdr
-- Looking for dladdr - not found
-- Performing Test HAVE_STRUCT_STAT_ST_MTIMESPEC_TV_NSEC
-- Performing Test HAVE_STRUCT_STAT_ST_MTIMESPEC_TV_NSEC - Failed
-- Performing Test HAVE_STRUCT_STAT_ST_MTIM_TV_NSEC
-- Performing Test HAVE_STRUCT_STAT_ST_MTIM_TV_NSEC - Success
-- Looking for GLIBC
-- Looking for GLIBC - found
-- Looking for pthread_getname_np
-- Looking for pthread_getname_np - found
-- Looking for pthread_setname_np
-- Looking for pthread_setname_np - found
-- Performing Test HAVE_STD_IS_TRIVIALLY_COPYABLE
-- Performing Test HAVE_STD_IS_TRIVIALLY_COPYABLE - Success
-- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB
-- Performing Test HAVE_CXX_ATOMICS_WITHOUT_LIB - Success
-- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB
-- Performing Test HAVE_CXX_ATOMICS64_WITHOUT_LIB - Success
-- Performing Test LLVM_HAS_ATOMICS
-- Performing Test LLVM_HAS_ATOMICS - Success
-- Performing Test SUPPORTS_VARIADIC_MACROS_FLAG
-- Performing Test SUPPORTS_VARIADIC_MACROS_FLAG - Success
-- Performing Test SUPPORTS_GNU_ZERO_VARIADIC_MACRO_ARGUMENTS_FLAG
-- Performing Test SUPPORTS_GNU_ZERO_VARIADIC_MACRO_ARGUMENTS_FLAG - Failed
-- Performing Test HAS_MAYBE_UNINITIALIZED
-- Performing Test HAS_MAYBE_UNINITIALIZED - Success
-- Native target AArch64 is not selected; lli will not JIT code
-- Threads enabled.
-- Doxygen disabled.
-- Go bindings disabled.
-- Ninja version: 1.10.0.git.kitware.jobserver-1
-- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH)
-- Could NOT find OCaml (missing: OCAMLFIND OCAML_VERSION OCAML_STDLIB_PATH)
-- OCaml bindings disabled.
-- Could NOT find Python module pygments
-- Could NOT find Python module pygments.lexers.c_cpp
-- Could NOT find Python module yaml
-- LLVM host triple: aarch64-unknown-linux-gnu
-- LLVM default target triple: aarch64-unknown-linux-gnu
-- Performing Test CXX_SUPPORTS_CUSTOM_LINKER
-- Performing Test CXX_SUPPORTS_CUSTOM_LINKER - Success
-- Performing Test C_SUPPORTS_FPIC
-- Performing Test C_SUPPORTS_FPIC - Success
-- Performing Test CXX_SUPPORTS_FPIC
-- Performing Test CXX_SUPPORTS_FPIC - Success
-- Building with -fPIC
-- Performing Test SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG
-- Performing Test SUPPORTS_FVISIBILITY_INLINES_HIDDEN_FLAG - Success
-- Performing Test C_SUPPORTS_WERROR_DATE_TIME
-- Performing Test C_SUPPORTS_WERROR_DATE_TIME - Success
-- Performing Test CXX_SUPPORTS_WERROR_DATE_TIME
-- Performing Test CXX_SUPPORTS_WERROR_DATE_TIME - Success
-- Performing Test CXX_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW
-- Performing Test CXX_SUPPORTS_WERROR_UNGUARDED_AVAILABILITY_NEW - Failed
-- Performing Test CXX_SUPPORTS_MISSING_FIELD_INITIALIZERS_FLAG
-- Performing Test CXX_SUPPORTS_MISSING_FIELD_INITIALIZERS_FLAG - Success
-- Performing Test C_SUPPORTS_IMPLICIT_FALLTHROUGH_FLAG
-- Performing Test C_SUPPORTS_IMPLICIT_FALLTHROUGH_FLAG - Success
-- Performing Test CXX_SUPPORTS_IMPLICIT_FALLTHROUGH_FLAG
-- Performing Test CXX_SUPPORTS_IMPLICIT_FALLTHROUGH_FLAG - Success
-- Performing Test C_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG
-- Performing Test C_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG - Failed
-- Performing Test CXX_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG
-- Performing Test CXX_SUPPORTS_COVERED_SWITCH_DEFAULT_FLAG - Failed
-- Performing Test CXX_SUPPORTS_CLASS_MEMACCESS_FLAG
-- Performing Test CXX_SUPPORTS_CLASS_MEMACCESS_FLAG - Success
-- Performing Test CXX_SUPPORTS_REDUNDANT_MOVE_FLAG
-- Performing Test CXX_SUPPORTS_REDUNDANT_MOVE_FLAG - Success
-- Performing Test CXX_SUPPORTS_NOEXCEPT_TYPE_FLAG
-- Performing Test CXX_SUPPORTS_NOEXCEPT_TYPE_FLAG - Success
-- Performing Test C_SUPPORTS_DELETE_NON_VIRTUAL_DTOR_FLAG
-- Performing Test C_SUPPORTS_DELETE_NON_VIRTUAL_DTOR_FLAG - Failed
-- Performing Test CXX_SUPPORTS_DELETE_NON_VIRTUAL_DTOR_FLAG
-- Performing Test CXX_SUPPORTS_DELETE_NON_VIRTUAL_DTOR_FLAG - Success
-- Performing Test CXX_SUPPORTS_SUGGEST_OVERRIDE_FLAG
-- Performing Test CXX_SUPPORTS_SUGGEST_OVERRIDE_FLAG - Success
-- Performing Test CXX_WSUGGEST_OVERRIDE_ALLOWS_ONLY_FINAL
-- Performing Test CXX_WSUGGEST_OVERRIDE_ALLOWS_ONLY_FINAL - Success
-- Performing Test C_WCOMMENT_ALLOWS_LINE_WRAP
-- Performing Test C_WCOMMENT_ALLOWS_LINE_WRAP - Failed
-- Performing Test C_SUPPORTS_STRING_CONVERSION_FLAG
-- Performing Test C_SUPPORTS_STRING_CONVERSION_FLAG - Failed
-- Performing Test CXX_SUPPORTS_STRING_CONVERSION_FLAG
-- Performing Test CXX_SUPPORTS_STRING_CONVERSION_FLAG - Failed
-- Performing Test LINKER_SUPPORTS_COLOR_DIAGNOSTICS
-- Performing Test LINKER_SUPPORTS_COLOR_DIAGNOSTICS - Failed
-- Looking for os_signpost_interval_begin
-- Looking for os_signpost_interval_begin - not found
-- Found Python3: /usr/bin/python3.8 (found version "3.8.5") found components: Interpreter
CMake Error at CMakeLists.txt:731 (message):
The target `Aarch64' is experimental and must be passed via
LLVM_EXPERIMENTAL_TARGETS_TO_BUILD.

-- Configuring incomplete, errors occurred!
See also "/home/oceanfish81/workarea/release/CMakeFiles/CMakeOutput.log".
See also "/home/oceanfish81/workarea/release/CMakeFiles/CMakeError.log".
@advancedwebdeveloper
Copy link
Author

gollvm_aarch64_gcc.zip

Here is the compressed build folder.

Super detailed CMake's log is available here https://gist.github.com/advancedwebdeveloper/29ed56eb49ba30af9c4c61bc59449d2b

@thanm
Copy link
Contributor

thanm commented Jan 19, 2021

Your log includes:

CMake Error at CMakeLists.txt:731 (message):
The target `Aarch64' is experimental and must be passed via
LLVM_EXPERIMENTAL_TARGETS_TO_BUILD.

I don't think this has anything to do with gollvm.

FYI, LLVM back end does not currently support -fsplit-stack for aarch64. Clang accepts the option but does not actually insert stack-check sequences in function prologs, to the best of my knowledge. For aarch64 builds I would recommend using -DGOLLVM_USE_SPLIT_STACK=OFF.

@advancedwebdeveloper
Copy link
Author

Well,
must say that I didn't new that.

@advancedwebdeveloper
Copy link
Author

@thanm, what about mmap ?

@advancedwebdeveloper
Copy link
Author

OK, I got some progress.

[ 3%] Linking CXX executable ../../../bin/llvm-goc
/usr/bin/ld: ../../../lib/libLLVMCppGoFrontEnd.a(go.cc.o): in function go_create_gogo(go_create_gogo_args const*)': /home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/go/go.cc:44: undefined reference to Gogo::read_embedcfg(char const*)'
/usr/bin/ld: ../../../lib/libLLVMCppGoFrontEnd.a(parse.cc.o): in function Parse::declaration()': /home/oceanfish81/workarea/llvm-project/llvm/tools/gollvm/gofrontend/go/parse.cc:1324: undefined reference to Gogo::is_embed_imported() const'
collect2: error: ld returned 1 exit status
make[3]: *** [tools/gollvm/driver-main/CMakeFiles/llvm-goc.dir/build.make:240: bin/llvm-goc] Error 1
make[2]: *** [CMakeFiles/Makefile2:30478: tools/gollvm/driver-main/CMakeFiles/llvm-goc.dir/all] Error 2
make[1]: *** [CMakeFiles/Makefile2:29975: tools/gollvm/CMakeFiles/gollvm.dir/rule] Error 2
make: *** [Makefile:7622: gollvm] Error 2
.
That's on

$ lscpu
Architecture: aarch64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 64
On-line CPU(s) list: 0-63
Thread(s) per core: 1
Core(s) per socket: 32
Socket(s): 2
NUMA node(s): 4
Vendor ID: ARM
Model: 2
Model name: Cortex-A72
Stepping: r0p2
BogoMIPS: 100.00
L1d cache: 2 MiB
L1i cache: 3 MiB
L2 cache: 16 MiB
L3 cache: 64 MiB
NUMA node0 CPU(s): 0-15
NUMA node1 CPU(s): 16-31
NUMA node2 CPU(s): 32-47
NUMA node3 CPU(s): 48-63
Vulnerability Itlb multihit: Not affected
Vulnerability L1tf: Not affected
Vulnerability Mds: Not affected
Vulnerability Meltdown: Not affected
Vulnerability Spec store bypass: Vulnerable
Vulnerability Spectre v1: Mitigation; __user pointer sanitization
Vulnerability Spectre v2: Vulnerable
Vulnerability Srbds: Not affected
Vulnerability Tsx async abort: Not affected
Flags: fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
.

$ llc-11 --version
LLVM (http://llvm.org/):
LLVM version 11.0.0

Optimized build.
Default target: aarch64-unknown-linux-gnu
Host CPU: cortex-a72

Any suggestions?

@thanm
Copy link
Contributor

thanm commented Jan 19, 2021

Thanks, I'll send a CL. This is due to a frontend change just recently submitted.

@gopherbot
Copy link

Change https://golang.org/cl/284581 mentions this issue: gollvm: update cmake rules for recent embed.cc addition

@advancedwebdeveloper
Copy link
Author

https://drive.google.com/file/d/11VZO_2zHe_gO77Jj19eetNVuKf4PP_Cx/view?usp=sharing - this my build folder

@thanm
Copy link
Contributor

thanm commented Jan 19, 2021

https://golang.org/cl/284581 is now submitted. Thanks.

@dmitshur dmitshur added this to the gollvm milestone Jan 19, 2021
@advancedwebdeveloper
Copy link
Author

Confirming: I was able to build gollvm and run GoBackendCoreTests suite.
So I assume that we have to initiate a discussion elsewhere, regarding GOLLVM_USE_SPLIT_STACK - and this issue could be closed.

@thanm thanm closed this as completed Jan 20, 2021
lumontec pushed a commit to lumontec/gollvm that referenced this issue Feb 9, 2021
@thanm
gollvm: update cmake rules for recent embed.cc addition
3cb23e5 
Include the newly added "embed.cc" in the list of gofrontend source
files when building the compiler.

Updates golang/go#43772

Change-Id: Ie685a5e55fcfbb9df40aa30815f8c8c07d2cae24
Reviewed-on: https://go-review.googlesource.com/c/gollvm/+/284581
Trust: Than McIntosh <thanm@google.com>
Reviewed-by: Cherry Zhang <cherryyz@google.com>
@advancedwebdeveloper
Copy link
Author

@thanm , please extract those from https://drive.google.com/file/d/1QnF73JBcHmfdDBP8RSF8ccpgYP6XHDxP/view?usp=sharing

@golang golang locked and limited conversation to collaborators Feb 12, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
FrozenDueToAge
Projects
None yet
Milestone
gollvm
Development

No branches or pull requests
4 participants
@dmitshur @advancedwebdeveloper @gopherbot @thanm