This appears to have broken at CL 345051.

I can reproduce this and it will definitely be a result of the call trampolines change.

That said, there is some pretty strange code generation going on here - the unicode/norm test compiles to a 40MB text object, with the norm.init function being a single function with more than 2MB of machine code (that's more than 500,000 machine instructions). There also seems to be very strange pointless repetition in this code, which I'm investigating/attempting to understand:

  2943f8:       00000617                auipc   a2,0x0
  2943fc:       00060613                mv      a2,a2
  294400:       00000697                auipc   a3,0x0
  294404:       00068693                mv      a3,a3
  294408:       00000717                auipc   a4,0x0
  29440c:       00070713                mv      a4,a4
  294410:       00000797                auipc   a5,0x0
  294414:       00078793                mv      a5,a5
  294418:       00000817                auipc   a6,0x0
  29441c:       00080813                mv      a6,a6

The issue is caused by the fact that we now use a JAL instruction for calls, which is only capable of reaching +/-1MB of text. When we cannot reach a function directly the linker lays out a trampoline after the function, however in the case of a >1MB function, we're not able to even reach the trampoline via JAL. It's pretty strange (but clearly not impossible) to have a 1MB+ function...

From the riscv64 side there are potential a number of options if we want/need to continue to support functions of this size (unfortunately all have complexity/downsides):

• Add code to the assembler and linker to switch from JAL to AUIPC/JALR where a symbol exceeds 1MB in size (this is probably the most optimal, but also the most complex).

• Use AUIPC/JALR sequences and rewrite to JAL/NOOP where possible (this fixes the issue, but means we're going to increase binary sizes considerably again).

• Split text symbols that exceed 1MB so that we can layout trampolines in the middle of the symbol (jumping from the first part to the second part, etc).

• Layout trampolines both before and after the function text (that would get us to ~2MB for a single symbol, as you would jump backwards from the first 1MB and forwards from the last 1MB).

• Revert the call trampolines all together and go back to using AUIPC/JALR for everything (slower and more bloated code).

@4a6f656c thanks for the investigation.

Add code to the assembler and linker to switch from JAL to AUIPC/JALR where a symbol exceeds 1MB in size (this is probably the most optimal, but also the most complex).

This doesn't sound too bad to me. I think we can do it in the assembler without much complexity. At the PC assignment loop https://cs.opensource.google/go/go/+/master:src/cmd/internal/obj/riscv/obj.go;l=589 we'll know the size of a function. If it is too large we can just redo the PC assignment while rewriting JAL to AUIPC/JALR.

For the record, it turns out that the strange code are address calculations that have not yet had relocations applied. Overall, it looks like the main code bloat results from initialisation of sparse arrays - I would have expected this to be data driven with minimal code, where as instead it seems to be almost entirely code driven.

@4a6f656c thanks for the investigation.

Add code to the assembler and linker to switch from JAL to AUIPC/JALR where a symbol exceeds 1MB in size (this is probably the most optimal, but also the most complex).

This doesn't sound too bad to me. I think we can do it in the assembler without much complexity. At the PC assignment loop https://cs.opensource.google/go/go/+/master:src/cmd/internal/obj/riscv/obj.go;l=589 we'll know the size of a function. If it is too large we can just redo the PC assignment while rewriting JAL to AUIPC/JALR.

Right, we can do that - it adds some complexity and interferes with some of the next steps to further clean up the assembler and eliminate obj.Prog rewriting, but I'll deal with that in due course.

Change https://golang.org/cl/356250 mentions this issue: cmd/internal/obj/riscv: fix trampoline calls from large functions