x/arch/x86/x86asm: missing instructions: FLDZ, FLDLN2 #18665

minux · 2017-01-15T05:46:58Z

package main

import "golang.org/x/arch/x86/x86asm"

func main() {
	text := []byte{0xd9, 0xee} // fldz
	_, err := x86asm.Decode(text[:], 64)
	if err != nil {
		panic(err)
	}
}

I found two missing instructions (they're also missing in x86.csv, which suggests that x86.csv is not complete):
0xd9 0xed: FLDLN2
0xd9 0xee: FLDZ

There are also some missing AVX instructions: e.g.
c4 c1 f3 2a cc vcvtsi2sd %r12,%xmm1,%xmm1

/cc @rsc

The text was updated successfully, but these errors were encountered:

minux · 2017-01-26T07:33:31Z

This is blocked by lack of access to the x86spec program that generated x86.csv.
Fixing this issue is easy, but I want to also fix x86spec program to make sure there aren't other missed instructions.

/cc @rsc

mewmew · 2017-04-08T22:42:01Z

Thanks @minux for raising this issue.

I would also be very happy if support for these instructions were added to arch/x86. Back in 2015, I tried to submit a pull request to add these instructions rsc/x86#1

I'd also be very interested in the x86spec program which converts the Intel 64 and IA-32 Architectures Software Development Manual's PDF to x86.csv.

@rsc, did you ever make the source code of this program public?

mewmew · 2017-10-16T14:14:22Z

This is blocked by lack of access to the x86spec program that generated x86.csv.

@minux With rev golang/arch@c6bf551 the source code of the x86spec tool has now been added to the repo.

rsc · 2017-10-16T14:20:18Z

I believe @quasilyte is working on generating x86.csv from Intel's description tables as open sourced as part of xed, which will replace the x86spec pdf reader.

mewmew · 2017-10-16T14:20:48Z

Ah, glad to hear! Thanks!

quasilyte · 2018-04-01T05:36:30Z

I can confirm that XED-based x86.csv contains such instructions:

$ grep FLDZ x86xed.csv
"FLDZ","FLDZ","fldz","D9 EE","V","V","","","","",""
$ grep FLDLN x86xed.csv
"FLDLN2","FLDLN2","fldln2","D9 ED","V","V","","","","",""

$ grep FLDLN x86.csv
$ grep FLDZ x86.csv

It's a quite long way to fixing x86.csv-based tools though.
But at least XED files reader|CL88015 and new x86.csv generator (CL not mailed yet) are on their way.

gopherbot · 2018-04-03T16:16:12Z

Change https://golang.org/cl/104496 mentions this issue: x86/x86spec: enable XED-based x86.csv generation

mewmew · 2018-04-06T16:53:27Z

@quasilyte That's great to hear! Thanks.

mewmew · 2018-11-13T23:02:51Z

I tried again today at rev golang/arch@b19384d of x/arch, and I get the error unrecognized instruction for fldz instructions.

@quasilyte I understood from your comment that fldz should be supported by XED, and with https://golang.org/cl/104496 the tables should be generated based on XED. Is there some step missing to resolve this issue?

Cheers,
Robin

Fixes golang/go#18665.

gopherbot · 2018-11-13T23:10:17Z

Change https://golang.org/cl/149438 mentions this issue: x86asm: add support for FLDZ and FLDLN2

quasilyte · 2018-11-14T07:25:29Z

@mewmew, there were some unresolved format and implementation related questions that made XED-based csv generation abandoned.

mewmew · 2018-11-14T08:16:02Z

@mewmew, there were some unresolved format and implementation related questions that made XED-based csv generation abandoned.

Ok, good to know. I submitted https://go-review.googlesource.com/c/arch/+/149438/ which at least fixes this issue by adding FLZ and FLDLN2 to x86.csv. Is it x86.csv that is the most up-to-date config, or would that be x86.v0.2.csv?

quasilyte · 2018-11-14T08:31:15Z

@mewmew I believe you should use x86.v0.2.csv.

I'm not sure editing auto-generated file is a good idea though
and it doesn't solve the problem with other missing instructions in the x86.csv.
This discussion can be continued in a CL itself.

pwaller · 2019-04-10T23:18:44Z

But at least XED files reader|CL88015 and new x86.csv generator (CL not mailed yet)

@quasilyte - please could you clarify - did you get around to mailing a new x86.csv generator?

quasilyte · 2019-04-11T05:58:46Z

It was mailed (https://go-review.googlesource.com/c/arch/+/104496/).
Never got through.
That CL can be (to some extent) used as an example of how XED can be used to generate that file, but there is probably a better way, so I would write it from scratch instead of trying to reuse code from there.

I believe the results of that program were mostly correct (probably more correct than the current csv file) and it included AVX512 instructions. But since the generator itself was not submitted, the generated file was left behind as well.

mewmew · 2019-09-04T09:24:21Z

I believe @quasilyte is working on generating x86.csv from Intel's description tables as open sourced as part of xed, which will replace the x86spec pdf reader.

@quasilyte and @rsc, I felt very excited to see the XED data being used for the decoding table generation, as it would help resolve issues with missing instructions, such as FLDZ, FLDLN2 reported in this issue.

However, now it seems that the CL (https://go-review.googlesource.com/c/arch/+/104496/) has been stalled?

Are there any technical reasons why the CL has not been merged. From my understanding, it seems that it worked well to use the XED data and also helped resolve issues with missing instructions.

If there are parts missing before https://go-review.googlesource.com/c/arch/+/104496/ could be merged, @rsc or @quasilyte would you help clarify those so that someone else could pick up the torch and get this merged?

Cheers,
Robin

quasilyte · 2019-09-04T10:04:02Z

To my knowledge, @fexolm was investigating an opportunity to work on x86 decoder.

@mewmew, it might be a problem that this needs an effort and it's already possible to do a workaround without it.

However, now it seems that the CL (https://go-review.googlesource.com/c/arch/+/104496/) has been stalled?

Correct. It's there as a reference for further attempts. One should start a new CL and think about it from scratch, while keeping in mind the context from CL104496.

Are there any technical reasons why the CL has not been merged. From my understanding, it seems that it worked well to use the XED data and also helped resolve issues with missing instructions.

If I understood it correctly, the x86.csv was intended to be used for generating all x86 asm related tools, like encoders and decoders. The problem is, x86.csv is a simplified view of an arch, it had no AVX-512 in the beginning. To generate AVX-512 aware tools, we need it to contain additional data. The CL you mentioned stuck somewhere in:

Can't propose simple enough and functionally complete format for updated x86.csv.
There were some discussions about implementation complexity.

These 2 problems may not be purely technical, but they come to my mind before anything.

In the end, AVX-512 related code was generated out of XED tables directly to avoid discussions around x86.csv. It can be proposed that we use XED data files instead of x86.csv everywhere, but that format is not well-defined and is not part of the stable API (my knowledge might be outdated). Some kind of version-aware wrapper would be more reliable I guess, but again it's a work that someone should perform. Or x86.csv needs a revisit after we merged AVX-512 and can avoid rushing it.

As another example of how XED data files were used during that, there is avx512test (but it's not pretty).

Hope that helps.

mewmew · 2019-09-04T10:27:29Z

Thanks @quasilyte for the writeup and added context.

re:

To my knowledge, @fexolm was investigating an opportunity to work on x86 decoder.

@fexolm how did your investigation go? If you would care to, please share your experiences, good or bad, as it could help us move this along.

Cheers,
Robin

Fixes golang/go#18665.

Updates golang/go#18665.

mewmew · 2019-11-26T22:59:50Z

🎉

minux added this to the Unreleased milestone Jan 15, 2017

minux assigned rsc Jan 15, 2017

minux mentioned this issue Jan 15, 2017

x/arch/x86/x86asm: Mem.Disp is positive for negative displacements #18666

Closed

mewmew added a commit to decomp/exp that referenced this issue Jun 3, 2017

travis: use x/arch/x86/x86asm fork until golang/go#18665 is fixed

dd54998

mewmew added a commit to mewpull/arch that referenced this issue Nov 13, 2018

x86asm: add support for FLDZ and FLDLN2

5603d0e

Fixes golang/go#18665.

mewmew mentioned this issue Nov 13, 2018

x86asm: add support for FLDZ and FLDLN2 golang/arch#3

Closed

mewmew added a commit to mewpull/arch that referenced this issue Nov 23, 2019

x86asm: add support for FLDZ and FLDLN2

97cdbbe

Fixes golang/go#18665.

mewmew added a commit to mewpull/arch that referenced this issue Nov 23, 2019

x86/x86asm: add test cases for fldln2 and fldz

60cd639

Updates golang/go#18665.

gopherbot closed this as completed in golang/arch@368ea8f Nov 26, 2019

mewmew mentioned this issue Nov 27, 2019

x/arch/x86/x86asm: missing Control-flow Enforcement instructions; e.g. ENDBR64 #35865

Open

golang locked and limited conversation to collaborators Nov 25, 2020

gopherbot added the FrozenDueToAge label Nov 25, 2020

rsc removed their assignment Jun 23, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

x/arch/x86/x86asm: missing instructions: FLDZ, FLDLN2 #18665

x/arch/x86/x86asm: missing instructions: FLDZ, FLDLN2 #18665

minux commented Jan 15, 2017 •

edited

minux commented Jan 26, 2017

mewmew commented Apr 8, 2017

mewmew commented Oct 16, 2017 •

edited

rsc commented Oct 16, 2017

mewmew commented Oct 16, 2017

quasilyte commented Apr 1, 2018

gopherbot commented Apr 3, 2018

mewmew commented Apr 6, 2018

mewmew commented Nov 13, 2018 •

edited

gopherbot commented Nov 13, 2018

quasilyte commented Nov 14, 2018

mewmew commented Nov 14, 2018

quasilyte commented Nov 14, 2018

pwaller commented Apr 10, 2019

quasilyte commented Apr 11, 2019

mewmew commented Sep 4, 2019

quasilyte commented Sep 4, 2019

mewmew commented Sep 4, 2019

mewmew commented Nov 26, 2019

x/arch/x86/x86asm: missing instructions: FLDZ, FLDLN2 #18665

x/arch/x86/x86asm: missing instructions: FLDZ, FLDLN2 #18665

Comments

minux commented Jan 15, 2017 • edited

minux commented Jan 26, 2017

mewmew commented Apr 8, 2017

mewmew commented Oct 16, 2017 • edited

rsc commented Oct 16, 2017

mewmew commented Oct 16, 2017

quasilyte commented Apr 1, 2018

gopherbot commented Apr 3, 2018

mewmew commented Apr 6, 2018

mewmew commented Nov 13, 2018 • edited

gopherbot commented Nov 13, 2018

quasilyte commented Nov 14, 2018

mewmew commented Nov 14, 2018

quasilyte commented Nov 14, 2018

pwaller commented Apr 10, 2019

quasilyte commented Apr 11, 2019

mewmew commented Sep 4, 2019

quasilyte commented Sep 4, 2019

mewmew commented Sep 4, 2019

mewmew commented Nov 26, 2019

minux commented Jan 15, 2017 •

edited

mewmew commented Oct 16, 2017 •

edited

mewmew commented Nov 13, 2018 •

edited