Wrong #UD conditions for instructions with mask-register

Hi there,

while using some random data, I noticed XED beeing unable to decode certain instructions if one of the following conditions are met (64-bit mode):

EVEX
1. modrm.reg encodes a MASK register and EVEX.R   !=    0 (correct behavior)
2. modrm.reg encodes a MASK register and EVEX.R'  !=    0 (correct behavior)
3. modrm.rm  encodes a MASK register and EVEX.B   !=    0 (should be ignored)

VEX
4. VEX.vvvv  encodes a MASK register and VEX.vvvv != 1xxx
5. modrm.reg encodes a MASK register and VEX.R    !=    0 (might be correct)
6. modrm.rm  encodes a MASK register and VEX.B    !=    0

I did not find VEX-specific information about mask-register encoding, but according to table 2.33 (page 99) and table 2.39 (page 102/103) (Intel® 64 and IA-32 Architectures Software Developer’s Manual, March 2017) case 3. should not #UD for EVEX-instructions (EVEX.B is ignored instead).

Are these tables valid for VEX and MVEX instructions as well or do these ones follow different reg-encoding/#UD rules?

If they are valid, case 6. should be handled in the same way as case 3. and for case 4. only bits [2:0] are used, so I guess ignoring the most-significant bit of .vvvv should be the correct way of handling this one.

In addition EVEX.R' should always be ignored in non 64-bit modes and not cause a decoding-error.

[markcharney]

Thanks for looking at the old docs. Over the years, I’ve found enough errors in the docs that, when I can, I usually check the hardware directly or have someone look at the RTL for a couple of designs. I should have a more complete answer in a couple of days. For EVEX.vvvv, I believe the designs ignore EVEX.vvvv[3] in 32b mode but require that bit to be 1 in 64b mode. I’ll work with the doc people to have the VEX/EVEX #UD stuff improved for mask regs.

[markcharney]

I have implemented a small modification for the EVEX uses of GPR32_R and GPR64_R in mode64 patterns. I’ll push it next week.