AVX-512 are 512-bit extensions to the 256-bit Advanced Vector Extensions SIMD instructions for x86 instruction set architecture (ISA) proposed by Intel in July 2013, and is supported in Intel's Xeon Phi x200 (Knights Landing) processor. AVX-512 is not the first 512-bit SIMD instruction set that Intel has introduced in processors. The earlier 512-bit SIMD instructions used in Xeon Phi coprocessors, derived from Intel's Larrabee project, are similar but not binary compatible and only partially source compatible.
Contents
- Instruction set
- Encoding and features
- SIMD modes
- Extended registers
- Opmask registers
- New opmask instructions
- New instructions in AVX 512 foundation
- Blend using mask
- Compare into mask
- Logical set mask
- Compress and expand
- Permute
- Bitwise ternary logic
- Conversions
- Floating point decomposition
- Floating point arithmetics
- New instructions in AVX 512 conflict detection
- New instructions in AVX 512 exponential and reciprocal
- New instructions in AVX 512 prefetch
- New instructions in AVX 512 BW and DQ
- Floating point instructions
- CPUs with AVX 512
- Performance tools for AVX 512 analysis
- References
AVX-512 consists of multiple extensions not all meant to be supported by all processors implementing them. Only the core extension AVX-512F (AVX-512 Foundation) is required by all implementations.
Instruction set
The AVX-512 instruction set consists of several separate sets each having their own unique CPUID feature bit, however they are typically grouped by supporting processor generation.
Encoding and features
The VEX prefix used by AVX and AVX2, while flexible, did not leave enough room for the features Intel wanted to add to AVX-512. This has led them to define a new prefix called EVEX.
Compared to VEX, EVEX adds the following benefits:
The extended registers, SIMD width bit, and opmask registers of AVX-512 are mandatory and all require support from the OS.
SIMD modes
The AVX-512 instructions are designed to mix with 128/256-bit AVX/AVX2 instructions without a performance penalty. However AVX-512VL extensions allows the use of AVX-512 instructions on 128/256-bit registers XMM/YMM, so most SSE and AVX/AVX2 instructions have new AVX-512 versions encoded with the EVEX prefix which allow access to new features such as opmask and additional registers. Unlike AVX-256, the new instructions do not have new mnemonics but share namespace with AVX, making the distinction between VEX and EVEX encoded versions of an instruction ambiguous in the source code. Since AVX-512F only supports 32- and 64-bit values, SSE and AVX/AVX2 instructions that operate on bytes or words are only supported with the AVX-512BW extension (Byte & Word support).
Extended registers
The width of the SIMD register file is increased from 256 bits to 512 bits, with a total of 32 registers ZMM0-ZMM31. These registers can be addressed as 256 bit YMM registers from AVX extensions and 128-bit XMM registers from Streaming SIMD Extensions, and legacy AVX and SSE instructions can be extended to operate on the 16 additional registers XMM16-XMM31 and YMM16-YMM31 when using EVEX encoded form.
Opmask registers
Most AVX-512 instructions may indicate one of 8 opmask registers (k0–k7). The first one k0 is however a hardcoded constant used to indicate unmasked operations. The opmask are in most instructions used to control which values are written to the destination. A flag controls the opmask behavior, which can either be "zero", which zeros everything not selected by the mask, or "merge", which leaves everything not selected untouched. The merge behavior is identical to the blend instructions.
The opmask registers are normally 16-bit wide, but can be up to 64 bits with the AVX-512BW extension. How many of the bits are actually used, though, depends on the vector type of the instructions masked. For the 32-bit single float or double words, 16 bits are used to mask the 16 elements in a 512-bit register. For double float and quad words, at most 8 mask bits are used.
The opmask register is the reason why several bitwise instructions which naturally have no element widths, had them added in AVX-512. For instance, bitwise AND, OR or 128-bit shuffle, now exist in both double-word and quad-word variants with the only difference being in the final masking.
New opmask instructions
The opmask registers have a new mini extension of instructions operating directly on them. Unlike the rest of the AVX-512 instructions, these instructions are all VEX encoded. The initial opmask instructions are all 16-bit (Word) versions. With AVX-512DQ 8-bit (Byte) versions are added to better match the needs of masking 8 64-bit values, and with AVX-512BW 32-bit (Double) and 64-bit (Quad) versions will be added so they can mask up to 64 8-bit values. The instructions KORTEST and KTEST can be used to set the classic x86 flags based on mask registers, so that they may be used together with non-SIMD x86 branch and conditional instructions.
New instructions in AVX-512 foundation
Many AVX-512 instructions are simply EVEX versions of old SSE or AVX instructions. There are however several new instructions, and old instructions that have been replaced with new AVX-512 versions. The new or majorly reworked instructions are listed below. These foundation instructions also include the extensions from AVX-512VL and AVX-512BW since those extensions merely add new versions of these instructions instead of new instructions.
Blend using mask
There are no EVEX-prefixed versions of the blend instructions from SSE4; instead, AVX-512 has a new set of blending instructions using mask registers as selectors. Together with the general compare into mask instructions below, these may be used to implement generic ternary operations or cmov, similar to XOP's VPCMOV.
Since blending is an integral part of the EVEX encoding, these instruction may also be considered basic move instructions. Using the zeroing blend mode, they can also be used as masking instructions.
Compare into mask
AVX-512F has four new compare instructions. Like their XOP counterparts they use the immediate field to select between 8 different comparisons. Unlike their XOP inspiration however they save the result to a mask register and initially only support doubleword and quadword comparisons. The AVX-512BW extension provides the byte and word versions. Note that two mask registers may be specified for the instructions, one to write to and one to declare regular masking.
Logical set mask
The final way to set masks is using Logical Set Mask. These instructions perform either AND or NAND, and then set the destination opmask based on the result values being zero or non-zero. Note like the comparison instructions these take two opmask registers, one as destination and one a regular opmask.
Compress and expand
The compress and expand instructions matches the APL operations of the same name. They use the opmask in a slightly different way from other AVX-512 instructions. Compress only saves the values marked in the mask, but saves them compacted by skipping and not reserving space for unmarked values. Expand operates in the opposite way, by loading as many values as indicated in the mask and then spreading them to the selected positions.
Permute
A new set of permute instructions have been added for full two input permutations, they all take three arguments, two source registers and one index, the result is output by either overwriting the first source register or the index register. AVX-512BW extends the instructions to also include 16-bit (word) versions, but not 8-bit (byte) versions. The byte versions are considered separate instructions and are part of the AVX-512VBMI extension.
Bitwise ternary logic
Two new instructions added can logically implement all possible bitwise operations between three inputs. They take three registers as input and an 8-bit immediate field. Each bit in the output is generated using a lookup of the three corresponding bits in the inputs to select one of the 8 positions in the 8-bit immediate. Since only 8 combinations are possible using three bits, this allow all possible 3 input bitwise operations to be performed. These are the only bitwise vector instructions in AVX-512F, EVEX versions of the two source SSE and AVX bitwise vector instructions AND, ANDN, OR and XOR were added in AVX-512DQ.
The difference in the doubleword and quadword versions is only the application of the opmask.
Examples:
Conversions
A number of conversion or move instructions were added, that completes the set of conversion instructions available from SSE2.
Floating point decomposition
Among the unique new features in AVX-512F are instructions to decompose floating-point values and handle special floating-point values. Since these methods are completely new, they also exist in scalar versions.
Floating point arithmetics
This is the second set of new floating-point methods, which includes new scaling and approximate calculation of reciprocal, and reciprocal of square root. The approximate reciprocal instructions guarantee to have at most a relative error of 2−14.
New instructions in AVX-512 conflict detection
The instructions in AVX-512 conflict detection (AVX-512CD) are designed to help efficiently calculate conflict-free subsets of elements in loops that normally could not be safely vectorized.
New instructions in AVX-512 exponential and reciprocal
AVX-512 exponential and reciprocal instructions contain more accurate approximate reciprocal instructions than those in the AVX-512 foundation; relative error is at most 2−28. They also contain two new exponential functions that have a relative error of at most 2−23.
New instructions in AVX-512 prefetch
AVX-512 prefetch instructions contain new prefetch operations for the new scatter and gather functionality introduced in AVX2 and AVX-512. T0 prefetch means prefetching into level 1 cache and T1 means prefetching into level 2 cache.
New instructions in AVX-512 BW and DQ
AVX-512BW adds byte and word version of instructions in AVX-512F and adds AVX-512 versions of several byte and word instructions that haven't had one. AVX-512DQ adds new instructions for doubleword and quadword registers, and AVX-512BW adds byte and words versions of the same instructions. Two new instructions were added to the mask instructions set, for those see the earlier section.
Among the instructions added by AVX-512DQ are several SSE, AVX instruction that didn't get AVX-512 versions with AVX-512F, among those are all the two input bitwise instructions and extract/insert integer instructions.
Instructions that are completely new are covered below.
Floating point instructions
Three new floating point operations are introduced. Since they are not only new to AVX-512 they have both packed/SIMD and scalar versions.
The VFPCLASS instructions tests if the floating point value is one of eight special floating-point values, which of the eight values will trigger a bit in the output mask register is controlled by the immediate field. The VRANGE instructions performs minimum or maximum operations depending on the value of the immediate field, which can also control if the operation is done absolute or not and separately how the sign is handled. The VREDUCE instructions operates on a single source, and subtracts from that the integer part of the source value plus a number of bits specified in the immediate field of its fraction.
CPUs with AVX-512
Performance tools for AVX-512 analysis
Intel "Vectorization" Advisor (starting from version 2016 Update 3) supports native AVX-512 performance and vector code quality analysis for 2nd generation Intel® Xeon Phi™ (codenamed Knights Landing) processor. Along with traditional hotspots profile, Advisor Recommendations and "seamless" integration of Intel Compiler vectorization diagnostics, Advisor Survey analysis also provides AVX-512 ISA metrics and new AVX-512-specific "traits", e.g. Scatter, Compress/Expand, mask utilization,