It stands for "End Branch 64 bit" (there is also a 32 bit counter part) -- or more precisely, Terminate Indirect Branch in 64 bit.
Here is the operation:
IF EndbranchEnabled(CPL) & EFER.LMA = 1 & CS.L = 1
IF CPL = 3
THEN
IA32_U_CET.TRACKER = IDLE
IA32_U_CET.SUPPRESS = 0
ELSE
IA32_S_CET.TRACKER = IDLE
IA32_S_CET.SUPPRESS = 0
FI
FI;
If the instruction can't clear the TRACKER flag, then the CPU generates a #CP exception. In other words, if a hacker was able to change the destination address of an indirect jump, the program is very likely to terminate even if the destination is legal assembly code.
The instruction is otherwise considered a NOP.
In other woerdswords, the CET feature is used to make sure that your indirect branches actually go to a valid location. This allows for additional safety. Here is the paragraph from Intel about it:
The ENDBRANCH (see Section 73 for details) is a new instruction that is used to mark valid jump target addresses of indirect calls and jumps in the program. This instruction opcode is selected to be one that is a NOP on legacy machines such that programs compiled with ENDBRANCH new instruction continue to function on old machines without the CET enforcement. On processors that support CET the ENDBRANCH is still a NOP and is primarily used as a marker instruction by the processor pipeline to detect control flow violations. The CPU implements a state machine that tracks indirect jmp and call instructions. When one of these instructions is seen, the state machine moves from IDLE to WAIT_FOR_ENDBRANCH state. In WAIT_FOR_ENDBRANCH state the next instruction in the program stream must be an ENDBRANCH. If an ENDBRANCH is not seen the processor causes a control protection exception (#CP), else the state machine moves back to IDLE state.
As a side note, it is possible to tell the processor to allow for no ENDBR64. This is done with a prefix (3Eh). This is useful for cases such as a switch where the addresses are in a table located in read-only memory. However, the CPU ignores that prefix in many cases.
