Download as PDF, PPTX
Uploaded byThe Linux Foundation
Introduction of AMD Virtual Interrupt Controller
The document introduces AMD's Advanced Virtual Interrupt Controller (AVIC), a hardware enhancement for efficient virtualization of APIC functionality in AMD's SVM architecture. AVIC allows guest operating systems to access APIC registers directly, improving performance by reducing the overhead associated with interrupt delivery and scheduling. Key components include the virtual APIC, physical and logical ID tables, and a doorbell mechanism for delivering interrupts to specific physical cores.
Introduction of AMD Virtual Interrupt Controller
- 1. Introduction of AMD AdvancedVirtual Interrupt Controller XenSummit 2012 Wei Huang August 2012
- 2. What is AVIC? AVIC is Advanced Virtual Interrupt Controller A virtual APIC to guest OSs with hardware acceleration Enhancement to the AMD SVM architecture Changes to CPU, NorthBridge, and IOMMU 2 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 3. Agenda Motivationfor AVIC AVIC Architecture – APIC backing page – Physical & logical APIC ID tables – Door bell & IOMMU extension Hypervisor Design for AVIC Summary 3 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 4. Agenda Motivationfor AVIC AVIC Architecture – APIC backing page – Physical & logical APIC ID tables – Door bell & IOMMU extension Hypervisor Design for AVIC Summary 4 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 5. Local APIC inx86 Local APIC (LAPIC) is vital for x86 SMP system Handles various interrupt sources: – Inter-processor interrupts (IPIs) – I/O devices – APIC timer-generated interrupts – Internal events CPU 1 CPU 2 CPU 3 CPU 4 Local APIC Local APIC Local APIC Local APIC Interrupt Interrupt Interrupt Interrupt IPIs IPIs IPIs IPIs messages messages messages messages Interrupt messages External I/O APIC interrupts 5 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 6. Many LAPIC Registers… *AMD APM Vol 2 6 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 7. LAPIC + Virtualization= Slow 1. APIC register reads/writes 2. Inter-processor Interrupts Guest OS Guest OS vCPU vCPU X vCPU X APIC Reg Rd/Wrt IPI Hypervisor Hypervisor 3. I/O interrupts from peripherals (e.g., pass-through device) Guest OS device vCPU mem Hypervisor IOMMU DMA I/O dev interrupt Device 7 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 8. Attacking Problems withAVIC We define new architectural components to present a virtualized APIC to guests, thus allowing most APIC accesses and interrupt delivery into the guests directly. AVIC components: Component Problem Solved vAPIC Direct access to APIC registers Physical & logical APIC ID tables Mapping of physical & virtual cores (scheduling) Doorbell interrupt Interrupt delivery IOMMU extension I/O interrupt delivery 8 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 9. Agenda Motivationfor AVIC AVIC Architecture – APIC backing page – Physical & logical APIC ID tables – Doorbell & IOMMU extension Hypervisor Design for AVIC Summary 9 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 10. vAPIC vAPIC is a virtual APIC to the guest – Backed by a 4KB memory page – Allocated and initialized by hypervisor – vAPIC pages are allocated on a per-vCPU basis – Contains storage for guest APIC values for the vCPU – Mapped into the guest physical address space APIC Backing Page ICR guest memory ICR ICR IRR IRR IRR ISR ISR ISR guest APIC ICR TPR ICR TPR ICR TPR IRR EOI IRR EOI IRR EOI BACKING_PAGE ptr ISR myAPIC ISR myAPIC ISR Physical APIC Table Physical APIC Table TPR Logical APIC Table TPR Logical APIC Table TPR V_APIC_BAR EOI EOI = EOI VMCB 10 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 11. Handling Guest APICAccesses Four types of actions for guest APIC accesses: Action VMEXIT? Details ALLOWED NO The field will be updated (for Write) or read (for Read). ACCLERATED NO The field will be updated and CPU will take further actions, such as sending IPI to CPUs. TRAP YES The field will be updated and CPU will take VMEXIT immediately after access. FAULT YES The field will NOT be updated and CPU will take VMEXIT immediately before the access. Example Offset APIC Register Name Read Write 200-270h Interrupt Request Register (IRR) ALLOWED FAULT 300h Interrupt Command Register Low (ICR) ALLOWED ACCLERATED or TRAP 11 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 12. Regarding APIC Acceleration… Three performance-critical areas are accelerated – TPR reads and writes – EOI writes – ICRL writes 12 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 13. Agenda Motivation for AVIC AVIC Architecture – APIC backing page – Physical & logical APIC ID tables – Door bell & IOMMU extension Hypervisor Design for AVIC Summary 13 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 14. How About vCPUScheduling? Hypervisor schedules guest VM’s VCPUs on-the-fly – VCPUs can be ON or OFF – VCPUs can be migrated AVIC uses physical and logical APIC ID tables to show VCPU’s mapping relationship with physical cores 14 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 15. Physical APIC IDTable Purpose: Used by the hardware to route the virtual interrupt messages to the proper physical core vAPIC Backing Page Guest CPU 4 Physical APIC Table Guest APIC ID 5 entry VMCB Guest APIC ID 4 entry Guest APIC ID 3 Doorbell Destination ID entry AVIC_PHYSICAL_TABLE ptr 15 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 16. Logical APIC IDTable Purpose: – Maps guest logical APIC IDs to guest physical APIC IDs – Used to route logically addressed IPIs within a guest VM vAPIC Backing Page Guest CPU 0 Physical APIC Table Logical APIC Table Guest APIC ID 2 entry Guest logical 7 Guest APIC ID 1 VMCB to physical ID entry Guest logical 6 Guest APIC ID 0 to physical 0 entry Guest logical 5 Doorbell Destination ID to physical ID AVIC_PHYSICAL_TABLE ptr 16 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 17. Agenda Motivation for AVIC AVIC Architecture – APIC backing page – Physical & logical APIC ID tables – Doorbell & IOMMU extension Hypervisor Design for AVIC Summary 17 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 18. Doorbell A new interrupt mechanism that is used to deliver guest interrupts to a specific physical core Doorbell interrupts are sent in three ways: Doorbell Source Reason AVIC hardware In response to a guest ICRL write for a supported IPI type IOMMU An incoming I/O interrupt is remapped to an AVIC mode guest System software Writing to the doorbell interrupt MSR 18 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 19. IOMMU Extension The AVIC architecture leverages the existing IOMMU interrupt redirection mechanism to provide a new guest-delivered interrupt type A new field in the IOMMU device table specifies whether AVIC is available IOMMU uses doorbell mechanism to delivery interrupts into guest VMs 19 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 20. Agenda Motivation for AVIC AVIC Architecture – APIC backing page – Physical & logical APIC ID tables – Door bell & IOMMU extension Hypervisor Design for AVIC Summary 20 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 21. How to Designa Hypervisor for AVIC? At guest start-up: – Allocate and initialize one vAPIC page for each vCPU – Allocate one physical APIC table and logical APIC table for the guest – Set up VMCB pointers for these structures and enable AVIC mode in VMCB – Set up IOMMU tables for direct-assigned I/O devices While running: – Handle un-accelerated cases similar to current approach – Update physical APIC table and IOMMU IRTE entries to change running status whenever a vCPU is moved to or from a physical CPU 21 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 22. Setting up AVICTables vAPIC backing pages and physical/logical ID tables are allocated in host physical memory Related VMCB fields: VMCB Field Details V_APIC_BAR Guest physical base address of the virtual APIC. AVIC_BACKING_PAGE Host physical address of the APIC backing page for the associated VM. AVIC_LOGICAL_PAGE Host physical address of the logical APIC look-up table for the associated VM. AVIC_PHYSICAL_PAGE Host physical address for the physical APIC look-up table for the associated VM. 22 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 23. Handle New VMEXITs VMEXIT 401h – For incomplete IPI Delivery – This VMEXIT describes the specific reason for the IPI delivery failure VMEXIT 402h – For ccess to un-accelerated vAPIC field – This VMEXIT indicates the offset of the un-accelerated vAPIC register, as well as whether a read or write operation was attempted 23 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 24. Summary AVIC is an hardware extension to AMD SVM for APIC acceleration AVIC targets critical APIC operations for optimal performance – APIC read/write accesses – Interrupt delivery The design of AVIC is simple for fast hypervisor integration We expect AVIC to eliminate lots of overhead introduced by virtual local APIC 24 | Introduction of AMD AVIC | Xen Summit | August, 2012
- 25. Questions? 25 | Introductionof AMD AVIC | Xen Summit | August, 2012
- 26. Trademark Attribution AMD, theAMD Arrow logo and combinations thereof are trademarks of Advanced Micro Devices, Inc. in the United States and/or other jurisdictions. Other names used in this presentation are for identification purposes only and may be trademarks of their respective owners. ©2012 Advanced Micro Devices, Inc. All rights reserved. 26 | Introduction of AMD AVIC | Xen Summit | August, 2012