View Issue Details

IDProjectCategoryView StatusLast Update
0015814CentOS-7qemu-kvmpublic2019-02-22 17:55
Reporterrrb3942 
PrioritynormalSeverityminorReproducibilityalways
Status newResolutionopen 
PlatformLinuxOSCentosOS Version7
Product Version7.6.1810 
Target VersionFixed in Version 
Summary0015814: Setting kvm_amd avic=1 requires x2apic (intel) on AMD Processors
DescriptionNot sure if this libvirt or qemu

This is verified by checking the libvirt qemu capabilities cache (/var/cache/libvirt/qemu/capabilities/)

If you are running an AMD CPU and enable avic=1, qemu will show AMD CPU models as usable='no' with <blocker name='x2apic'/>.

As far as I can tell 'x2apic' is an Intel only cpu flag. This should probably be checking for the AMD 'avic' flag.
Steps To ReproduceOn an AMD host.
1. Start libvirtd
2. Check qemu capabilities cache. Applicable AMD cpu types will be shown as useable='yes'
3. Set 'options kvm_amd avic=1' in /etc/modprobe.d/kvm.conf
4. Unload and reload the module
5. Verify avic is set 'cat /sys/module/kvm_amd/parameters/avic' should be 1
6. Delete the qemu capabilities cache to ensure it gets re-detected. ' rm -f /var/cache/libvirt/qemu/capabilities/*.xml
7. Restart or start libvirtd
8. Check the capabilities cache. less /var/cache/libvirt/qemu/capabilities/*.xml
Applicable AMD CPUs will be shown with usable='no' and <blocker name='x2apic'/>
Additional InformationTesting was done an AMD EPYC CPU.

I think qemu will silently drop unsupported flags when running a VM, so even though we had avic=1 enabled we could run VMs without error.

I don't know if enabling avic actually does anything if the x2apic flag is not present.

This caused issues for us during an upgrade to oVirt. Mailing list thread:
https://lists.ovirt.org/archives/list/users@ovirt.org/thread/4Y4X7UGDEYSB5JK45TLDERNM7IMTHIYY/

cpuinfo:
head -n26 /proc/cpuinfo
processor : 0
vendor_id : AuthenticAMD
cpu family : 23
model : 1
model name : AMD EPYC 7551P 32-Core Processor
stepping : 2
microcode : 0x8001227
cpu MHz : 2000.000
cache size : 512 KB
physical id : 0
siblings : 64
core id : 0
cpu cores : 32
apicid : 0
initial apicid : 0
fpu : yes
fpu_exception : yes
cpuid level : 13
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc art rep_good nopl nonstop_tsc extd_apicid amd_dcm aperfmperf eagerfpu pni pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_l2 cpb hw_pstate sme retpoline_amd ssbd ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec xgetbv1 clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pausefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca
bogomips : 3992.33
TLB size : 2560 4K pages
clflush size : 64
cache_alignment : 64
address sizes : 43 bits physical, 48 bits virtual
power management: ts ttp tm hwpstate cpb eff_freq_ro [13] [14]

dmesg output showing vapic support:
dmesg | grep -i AMD-Vi
[ 1.997765] AMD-Vi: IOMMU performance counters supported
[ 1.997824] AMD-Vi: IOMMU performance counters supported
[ 1.997854] AMD-Vi: IOMMU performance counters supported
[ 1.997897] AMD-Vi: IOMMU performance counters supported
[ 2.021815] AMD-Vi: Found IOMMU at 0000:00:00.2 cap 0x40
[ 2.021816] AMD-Vi: Extended features (0xf77ef22294ada):
[ 2.021821] AMD-Vi: Found IOMMU at 0000:20:00.2 cap 0x40
[ 2.021822] AMD-Vi: Extended features (0xf77ef22294ada):
[ 2.021825] AMD-Vi: Found IOMMU at 0000:40:00.2 cap 0x40
[ 2.021827] AMD-Vi: Extended features (0xf77ef22294ada):
[ 2.021830] AMD-Vi: Found IOMMU at 0000:60:00.2 cap 0x40
[ 2.021831] AMD-Vi: Extended features (0xf77ef22294ada):
[ 2.021833] AMD-Vi: Interrupt remapping enabled
[ 2.021834] AMD-Vi: virtual APIC enabled
[ 2.022218] AMD-Vi: Lazy IO/TLB flushing enabled
TagsNo tags attached.
abrt_hash
URL

Activities

TrevorH

TrevorH

2019-02-11 18:18

manager   ~0033806

Will need to be reported on upstream bugzilla.redhat.com as CentOS is a rebuild of RHEL sources.
rrb3942

rrb3942

2019-02-11 18:36

reporter   ~0033807

Upstream Bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1675030
rrb3942

rrb3942

2019-02-22 17:55

reporter   ~0033889

Upstream hasn't been able to reproduce this so far. However oVirt installs qemu-kvm from the CentOS Virt SIG (http://mirror.centos.org/centos/7/virt/x86_64/kvm-common/). So it may be a bug in those packages.

What would be the best way to loop that group in?

oVirt bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1674265

Issue History

Date Modified Username Field Change
2019-02-11 18:02 rrb3942 New Issue
2019-02-11 18:18 TrevorH Note Added: 0033806
2019-02-11 18:36 rrb3942 Note Added: 0033807
2019-02-22 17:55 rrb3942 Note Added: 0033889