2018-01-18 07:44 UTC

View Issue Details Jump to Notes ]
IDProjectCategoryView StatusLast Update
0013763CentOS-7kernelpublic2018-01-13 17:01
Reporterkstange 
PriorityhighSeveritycrashReproducibilityalways
StatusacknowledgedResolutionopen 
Platformx86_64OScentosOS Version7.4
Product Version 
Target VersionFixed in Version 
Summary0013763: CentOS 7.4 kernel (3.10.0-693*) fails to boot as Xen PV guest
DescriptionAfter a mailing list post indicating Xen PV guest not booting with CR kernel for 7.4:

https://lists.centos.org/pipermail/centos-virt/2017-August/005618.html

I began to investigate and determined that it appears something is causing a page allocation failure during systemd-udev initialization, which prevents proper booting of the system.

https://lists.centos.org/pipermail/centos-virt/2017-September/005623.html

Failure is:

[ 1.970630] ------------[ cut here ]------------
[ 1.970651] WARNING: CPU: 2 PID: 225 at mm/vmalloc.c:131
vmap_page_range_noflush+0x2c1/0x350
[ 1.970660] Modules linked in:
[ 1.970668] CPU: 2 PID: 225 Comm: systemd-udevd Not tainted
3.10.0-693.1.1.el7.x86_64 #1
[ 1.970677] 0000000000000000 000000008cddc75d ffff8803e8587bd8
ffffffff816a3d91
[ 1.970688] ffff8803e8587c18 ffffffff810879c8 00000083811c14e8
ffff8800066eb000
[ 1.970698] 0000000000000001 ffff8803e86d6940 ffffffffc0000000
0000000000000000
[ 1.970708] Call Trace:
[ 1.970725] [<ffffffff816a3d91>] dump_stack+0x19/0x1b
[ 1.970736] [<ffffffff810879c8>] __warn+0xd8/0x100
[ 1.970742] [<ffffffff81087b0d>] warn_slowpath_null+0x1d/0x20
[ 1.970748] [<ffffffff811c0781>] vmap_page_range_noflush+0x2c1/0x350
[ 1.970758] [<ffffffff811c083e>] map_vm_area+0x2e/0x40
[ 1.970765] [<ffffffff811c1590>] __vmalloc_node_range+0x170/0x270
[ 1.970774] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.970781] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.970792] [<ffffffff8105f143>] module_alloc+0x73/0xd0
[ 1.970798] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.970804] [<ffffffff810fe754>] module_alloc_update_bounds+0x14/0x70
[ 1.970811] [<ffffffff810ff2d2>] load_module+0xb02/0x29e0
[ 1.970817] [<ffffffff811c0717>] ? vmap_page_range_noflush+0x257/0x350
[ 1.970823] [<ffffffff811c083e>] ? map_vm_area+0x2e/0x40
[ 1.970829] [<ffffffff811c1590>] ? __vmalloc_node_range+0x170/0x270
[ 1.970838] [<ffffffff81101249>] ? SyS_init_module+0x99/0x110
[ 1.970846] [<ffffffff81101275>] SyS_init_module+0xc5/0x110
[ 1.970856] [<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
[ 1.970862] ---[ end trace 2117480876ed90d2 ]---
[ 1.970869] vmalloc: allocation failure, allocated 24576 of 28672 bytes
[ 1.970874] systemd-udevd: page allocation failure: order:0, mode:0xd2
[ 1.970883] CPU: 2 PID: 225 Comm: systemd-udevd Tainted: G W
   ------------ 3.10.0-693.1.1.el7.x86_64 #1
[ 1.970894] 00000000000000d2 000000008cddc75d ffff8803e8587c48
ffffffff816a3d91
[ 1.970910] ffff8803e8587cd8 ffffffff81188810 ffffffff8190ea38
ffff8803e8587c68
[ 1.970923] ffffffff00000018 ffff8803e8587ce8 ffff8803e8587c88
000000008cddc75d
[ 1.970939] Call Trace:
[ 1.970946] [<ffffffff816a3d91>] dump_stack+0x19/0x1b
[ 1.970961] [<ffffffff81188810>] warn_alloc_failed+0x110/0x180
[ 1.970971] [<ffffffff811c1654>] __vmalloc_node_range+0x234/0x270
[ 1.970981] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.970989] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.970999] [<ffffffff8105f143>] module_alloc+0x73/0xd0
[ 1.971031] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.971038] [<ffffffff810fe754>] module_alloc_update_bounds+0x14/0x70
[ 1.971046] [<ffffffff810ff2d2>] load_module+0xb02/0x29e0
[ 1.971052] [<ffffffff811c0717>] ? vmap_page_range_noflush+0x257/0x350
[ 1.971061] [<ffffffff811c083e>] ? map_vm_area+0x2e/0x40
[ 1.971067] [<ffffffff811c1590>] ? __vmalloc_node_range+0x170/0x270
[ 1.971075] [<ffffffff81101249>] ? SyS_init_module+0x99/0x110
[ 1.971081] [<ffffffff81101275>] SyS_init_module+0xc5/0x110
[ 1.971088] [<ffffffff816b4fc9>] system_call_fastpath+0x16/0x1b
[ 1.971094] Mem-Info:
[ 1.971103] active_anon:875 inactive_anon:2049 isolated_anon:0
[ 1.971103] active_file:791 inactive_file:8841 isolated_file:0
[ 1.971103] unevictable:0 dirty:0 writeback:0 unstable:0
[ 1.971103] slab_reclaimable:1732 slab_unreclaimable:1629
[ 1.971103] mapped:1464 shmem:2053 pagetables:480 bounce:0
[ 1.971103] free:4065966 free_pcp:763 free_cma:0
[ 1.971131] Node 0 DMA free:15912kB min:12kB low:12kB high:16kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB
managed:15912kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB
free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[ 1.971217] lowmem_reserve[]: 0 4063 16028 16028
[ 1.971226] Node 0 DMA32 free:4156584kB min:4104kB low:5128kB
high:6156kB active_anon:952kB inactive_anon:1924kB active_file:0kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB
present:4177920kB managed:4162956kB mlocked:0kB dirty:0kB writeback:0kB
mapped:4kB shmem:1928kB slab_reclaimable:240kB slab_unreclaimable:504kB
kernel_stack:32kB pagetables:592kB unstable:0kB bounce:0kB
free_pcp:1760kB local_pcp:288kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? no
[ 1.971264] lowmem_reserve[]: 0 0 11964 11964
[ 1.971273] Node 0 Normal free:12091564kB min:12088kB low:15108kB
high:18132kB active_anon:2352kB inactive_anon:6272kB active_file:3164kB
inactive_file:35364kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:12591104kB managed:12251788kB mlocked:0kB
dirty:0kB writeback:0kB mapped:5852kB shmem:6284kB
slab_reclaimable:6688kB slab_unreclaimable:6012kB kernel_stack:880kB
pagetables:1328kB unstable:0kB bounce:0kB free_pcp:1196kB
local_pcp:152kB free_cma:0kB writeback_tmp:0kB pages_scanned:0
all_unreclaimable? no
[ 1.971309] lowmem_reserve[]: 0 0 0 0
[ 1.971316] Node 0 DMA: 0*4kB 1*8kB (U) 0*16kB 1*32kB (U) 2*64kB (U)
1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (M) 3*4096kB (M) =
15912kB
[ 1.971343] Node 0 DMA32: 7*4kB (M) 18*8kB (UM) 7*16kB (EM) 3*32kB
(EM) 1*64kB (E) 2*128kB (UM) 1*256kB (E) 4*512kB (UM) 4*1024kB (UEM)
4*2048kB (EM) 1011*4096kB (M) = 4156348kB
[ 1.971377] Node 0 Normal: 64*4kB (UEM) 10*8kB (UEM) 6*16kB (EM)
3*32kB (EM) 3*64kB (UE) 3*128kB (UEM) 1*256kB (E) 2*512kB (UE) 0*1024kB
1*2048kB (M) 2951*4096kB (M) = 12091728kB
[ 1.971413] Node 0 hugepages_total=0 hugepages_free=0
hugepages_surp=0 hugepages_size=2048kB
[ 1.971425] 11685 total pagecache pages
[ 1.971430] 0 pages in swap cache
[ 1.971437] Swap cache stats: add 0, delete 0, find 0/0
[ 1.971444] Free swap = 0kB
[ 1.971451] Total swap = 0kB
[ 1.971456] 4196255 pages RAM
[ 1.971462] 0 pages HighMem/MovableOnly
[ 1.971467] 88591 pages reserved
Steps To ReproduceUpgrade to 3.10.0-693 or later and reboot.
Additional InformationKernel boots fine if you switch back to 514.26.2. I'm putting this here and then filing a bug upstream to RHEL as well.
TagsNo tags attached.
abrt_hash
URL
Attached Files
  • log file icon kernel-3.10.0-693.1.1-boot.log (107,196 bytes) 2017-09-01 23:09
  • patch file icon xen-dont-copy-bogus-duplicate-entries-into-kernel-page-tables.patch (5,502 bytes) 2017-09-06 19:06 -
    From 0b5a50635fc916cf46e3de0b819a61fc3f17e7ee Mon Sep 17 00:00:00 2001
    From: Stefan Bader <stefan.bader@canonical.com>
    Date: Tue, 2 Sep 2014 11:16:01 +0100
    Subject: x86/xen: don't copy bogus duplicate entries into kernel page tables
    
    When RANDOMIZE_BASE (KASLR) is enabled; or the sum of all loaded
    modules exceeds 512 MiB, then loading modules fails with a warning
    (and hence a vmalloc allocation failure) because the PTEs for the
    newly-allocated vmalloc address space are not zero.
    
      WARNING: CPU: 0 PID: 494 at linux/mm/vmalloc.c:128
               vmap_page_range_noflush+0x2a1/0x360()
    
    This is caused by xen_setup_kernel_pagetables() copying
    level2_kernel_pgt into level2_fixmap_pgt, overwriting many non-present
    entries.
    
    Without KASLR, the normal kernel image size only covers the first half
    of level2_kernel_pgt and module space starts after that.
    
    L4[511]->level3_kernel_pgt[510]->level2_kernel_pgt[  0..255]->kernel
                                                      [256..511]->module
                              [511]->level2_fixmap_pgt[  0..505]->module
    
    This allows 512 MiB of of module vmalloc space to be used before
    having to use the corrupted level2_fixmap_pgt entries.
    
    With KASLR enabled, the kernel image uses the full PUD range of 1G and
    module space starts in the level2_fixmap_pgt. So basically:
    
    L4[511]->level3_kernel_pgt[510]->level2_kernel_pgt[0..511]->kernel
                              [511]->level2_fixmap_pgt[0..505]->module
    
    And now no module vmalloc space can be used without using the corrupt
    level2_fixmap_pgt entries.
    
    Fix this by properly converting the level2_fixmap_pgt entries to MFNs,
    and setting level1_fixmap_pgt as read-only.
    
    A number of comments were also using the the wrong L3 offset for
    level2_kernel_pgt.  These have been corrected.
    
    Signed-off-by: Stefan Bader <stefan.bader@canonical.com>
    Signed-off-by: David Vrabel <david.vrabel@citrix.com>
    Reviewed-by: Boris Ostrovsky <boris.ostrovsky@oracle.com>
    Cc: stable@vger.kernel.org
    ---
     arch/x86/include/asm/pgtable_64.h |  1 +
     arch/x86/xen/mmu.c                | 27 ++++++++++++---------------
     2 files changed, 13 insertions(+), 15 deletions(-)
    
    diff --git a/arch/x86/include/asm/pgtable_64.h b/arch/x86/include/asm/pgtable_64.h
    index 5be9063..3874693 100644
    --- a/arch/x86/include/asm/pgtable_64.h
    +++ b/arch/x86/include/asm/pgtable_64.h
    @@ -19,6 +19,7 @@ extern pud_t level3_ident_pgt[512];
     extern pmd_t level2_kernel_pgt[512];
     extern pmd_t level2_fixmap_pgt[512];
     extern pmd_t level2_ident_pgt[512];
    +extern pte_t level1_fixmap_pgt[512];
     extern pgd_t init_level4_pgt[];
     
     #define swapper_pg_dir init_level4_pgt
    diff --git a/arch/x86/xen/mmu.c b/arch/x86/xen/mmu.c
    index e8a1201..16fb009 100644
    --- a/arch/x86/xen/mmu.c
    +++ b/arch/x86/xen/mmu.c
    @@ -1858,11 +1858,10 @@
      *
      * We can construct this by grafting the Xen provided pagetable into
      * head_64.S's preconstructed pagetables.  We copy the Xen L2's into
    - * level2_ident_pgt, level2_kernel_pgt and level2_fixmap_pgt.  This
    - * means that only the kernel has a physical mapping to start with -
    - * but that's enough to get __va working.  We need to fill in the rest
    - * of the physical mapping once some sort of allocator has been set
    - * up.
    + * level2_ident_pgt, and level2_kernel_pgt.  This means that only the
    + * kernel has a physical mapping to start with - but that's enough to
    + * get __va working.  We need to fill in the rest of the physical
    + * mapping once some sort of allocator has been set up.
      */
     void __init xen_setup_kernel_pagetable(pgd_t *pgd, unsigned long max_pfn)
     {
    @@ -1892,9 +1891,12 @@
     	/* L3_i[0] -> level2_ident_pgt */
     	convert_pfn_mfn(level3_ident_pgt);
     	/* L3_k[510] -> level2_kernel_pgt
    -	 * L3_i[511] -> level2_fixmap_pgt */
    +	 * L3_k[511] -> level2_fixmap_pgt */
     	convert_pfn_mfn(level3_kernel_pgt);
     
    +	/* L3_k[511][506] -> level1_fixmap_pgt */
    +	convert_pfn_mfn(level2_fixmap_pgt);
    +
     	/* We get [511][511] and have Xen's version of level2_kernel_pgt */
     	l3 = m2v(pgd[pgd_index(__START_KERNEL_map)].pgd);
     	l2 = m2v(l3[pud_index(__START_KERNEL_map)].pud);
    @@ -1903,22 +1905,15 @@
     	addr[1] = (unsigned long)l3;
     	addr[2] = (unsigned long)l2;
     	/* Graft it onto L4[272][0]. Note that we creating an aliasing problem:
    -	 * Both L4[272][0] and L4[511][511] have entries that point to the same
    +	 * Both L4[272][0] and L4[511][510] have entries that point to the same
     	 * L2 (PMD) tables. Meaning that if you modify it in __va space
     	 * it will be also modified in the __ka space! (But if you just
     	 * modify the PMD table to point to other PTE's or none, then you
     	 * are OK - which is what cleanup_highmap does) */
     	copy_page(level2_ident_pgt, l2);
    -	/* Graft it onto L4[511][511] */
    +	/* Graft it onto L4[511][510] */
     	copy_page(level2_kernel_pgt, l2);
     
    -	/* Get [511][510] and graft that in level2_fixmap_pgt */
    -	l3 = m2v(pgd[pgd_index(__START_KERNEL_map + PMD_SIZE)].pgd);
    -	l2 = m2v(l3[pud_index(__START_KERNEL_map + PMD_SIZE)].pud);
    -	copy_page(level2_fixmap_pgt, l2);
    -	/* Note that we don't do anything with level1_fixmap_pgt which
    -	 * we don't need. */
    -
     	/* Make pagetable pieces RO */
     	set_page_prot(init_level4_pgt, PAGE_KERNEL_RO);
     	set_page_prot(level3_ident_pgt, PAGE_KERNEL_RO);
    @@ -1927,6 +1922,7 @@
     	set_page_prot(level2_ident_pgt, PAGE_KERNEL_RO);
     	set_page_prot(level2_kernel_pgt, PAGE_KERNEL_RO);
     	set_page_prot(level2_fixmap_pgt, PAGE_KERNEL_RO);
    +	set_page_prot(level1_fixmap_pgt, PAGE_KERNEL_RO);
     
     	/* Pin down new L4 */
     	pin_pagetable_pfn(MMUEXT_PIN_L4_TABLE,
    
  • ? file icon run_lorax (280 bytes) 2017-09-22 03:21 -
    #sudo -i
    #setenforce 0
    DEFAULTKERNEL=kernel-plus lorax -p Centos-Minimal -v 7 -r 7.4 \
    -s http://mirrors.us.kernel.org/centos/7/os/x86_64/ \
    -s http://mirrors.us.kernel.org/centos/7/updates/x86_64/ \
    -s http://mirrors.us.kernel.org/centos/7/centosplus/x86_64/ \
    ./results/
    exit 0
    
    ? file icon run_lorax (280 bytes) 2017-09-22 03:21 +
  • txt file icon aws-kernel-plus-3.10.0-693.11.6.el7.centos.plus.x86_64.log.txt (65,581 bytes) 2018-01-13 14:29 -
    i-0071d198fc56b75db	e: registered new interface driver hub
    
    [    0.128069] usbcore: registered new device driver usb
    
    [    0.129126] dmi: Firmware registration failed.
    
    [    0.130037] PCI: System does not support PCI
    
    [    0.131009] PCI: System does not support PCI
    
    [    0.132169] NetLabel: Initializing
    
    [    0.133024] NetLabel:  domain hash size = 128
    
    [    0.134017] NetLabel:  protocols = UNLABELED CIPSOv4
    
    [    0.135055] NetLabel:  unlabeled traffic allowed by default
    
    [    0.136050] amd_nb: Cannot enumerate AMD northbridges
    
    [    0.137037] Switched to clocksource xen
    
    [    0.150363] pnp: PnP ACPI: disabled
    
    [    0.158911] NET: Registered protocol family 2
    
    [    0.166778] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
    
    [    0.178769] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
    
    [    0.189316] TCP: Hash tables configured (established 16384 bind 16384)
    
    [    0.199539] TCP: reno registered
    
    [    0.204854] UDP hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.214132] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.224257] NET: Registered protocol family 1
    
    [    0.231702] Unpacking initramfs...
    
    [    0.322421] random: fast init done
    
    [    0.795814] Freeing initrd memory: 19404k freed
    
    [    0.836023] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 163840 ms ovfl timer
    
    [    0.849038] RAPL PMU: hw unit of domain pp0-core 2^-16 Joules
    
    [    0.858265] RAPL PMU: hw unit of domain package 2^-16 Joules
    
    [    0.868441] RAPL PMU: hw unit of domain dram 2^-16 Joules
    
    [    0.877679] platform rtc_cmos: registered platform RTC device (no PNP device found)
    
    [    0.889792] sha1_ssse3: Using SSSE3 optimized SHA-1 implementation
    
    [    0.899877] sha256_ssse3: Using SSSE3 optimized SHA-256 implementation
    
    [    0.910971] futex hash table entries: 256 (order: 2, 16384 bytes)
    
    [    0.921383] Initialise system trusted keyring
    
    [    0.929467] audit: initializing netlink socket (disabled)
    
    [    0.938389] type=2000 audit(1515851068.643:1): initialized
    
    [    0.981758] HugeTLB registered 2 MB page size, pre-allocated 0 pages
    
    [    0.994775] zpool: loaded
    
    [    0.999321] zbud: loaded
    
    [    1.003769] VFS: Disk quotas dquot_6.5.2
    
    [    1.010182] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
    
    [    1.020523] msgmni has been set to 3031
    
    [    1.027239] Key type big_key registered
    
    [    1.034631] NET: Registered protocol family 38
    
    [    1.041808] Key type asymmetric registered
    
    [    1.048082] Asymmetric key parser 'x509' registered
    
    [    1.055638] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
    
    [    1.067494] io scheduler noop registered
    
    [    1.073972] io scheduler deadline registered (default)
    
    [    1.082193] io scheduler cfq registered
    
    [    1.088598] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
    
    [    1.097006] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
    
    [    1.108019] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
    
    [    1.118109] Non-volatile memory driver v1.3
    
    [    1.124671] Linux agpgart interface v0.103
    
    [    1.131759] crash memory driver: version 1.1
    
    [    1.138607] rdac: device handler registered
    
    [    1.145365] hp_sw: device handler registered
    
    [    1.152263] emc: device handler registered
    
    [    1.158974] alua: device handler registered
    
    [    1.166059] libphy: Fixed MDIO Bus: probed
    
    [    1.173435] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    
    [    1.184515] ehci-pci: EHCI PCI platform driver
    
    [    1.191747] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
    
    [    1.201627] ohci-pci: OHCI PCI platform driver
    
    [    1.209097] uhci_hcd: USB Universal Host Controller Interface driver
    
    [    1.219336] usbcore: registered new interface driver usbserial
    
    [    1.228610] usbcore: registered new interface driver usbserial_generic
    
    [    1.239959] usbserial: USB Serial support registered for generic
    
    [    1.249822] i8042: PNP: No PS/2 controller found. Probing ports directly.
    
    [    3.268318] i8042: No controller found
    
    [    3.274232] mousedev: PS/2 mouse device common for all mice
    
    [    3.391486] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
    
    [    3.426704] rtc_cmos: probe of rtc_cmos failed with error -38
    
    [    3.436801] dmi-sysfs: dmi entry is absent.
    
    [    3.444127] hidraw: raw HID events driver (C) Jiri Kosina
    
    [    3.452806] usbcore: registered new interface driver usbhid
    
    [    3.461781] usbhid: USB HID core driver
    
    [    3.469664] drop_monitor: Initializing network drop monitor service
    
    [    3.481349] TCP: cubic registered
    
    [    3.488241] Initializing XFRM netlink socket
    
    [    3.497567] NET: Registered protocol family 10
    
    [    3.507254] NET: Registered protocol family 17
    
    [    3.515980] mce: Unable to init device /dev/mcelog (rc: -5)
    
    [    3.528628] microcode: CPU0 sig=0x206d7, pf=0x1, revision=0x712
    
    [    3.539843] microcode: Microcode Update Driver: v2.01 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
    
    [    3.556197] Loading compiled-in X.509 certificates
    
    [    3.565330] Loaded X.509 cert 'CentOS Linux kpatch signing key: ea0413152cde1d98ebdca3fe6f0230904c9ef717'
    
    [    3.583448] Loaded X.509 cert 'CentOS Linux Driver update signing key: 7f421ee0ab69461574bb358861dbe77762a4201b'
    
    [    3.603163] Loaded X.509 cert 'CentOS Linux kernel signing key: 2bade9ee54e382c6692955a95011589b663d6aef'
    
    [    3.622367] registered taskstats version 1
    
    Xen Minimal OS!
    
      start_info: 0xcba000(VA)
    
        nr_pages: 0x61d05
    
      shared_inf: 0x68e37000(MA)
    
         pt_base: 0xcbd000(VA)
    
    nr_pt_frames: 0xb
    
        mfn_list: 0x9ab000(VA)
    
       mod_start: 0x0(VA)
    
         mod_len: 0
    
           flags: 0x300
    
        cmd_line: root=/dev/sda1 ro console=hvc0 4
    
      stack:      0x96a100-0x98a100
    
    MM: Init
    
          _text: 0x0(VA)
    
         _etext: 0x7b824(VA)
    
       _erodata: 0x97000(VA)
    
         _edata: 0x9cce0(VA)
    
    stack start: 0x96a100(VA)
    
           _end: 0x9aa700(VA)
    
      start_pfn: ccb
    
        max_pfn: 61d05
    
    Mapping memory range 0x1000000 - 0x61d05000
    
    setting 0x0-0x97000 readonly
    
    skipped 0x1000
    
    MM: Initialise page allocator for fd3000(fd3000)-61d05000(61d05000)
    
    MM: done
    
    Demand map pfns at 61d06000-2061d06000.
    
    Heap resides at 2061d07000-4061d07000.
    
    Initialising timer interface
    
    Initialising console ... done.
    
    gnttab_table mapped at 0x61d06000.
    
    Initialising scheduler
    
    Thread "Idle": pointer: 0x2061d07050, stack: 0x619f0000
    
    Thread "xenstore": pointer: 0x2061d07800, stack: 0x61a00000
    
    xenbus initialised on irq 2 mfn 0xfeffc
    
    Thread "shutdown": pointer: 0x2061d07fb0, stack: 0x61a20000
    
    Dummy main: start_info=0x98a200
    
    Thread "main": pointer: 0x2061d08760, stack: 0x61a30000
    
    "main" "root=/dev/sda1" "ro" "console=hvc0" "4" 
    
    vbd 2049 is hd0
    
    ******************* BLKFRONT for device/vbd/2049 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21479/2049
    
    Failed to read /local/domain/0/backend/vbd/21479/2049/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21479/2049/feature-flush-cache.
    
    4194304 sectors of 512 bytes
    
    **************************
    
    vbd 2050 is hd1
    
    ******************* BLKFRONT for device/vbd/2050 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21479/2050
    
    Failed to read /local/domain/0/backend/vbd/21479/2050/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21479/2050/feature-flush-cache.
    
    312705024 sectors of 512 bytes
    
    **************************
    
    vbd 2051 is hd2
    
    ******************* BLKFRONT for device/vbd/2051 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21479/2051
    
    Failed to read /local/domain/0/backend/vbd/21479/2051/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21479/2051/feature-flush-cache.
    
    1835008 sectors of 512 bytes
    
    **************************
    
      Booting 'CentOS Linux (3.10.0-693.11.6.el7.centos.plus.x86_64) 7 (Core)'
    
    
    
    
    
    root (hd0)
    
    
     Filesystem type is ext2fs, using whole disk
    
    
    kernel /boot/vmlinuz-3.10.0-693.11.6.el7.centos.plus.x86_64 ro root=UUID=ade108
    
    
    6f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    
    initrd /boot/initramfs-3.10.0-693.11.6.el7.centos.plus.x86_64.img
    
    
    
    
    
    ============= Init TPM Front ================
    
    Tpmfront:Error Unable to read device/vtpm/0/backend-id during tpmfront initialization! error = ENOENT
    
    Tpmfront:Info Shutting down tpmfront
    
    close blk: backend=/local/domain/0/backend/vbd/21479/2049 node=device/vbd/2049
    
    close blk: backend=/local/domain/0/backend/vbd/21479/2050 node=device/vbd/2050
    
    close blk: backend=/local/domain/0/backend/vbd/21479/2051 node=device/vbd/2051
    
    [    0.000000] Initializing cgroup subsys cpuset
    
    [    0.000000] Initializing cgroup subsys cpu
    
    [    0.000000] Initializing cgroup subsys cpuacct
    
    [    0.000000] Linux version 3.10.0-693.11.6.el7.centos.plus.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Thu Jan 4 14:27:13 UTC 2018
    
    [    0.000000] Command line: ro root=UUID=ade1086f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    [    0.000000] ACPI in unprivileged domain disabled
    
    [    0.000000] e820: BIOS-provided physical RAM map:
    
    [    0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
    
    [    0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
    
    [    0.000000] Xen: [mem 0x0000000000100000-0x0000000062504fff] usable
    
    [    0.000000] NX (Execute Disable) protection: active
    
    [    0.000000] DMI not present or invalid.
    
    [    0.000000] e820: last_pfn = 0x62505 max_arch_pfn = 0x400000000
    
    [    0.000000] RAMDISK: [mem 0x0243e000-0x03730fff]
    
    [    0.000000] NUMA turned off
    
    [    0.000000] Faking a node at [mem 0x0000000000000000-0x0000000062504fff]
    
    [    0.000000] NODE_DATA(0) allocated [mem 0x61cde000-0x61d04fff]
    
    [    0.000000] Zone ranges:
    
    [    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
    
    [    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
    
    [    0.000000]   Normal   empty
    
    [    0.000000] Movable zone start for each node
    
    [    0.000000] Early memory node ranges
    
    [    0.000000]   node   0: [mem 0x00001000-0x0009ffff]
    
    [    0.000000]   node   0: [mem 0x00100000-0x62504fff]
    
    [    0.000000] Initmem setup node 0 [mem 0x00001000-0x62504fff]
    
    [    0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org
    
    [    0.000000] No local APIC present
    
    [    0.000000] APIC: disable apic facility
    
    [    0.000000] APIC: switched to apic NOOP
    
    [    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
    
    [    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
    
    [    0.000000] e820: [mem 0x62505000-0xffffffff] available for PCI devices
    
    [    0.000000] Booting paravirtualized kernel on Xen
    
    [    0.000000] Xen version: 3.4.3.amazon (preserve-AD)
    
    [    0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
    
    [    0.000000] PERCPU: Embedded 35 pages/cpu @ffff880061400000 s104600 r8192 d30568 u2097152
    
    [    0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes)
    
    [    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 396282
    
    [    0.000000] Policy zone: DMA32
    
    [    0.000000] Kernel command line: ro root=UUID=ade1086f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.000000] Memory: 1532640k/1610772k available (6984k kernel code, 388k absent, 77744k reserved, 4581k data, 1800k init)
    
    [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
    
    [    0.000000] x86/pti: Xen PV detected, disabling PTI protection
    
    [    0.000000] Hierarchical RCU implementation.
    
    [    0.000000] 	RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=1.
    
    [    0.000000] NR_IRQS:327936 nr_irqs:24 0
    
    [    0.000000] Console: colour dummy device 80x25
    
    [    0.000000] console [tty0] enabled
    
    [    0.000000] console [hvc0] enabled
    
    [    0.000000] allocated 6815744 bytes of page_cgroup
    
    [    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
    
    [    0.000000] installing Xen timer for CPU 0
    
    [    0.000000] tsc: Fast TSC calibration using PIT
    
    [    0.000000] tsc: Detected 1795.712 MHz processor
    
    [    0.003000] Calibrating delay loop (skipped), value calculated using timer frequency.. 3591.34 BogoMIPS (lpj=1795672)
    
    [    0.004019] pid_max: default: 32768 minimum: 301
    
    [    0.005080] Security Framework initialized
    
    [    0.006027] SELinux:  Initializing.
    
    [    0.007047] Yama: becoming mindful.
    
    [    0.008631] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
    
    [    0.009826] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
    
    [    0.010285] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.011046] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.012401] Initializing cgroup subsys memory
    
    [    0.013032] Initializing cgroup subsys devices
    
    [    0.014015] Initializing cgroup subsys freezer
    
    [    0.015022] Initializing cgroup subsys net_cls
    
    [    0.016015] Initializing cgroup subsys blkio
    
    [    0.017021] Initializing cgroup subsys perf_event
    
    [    0.018022] Initializing cgroup subsys hugetlb
    
    [    0.019015] Initializing cgroup subsys pids
    
    [    0.020022] Initializing cgroup subsys net_prio
    
    [    0.021131] FEATURE SPEC_CTRL Not Present
    
    [    0.022017] FEATURE IBPB_SUPPORT Not Present
    
    [    0.023011] CPU: Physical Processor ID: 0
    
    [    0.024017] CPU: Processor Core ID: 0
    
    [    0.025025] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
    
    [    0.026010] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32
    
    [    0.027018] tlb_flushall_shift: 6
    
    [    0.073170] Freeing SMP alternatives: 24k freed
    
    [    0.080600] ftrace: allocating 26850 entries in 105 pages
    
    [    0.087145] cpu 0 spinlock event irq 17
    
    [    0.088017] smpboot: Max logical packages: 1
    
    [    0.089072] Performance Events: unsupported p6 CPU model 45 no PMU driver, software events only.
    
    [    0.093354] Brought up 1 CPUs
    
    [    0.094159] NMI watchdog: disabled (cpu0): hardware events not enabled
    
    [    0.095009] NMI watchdog: Shutting down hard lockup detector on all cpus
    
    [    0.096161] devtmpfs: initialized
    
    [    0.099151] EVM: security.selinux
    
    [    0.100017] EVM: security.ima
    
    [    0.101017] EVM: security.capability
    
    [    0.103000] atomic64 test passed for x86-64 platform with CX8 and with SSE
    
    [    0.103032] pinctrl core: initialized pinctrl subsystem
    
    [    0.104072] xen:grant_table: Grant tables using version 1 layout
    
    [    0.105064] Grant table initialized
    
    [    0.119000] RTC time: 165:165:165, date: 165/165/65
    
    [    0.119167] NET: Registered protocol family 16
    
    [    0.121019] PCI: setting up Xen PCI frontend stub
    
    [    0.123321] ACPI: Interpreter disabled.
    
    [    0.124054] xen:balloon: Initialising balloon driver
    
    [    0.127085] xen_balloon: Initialising balloon driver
    
    [    0.128239] vgaarb: loaded
    
    [    0.129123] SCSI subsystem initialized
    
    [    0.130082] usbcore: registered new interface driver usbfs
    
    [    0.131052] usbcore: registered new interface driver hub
    
    [    0.132077] usbcore: registered new device driver usb
    
    [    0.133130] dmi: Firmware registration failed.
    
    [    0.134043] PCI: System does not support PCI
    
    [    0.135009] PCI: System does not support PCI
    
    [    0.136183] NetLabel: Initializing
    
    [    0.137018] NetLabel:  domain hash size = 128
    
    [    0.138008] NetLabel:  protocols = UNLABELED CIPSOv4
    
    [    0.139043] NetLabel:  unlabeled traffic allowed by default
    
    [    0.140058] amd_nb: Cannot enumerate AMD northbridges
    
    [    0.141033] Switched to clocksource xen
    
    [    0.155171] pnp: PnP ACPI: disabled
    
    [    0.164175] NET: Registered protocol family 2
    
    [    0.173230] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
    
    [    0.187132] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
    
    [    0.199634] TCP: Hash tables configured (established 16384 bind 16384)
    
    [    0.211944] TCP: reno registered
    
    [    0.218365] UDP hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.230016] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.242567] NET: Registered protocol family 1
    
    [    0.251336] Unpacking initramfs...
    
    [    0.331396] random: fast init done
    
    [    0.811863] Freeing initrd memory: 19404k freed
    
    [    0.828857] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 163840 ms ovfl timer
    
    [    0.844194] RAPL PMU: hw unit of domain pp0-core 2^-16 Joules
    
    [    0.855245] RAPL PMU: hw unit of domain package 2^-16 Joules
    
    [    0.866217] RAPL PMU: hw unit of domain dram 2^-16 Joules
    
    [    0.876589] platform rtc_cmos: registered platform RTC device (no PNP device found)
    
    [    0.891732] sha1_ssse3: Using SSSE3 optimized SHA-1 implementation
    
    [    0.903982] sha256_ssse3: Using SSSE3 optimized SHA-256 implementation
    
    [    0.917256] futex hash table entries: 256 (order: 2, 16384 bytes)
    
    [    0.929113] Initialise system trusted keyring
    
    [    0.937869] audit: initializing netlink socket (disabled)
    
    [    0.948101] type=2000 audit(1515851106.545:1): initialized
    
    [    0.992975] HugeTLB registered 2 MB page size, pre-allocated 0 pages
    
    [    1.006345] zpool: loaded
    
    [    1.011753] zbud: loaded
    
    [    1.016974] VFS: Disk quotas dquot_6.5.2
    
    [    1.024513] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
    
    [    1.037019] msgmni has been set to 3031
    
    [    1.044514] Key type big_key registered
    
    [    1.053089] NET: Registered protocol family 38
    
    [    1.061772] Key type asymmetric registered
    
    [    1.069658] Asymmetric key parser 'x509' registered
    
    [    1.079075] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
    
    [    1.093266] io scheduler noop registered
    
    [    1.100910] io scheduler deadline registered (default)
    
    [    1.110763] io scheduler cfq registered
    
    [    1.118346] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
    
    [    1.128918] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
    
    [    1.141975] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
    
    [    1.154649] Non-volatile memory driver v1.3
    
    [    1.162813] Linux agpgart interface v0.103
    
    [    1.170746] crash memory driver: version 1.1
    
    [    1.179257] rdac: device handler registered
    
    [    1.187305] hp_sw: device handler registered
    
    [    1.195603] emc: device handler registered
    
    [    1.203646] alua: device handler registered
    
    [    1.211784] libphy: Fixed MDIO Bus: probed
    
    [    1.219453] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    
    [    1.231762] ehci-pci: EHCI PCI platform driver
    
    [    1.240466] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
    
    [    1.252368] ohci-pci: OHCI PCI platform driver
    
    [    1.261066] uhci_hcd: USB Universal Host Controller Interface driver
    
    [    1.273539] usbcore: registered new interface driver usbserial
    
    [    1.285321] usbcore: registered new interface driver usbserial_generic
    
    [    1.297925] usbserial: USB Serial support registered for generic
    
    [    1.309542] i8042: PNP: No PS/2 controller found. Probing ports directly.
    
    [    3.339419] i8042: No controller found
    
    [    3.346952] mousedev: PS/2 mouse device common for all mice
    
    [    3.465359] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
    
    [    3.478094] rtc_cmos: probe of rtc_cmos failed with error -38
    
    [    3.489522] dmi-sysfs: dmi entry is absent.
    
    [    3.497741] hidraw: raw HID events driver (C) Jiri Kosina
    
    [    3.508257] usbcore: registered new interface driver usbhid
    
    [    3.519108] usbhid: USB HID core driver
    
    [    3.526549] drop_monitor: Initializing network drop monitor service
    
    [    3.538685] TCP: cubic registered
    
    [    3.545311] Initializing XFRM netlink socket
    
    [    3.553795] NET: Registered protocol family 10
    
    [    3.562884] NET: Registered protocol family 17
    
    [    3.571651] mce: Unable to init device /dev/mcelog (rc: -5)
    
    [    3.582560] microcode: CPU0 sig=0x206d7, pf=0x1, revision=0x712
    
    [    3.593957] microcode: Microcode Update Driver: v2.01 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
    
    [    3.611079] Loading compiled-in X.509 certificates
    
    [    3.620654] Loaded X.509 cert 'CentOS Linux kpatch signing key: ea0413152cde1d98ebdca3fe6f0230904c9ef717'
    
    [    3.638693] Loaded X.509 cert 'CentOS Linux Driver update signing key: 7f421ee0ab69461574bb358861dbe77762a4201b'
    
    [    3.659264] Loaded X.509 cert 'CentOS Linux kernel signing key: 2bade9ee54e382c6692955a95011589b663d6aef'
    
    [    3.677322] registered taskstats version 1
    
    Xen Minimal OS!
    
      start_info: 0xcba000(VA)
    
        nr_pages: 0x61d05
    
      shared_inf: 0x68e37000(MA)
    
         pt_base: 0xcbd000(VA)
    
    nr_pt_frames: 0xb
    
        mfn_list: 0x9ab000(VA)
    
       mod_start: 0x0(VA)
    
         mod_len: 0
    
           flags: 0x300
    
        cmd_line: root=/dev/sda1 ro console=hvc0 4
    
      stack:      0x96a100-0x98a100
    
    MM: Init
    
          _text: 0x0(VA)
    
         _etext: 0x7b824(VA)
    
       _erodata: 0x97000(VA)
    
         _edata: 0x9cce0(VA)
    
    stack start: 0x96a100(VA)
    
           _end: 0x9aa700(VA)
    
      start_pfn: ccb
    
        max_pfn: 61d05
    
    Mapping memory range 0x1000000 - 0x61d05000
    
    setting 0x0-0x97000 readonly
    
    skipped 0x1000
    
    MM: Initialise page allocator for fd3000(fd3000)-61d05000(61d05000)
    
    MM: done
    
    Demand map pfns at 61d06000-2061d06000.
    
    Heap resides at 2061d07000-4061d07000.
    
    Initialising timer interface
    
    Initialising console ... done.
    
    gnttab_table mapped at 0x61d06000.
    
    Initialising scheduler
    
    Thread "Idle": pointer: 0x2061d07050, stack: 0x619f0000
    
    Thread "xenstore": pointer: 0x2061d07800, stack: 0x61a00000
    
    xenbus initialised on irq 2 mfn 0xfeffc
    
    Thread "shutdown": pointer: 0x2061d07fb0, stack: 0x61a20000
    
    Dummy main: start_info=0x98a200
    
    Thread "main": pointer: 0x2061d08760, stack: 0x61a30000
    
    "main" "root=/dev/sda1" "ro" "console=hvc0" "4" 
    
    vbd 2049 is hd0
    
    ******************* BLKFRONT for device/vbd/2049 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21480/2049
    
    Failed to read /local/domain/0/backend/vbd/21480/2049/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21480/2049/feature-flush-cache.
    
    4194304 sectors of 512 bytes
    
    **************************
    
    vbd 2050 is hd1
    
    ******************* BLKFRONT for device/vbd/2050 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21480/2050
    
    Failed to read /local/domain/0/backend/vbd/21480/2050/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21480/2050/feature-flush-cache.
    
    312705024 sectors of 512 bytes
    
    **************************
    
    vbd 2051 is hd2
    
    ******************* BLKFRONT for device/vbd/2051 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21480/2051
    
    Failed to read /local/domain/0/backend/vbd/21480/2051/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21480/2051/feature-flush-cache.
    
    1835008 sectors of 512 bytes
    
    **************************
    
      Booting 'CentOS Linux (3.10.0-693.11.6.el7.centos.plus.x86_64) 7 (Core)'
    
    
    
    
    
    root (hd0)
    
    
     Filesystem type is ext2fs, using whole disk
    
    
    kernel /boot/vmlinuz-3.10.0-693.11.6.el7.centos.plus.x86_64 ro root=UUID=ade108
    
    
    6f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    
    initrd /boot/initramfs-3.10.0-693.11.6.el7.centos.plus.x86_64.img
    
    
    
    
    
    ============= Init TPM Front ================
    
    Tpmfront:Error Unable to read device/vtpm/0/backend-id during tpmfront initialization! error = ENOENT
    
    Tpmfront:Info Shutting down tpmfront
    
    close blk: backend=/local/domain/0/backend/vbd/21480/2049 node=device/vbd/2049
    
    close blk: backend=/local/domain/0/backend/vbd/21480/2050 node=device/vbd/2050
    
    close blk: backend=/local/domain/0/backend/vbd/21480/2051 node=device/vbd/2051
    
    [    0.000000] Initializing cgroup subsys cpuset
    
    [    0.000000] Initializing cgroup subsys cpu
    
    [    0.000000] Initializing cgroup subsys cpuacct
    
    [    0.000000] Linux version 3.10.0-693.11.6.el7.centos.plus.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Thu Jan 4 14:27:13 UTC 2018
    
    [    0.000000] Command line: ro root=UUID=ade1086f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    [    0.000000] ACPI in unprivileged domain disabled
    
    [    0.000000] e820: BIOS-provided physical RAM map:
    
    [    0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
    
    [    0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
    
    [    0.000000] Xen: [mem 0x0000000000100000-0x0000000062504fff] usable
    
    [    0.000000] NX (Execute Disable) protection: active
    
    [    0.000000] DMI not present or invalid.
    
    [    0.000000] e820: last_pfn = 0x62505 max_arch_pfn = 0x400000000
    
    [    0.000000] RAMDISK: [mem 0x0243e000-0x03730fff]
    
    [    0.000000] NUMA turned off
    
    [    0.000000] Faking a node at [mem 0x0000000000000000-0x0000000062504fff]
    
    [    0.000000] NODE_DATA(0) allocated [mem 0x61cde000-0x61d04fff]
    
    [    0.000000] Zone ranges:
    
    [    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
    
    [    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
    
    [    0.000000]   Normal   empty
    
    [    0.000000] Movable zone start for each node
    
    [    0.000000] Early memory node ranges
    
    [    0.000000]   node   0: [mem 0x00001000-0x0009ffff]
    
    [    0.000000]   node   0: [mem 0x00100000-0x62504fff]
    
    [    0.000000] Initmem setup node 0 [mem 0x00001000-0x62504fff]
    
    [    0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org
    
    [    0.000000] No local APIC present
    
    [    0.000000] APIC: disable apic facility
    
    [    0.000000] APIC: switched to apic NOOP
    
    [    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
    
    [    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
    
    [    0.000000] e820: [mem 0x62505000-0xffffffff] available for PCI devices
    
    [    0.000000] Booting paravirtualized kernel on Xen
    
    [    0.000000] Xen version: 3.4.3.amazon (preserve-AD)
    
    [    0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
    
    [    0.000000] PERCPU: Embedded 35 pages/cpu @ffff880061400000 s104600 r8192 d30568 u2097152
    
    [    0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes)
    
    [    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 396282
    
    [    0.000000] Policy zone: DMA32
    
    [    0.000000] Kernel command line: ro root=UUID=ade1086f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.000000] Memory: 1532640k/1610772k available (6984k kernel code, 388k absent, 77744k reserved, 4581k data, 1800k init)
    
    [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
    
    [    0.000000] x86/pti: Xen PV detected, disabling PTI protection
    
    [    0.000000] Hierarchical RCU implementation.
    
    [    0.000000] 	RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=1.
    
    [    0.000000] NR_IRQS:327936 nr_irqs:24 0
    
    [    0.000000] Console: colour dummy device 80x25
    
    [    0.000000] console [tty0] enabled
    
    [    0.000000] console [hvc0] enabled
    
    [    0.000000] allocated 6815744 bytes of page_cgroup
    
    [    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
    
    [    0.000000] installing Xen timer for CPU 0
    
    [    0.000000] tsc: Fast TSC calibration using PIT
    
    [    0.000000] tsc: Detected 1795.819 MHz processor
    
    [    0.003000] Calibrating delay loop (skipped), value calculated using timer frequency.. 3591.34 BogoMIPS (lpj=1795673)
    
    [    0.004009] pid_max: default: 32768 minimum: 301
    
    [    0.005088] Security Framework initialized
    
    [    0.006027] SELinux:  Initializing.
    
    [    0.007046] Yama: becoming mindful.
    
    [    0.008643] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
    
    [    0.009752] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
    
    [    0.010278] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.011040] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.012392] Initializing cgroup subsys memory
    
    [    0.013038] Initializing cgroup subsys devices
    
    [    0.014016] Initializing cgroup subsys freezer
    
    [    0.015021] Initializing cgroup subsys net_cls
    
    [    0.016013] Initializing cgroup subsys blkio
    
    [    0.017017] Initializing cgroup subsys perf_event
    
    [    0.018017] Initializing cgroup subsys hugetlb
    
    [    0.019022] Initializing cgroup subsys pids
    
    [    0.020014] Initializing cgroup subsys net_prio
    
    [    0.021133] FEATURE SPEC_CTRL Not Present
    
    [    0.022009] FEATURE IBPB_SUPPORT Not Present
    
    [    0.023019] CPU: Physical Processor ID: 0
    
    [    0.024008] CPU: Processor Core ID: 0
    
    [    0.025028] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
    
    [    0.026017] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32
    
    [    0.027010] tlb_flushall_shift: 6
    
    [    0.067936] Freeing SMP alternatives: 24k freed
    
    [    0.074423] ftrace: allocating 26850 entries in 105 pages
    
    [    0.080150] cpu 0 spinlock event irq 17
    
    [    0.081019] smpboot: Max logical packages: 1
    
    [    0.082072] Performance Events: unsupported p6 CPU model 45 no PMU driver, software events only.
    
    [    0.086512] Brought up 1 CPUs
    
    [    0.087178] NMI watchdog: disabled (cpu0): hardware events not enabled
    
    [    0.088019] NMI watchdog: Shutting down hard lockup detector on all cpus
    
    [    0.089209] devtmpfs: initialized
    
    [    0.092000] EVM: security.selinux
    
    [    0.092015] EVM: security.ima
    
    [    0.093017] EVM: security.capability
    
    [    0.095000] atomic64 test passed for x86-64 platform with CX8 and with SSE
    
    [    0.095034] pinctrl core: initialized pinctrl subsystem
    
    [    0.096067] xen:grant_table: Grant tables using version 1 layout
    
    [    0.097070] Grant table initialized
    
    [    0.109710] RTC time: 165:165:165, date: 165/165/65
    
    [    0.110157] NET: Registered protocol family 16
    
    [    0.111899] PCI: setting up Xen PCI frontend stub
    
    [    0.113000] ACPI: Interpreter disabled.
    
    [    0.113064] xen:balloon: Initialising balloon driver
    
    [    0.116035] xen_balloon: Initialising balloon driver
    
    [    0.117229] vgaarb: loaded
    
    [    0.118112] SCSI subsystem initialized
    
    [    0.119099] usbcore: registered new interface driver usbfs
    
    [    0.120030] usbcore: registered new interface driver hub
    
    [    0.121068] usbcore: registered new device driver usb
    
    [    0.122104] dmi: Firmware registration failed.
    
    [    0.123037] PCI: System does not support PCI
    
    [    0.124012] PCI: System does not support PCI
    
    [    0.125154] NetLabel: Initializing
    
    [    0.126007] NetLabel:  domain hash size = 128
    
    [    0.127007] NetLabel:  protocols = UNLABELED CIPSOv4
    
    [    0.128045] NetLabel:  unlabeled traffic allowed by default
    
    [    0.129044] amd_nb: Cannot enumerate AMD northbridges
    
    [    0.130032] Switched to clocksource xen
    
    [    0.142672] pnp: PnP ACPI: disabled
    
    [    0.149900] NET: Registered protocol family 2
    
    [    0.159748] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
    
    [    0.171361] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
    
    [    0.181446] TCP: Hash tables configured (established 16384 bind 16384)
    
    [    0.191429] TCP: reno registered
    
    [    0.196457] UDP hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.205809] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.215831] NET: Registered protocol family 1
    
    [    0.222682] Unpacking initramfs...
    
    [    0.313537] random: fast init done
    
    [    0.751449] Freeing initrd memory: 19404k freed
    
    [    0.790937] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 163840 ms ovfl timer
    
    [    0.803321] RAPL PMU: hw unit of domain pp0-core 2^-16 Joules
    
    [    0.812277] RAPL PMU: hw unit of domain package 2^-16 Joules
    
    [    0.821075] RAPL PMU: hw unit of domain dram 2^-16 Joules
    
    [    0.829506] platform rtc_cmos: registered platform RTC device (no PNP device found)
    
    [    0.841398] sha1_ssse3: Using SSSE3 optimized SHA-1 implementation
    
    [    0.850936] sha256_ssse3: Using SSSE3 optimized SHA-256 implementation
    
    [    0.861264] futex hash table entries: 256 (order: 2, 16384 bytes)
    
    [    0.870747] Initialise system trusted keyring
    
    [    0.877954] audit: initializing netlink socket (disabled)
    
    [    0.886444] type=2000 audit(1515851144.172:1): initialized
    
    [    0.928735] HugeTLB registered 2 MB page size, pre-allocated 0 pages
    
    [    0.940142] zpool: loaded
    
    [    0.944447] zbud: loaded
    
    [    0.948811] VFS: Disk quotas dquot_6.5.2
    
    [    0.955179] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
    
    [    0.965481] msgmni has been set to 3031
    
    [    0.971734] Key type big_key registered
    
    [    0.978980] NET: Registered protocol family 38
    
    [    0.985925] Key type asymmetric registered
    
    [    0.992181] Asymmetric key parser 'x509' registered
    
    [    0.999734] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
    
    [    1.011351] io scheduler noop registered
    
    [    1.017455] io scheduler deadline registered (default)
    
    [    1.025385] io scheduler cfq registered
    
    [    1.031529] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
    
    [    1.040211] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
    
    [    1.050422] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
    
    [    1.060390] Non-volatile memory driver v1.3
    
    [    1.066928] Linux agpgart interface v0.103
    
    [    1.073336] crash memory driver: version 1.1
    
    [    1.080100] rdac: device handler registered
    
    [    1.086457] hp_sw: device handler registered
    
    [    1.093264] emc: device handler registered
    
    [    1.099722] alua: device handler registered
    
    [    1.106137] libphy: Fixed MDIO Bus: probed
    
    [    1.112447] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    
    [    1.122401] ehci-pci: EHCI PCI platform driver
    
    [    1.129752] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
    
    [    1.139547] ohci-pci: OHCI PCI platform driver
    
    [    1.146220] uhci_hcd: USB Universal Host Controller Interface driver
    
    [    1.156146] usbcore: registered new interface driver usbserial
    
    [    1.165566] usbcore: registered new interface driver usbserial_generic
    
    [    1.175533] usbserial: USB Serial support registered for generic
    
    [    1.185025] i8042: PNP: No PS/2 controller found. Probing ports directly.
    
    [    3.175642] i8042: No controller found
    
    [    3.181966] mousedev: PS/2 mouse device common for all mice
    
    [    3.298903] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
    
    [    3.332947] rtc_cmos: probe of rtc_cmos failed with error -38
    
    [    3.341847] dmi-sysfs: dmi entry is absent.
    
    [    3.348309] hidraw: raw HID events driver (C) Jiri Kosina
    
    [    3.356475] usbcore: registered new interface driver usbhid
    
    [    3.365203] usbhid: USB HID core driver
    
    [    3.371135] drop_monitor: Initializing network drop monitor service
    
    [    3.380740] TCP: cubic registered
    
    [    3.385987] Initializing XFRM netlink socket
    
    [    3.392872] NET: Registered protocol family 10
    
    [    3.400357] NET: Registered protocol family 17
    
    [    3.407609] mce: Unable to init device /dev/mcelog (rc: -5)
    
    [    3.416494] microcode: CPU0 sig=0x206d7, pf=0x1, revision=0x712
    
    [    3.426053] microcode: Microcode Update Driver: v2.01 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
    
    [    3.439861] Loading compiled-in X.509 certificates
    
    [    3.447617] Loaded X.509 cert 'CentOS Linux kpatch signing key: ea0413152cde1d98ebdca3fe6f0230904c9ef717'
    
    [    3.462289] Loaded X.509 cert 'CentOS Linux Driver update signing key: 7f421ee0ab69461574bb358861dbe77762a4201b'
    
    [    3.478885] Loaded X.509 cert 'CentOS Linux kernel signing key: 2bade9ee54e382c6692955a95011589b663d6aef'
    
    [    3.493902] registered taskstats version 1
    
    Xen Minimal OS!
    
      start_info: 0xcba000(VA)
    
        nr_pages: 0x61d05
    
      shared_inf: 0x68e37000(MA)
    
         pt_base: 0xcbd000(VA)
    
    nr_pt_frames: 0xb
    
        mfn_list: 0x9ab000(VA)
    
       mod_start: 0x0(VA)
    
         mod_len: 0
    
           flags: 0x300
    
        cmd_line: root=/dev/sda1 ro console=hvc0 4
    
      stack:      0x96a100-0x98a100
    
    MM: Init
    
          _text: 0x0(VA)
    
         _etext: 0x7b824(VA)
    
       _erodata: 0x97000(VA)
    
         _edata: 0x9cce0(VA)
    
    stack start: 0x96a100(VA)
    
           _end: 0x9aa700(VA)
    
      start_pfn: ccb
    
        max_pfn: 61d05
    
    Mapping memory range 0x1000000 - 0x61d05000
    
    setting 0x0-0x97000 readonly
    
    skipped 0x1000
    
    MM: Initialise page allocator for fd3000(fd3000)-61d05000(61d05000)
    
    MM: done
    
    Demand map pfns at 61d06000-2061d06000.
    
    Heap resides at 2061d07000-4061d07000.
    
    Initialising timer interface
    
    Initialising console ... done.
    
    gnttab_table mapped at 0x61d06000.
    
    Initialising scheduler
    
    Thread "Idle": pointer: 0x2061d07050, stack: 0x619f0000
    
    Thread "xenstore": pointer: 0x2061d07800, stack: 0x61a00000
    
    xenbus initialised on irq 2 mfn 0xfeffc
    
    Thread "shutdown": pointer: 0x2061d07fb0, stack: 0x61a20000
    
    Dummy main: start_info=0x98a200
    
    Thread "main": pointer: 0x2061d08760, stack: 0x61a30000
    
    "main" "root=/dev/sda1" "ro" "console=hvc0" "4" 
    
    vbd 2049 is hd0
    
    ******************* BLKFRONT for device/vbd/2049 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21481/2049
    
    Failed to read /local/domain/0/backend/vbd/21481/2049/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21481/2049/feature-flush-cache.
    
    4194304 sectors of 512 bytes
    
    **************************
    
    vbd 2050 is hd1
    
    ******************* BLKFRONT for device/vbd/2050 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21481/2050
    
    Failed to read /local/domain/0/backend/vbd/21481/2050/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21481/2050/feature-flush-cache.
    
    312705024 sectors of 512 bytes
    
    **************************
    
    vbd 2051 is hd2
    
    ******************* BLKFRONT for device/vbd/2051 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21481/2051
    
    Failed to read /local/domain/0/backend/vbd/21481/2051/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21481/2051/feature-flush-cache.
    
    1835008 sectors of 512 bytes
    
    **************************
    
      Booting 'CentOS Linux (3.10.0-693.11.6.el7.centos.plus.x86_64) 7 (Core)'
    
    
    
    
    
    root (hd0)
    
    
     Filesystem type is ext2fs, using whole disk
    
    
    kernel /boot/vmlinuz-3.10.0-693.11.6.el7.centos.plus.x86_64 ro root=UUID=ade108
    
    
    6f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    
    initrd /boot/initramfs-3.10.0-693.11.6.el7.centos.plus.x86_64.img
    
    
    
    
    
    ============= Init TPM Front ================
    
    Tpmfront:Error Unable to read device/vtpm/0/backend-id during tpmfront initialization! error = ENOENT
    
    Tpmfront:Info Shutting down tpmfront
    
    close blk: backend=/local/domain/0/backend/vbd/21481/2049 node=device/vbd/2049
    
    close blk: backend=/local/domain/0/backend/vbd/21481/2050 node=device/vbd/2050
    
    close blk: backend=/local/domain/0/backend/vbd/21481/2051 node=device/vbd/2051
    
    [    0.000000] Initializing cgroup subsys cpuset
    
    [    0.000000] Initializing cgroup subsys cpu
    
    [    0.000000] Initializing cgroup subsys cpuacct
    
    [    0.000000] Linux version 3.10.0-693.11.6.el7.centos.plus.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Thu Jan 4 14:27:13 UTC 2018
    
    [    0.000000] Command line: ro root=UUID=ade1086f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    [    0.000000] ACPI in unprivileged domain disabled
    
    [    0.000000] e820: BIOS-provided physical RAM map:
    
    [    0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
    
    [    0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
    
    [    0.000000] Xen: [mem 0x0000000000100000-0x0000000062504fff] usable
    
    [    0.000000] NX (Execute Disable) protection: active
    
    [    0.000000] DMI not present or invalid.
    
    [    0.000000] e820: last_pfn = 0x62505 max_arch_pfn = 0x400000000
    
    [    0.000000] RAMDISK: [mem 0x0243e000-0x03730fff]
    
    [    0.000000] NUMA turned off
    
    [    0.000000] Faking a node at [mem 0x0000000000000000-0x0000000062504fff]
    
    [    0.000000] NODE_DATA(0) allocated [mem 0x61cde000-0x61d04fff]
    
    [    0.000000] Zone ranges:
    
    [    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
    
    [    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
    
    [    0.000000]   Normal   empty
    
    [    0.000000] Movable zone start for each node
    
    [    0.000000] Early memory node ranges
    
    [    0.000000]   node   0: [mem 0x00001000-0x0009ffff]
    
    [    0.000000]   node   0: [mem 0x00100000-0x62504fff]
    
    [    0.000000] Initmem setup node 0 [mem 0x00001000-0x62504fff]
    
    [    0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org
    
    [    0.000000] No local APIC present
    
    [    0.000000] APIC: disable apic facility
    
    [    0.000000] APIC: switched to apic NOOP
    
    [    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
    
    [    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
    
    [    0.000000] e820: [mem 0x62505000-0xffffffff] available for PCI devices
    
    [    0.000000] Booting paravirtualized kernel on Xen
    
    [    0.000000] Xen version: 3.4.3.amazon (preserve-AD)
    
    [    0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
    
    [    0.000000] PERCPU: Embedded 35 pages/cpu @ffff880061400000 s104600 r8192 d30568 u2097152
    
    [    0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes)
    
    [    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 396282
    
    [    0.000000] Policy zone: DMA32
    
    [    0.000000] Kernel command line: ro root=UUID=ade1086f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.000000] Memory: 1532640k/1610772k available (6984k kernel code, 388k absent, 77744k reserved, 4581k data, 1800k init)
    
    [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
    
    [    0.000000] x86/pti: Xen PV detected, disabling PTI protection
    
    [    0.000000] Hierarchical RCU implementation.
    
    [    0.000000] 	RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=1.
    
    [    0.000000] NR_IRQS:327936 nr_irqs:24 0
    
    [    0.000000] Console: colour dummy device 80x25
    
    [    0.000000] console [tty0] enabled
    
    [    0.000000] console [hvc0] enabled
    
    [    0.000000] allocated 6815744 bytes of page_cgroup
    
    [    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
    
    [    0.000000] installing Xen timer for CPU 0
    
    [    0.000000] tsc: Fast TSC calibration using PIT
    
    [    0.000000] tsc: Detected 1795.719 MHz processor
    
    [    0.003000] Calibrating delay loop (skipped), value calculated using timer frequency.. 3591.34 BogoMIPS (lpj=1795673)
    
    [    0.004017] pid_max: default: 32768 minimum: 301
    
    [    0.005098] Security Framework initialized
    
    [    0.006026] SELinux:  Initializing.
    
    [    0.007050] Yama: becoming mindful.
    
    [    0.008686] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
    
    [    0.009828] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
    
    [    0.010305] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.011046] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.012439] Initializing cgroup subsys memory
    
    [    0.013040] Initializing cgroup subsys devices
    
    [    0.014023] Initializing cgroup subsys freezer
    
    [    0.015023] Initializing cgroup subsys net_cls
    
    [    0.016027] Initializing cgroup subsys blkio
    
    [    0.017028] Initializing cgroup subsys perf_event
    
    [    0.018030] Initializing cgroup subsys hugetlb
    
    [    0.019020] Initializing cgroup subsys pids
    
    [    0.020020] Initializing cgroup subsys net_prio
    
    [    0.021153] FEATURE SPEC_CTRL Not Present
    
    [    0.022021] FEATURE IBPB_SUPPORT Not Present
    
    [    0.023014] CPU: Physical Processor ID: 0
    
    [    0.024016] CPU: Processor Core ID: 0
    
    [    0.025017] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
    
    [    0.026024] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32
    
    [    0.027014] tlb_flushall_shift: 6
    
    [    0.097031] Freeing SMP alternatives: 24k freed
    
    [    0.108049] ftrace: allocating 26850 entries in 105 pages
    
    [    0.118148] cpu 0 spinlock event irq 17
    
    [    0.119024] smpboot: Max logical packages: 1
    
    [    0.120081] Performance Events: unsupported p6 CPU model 45 no PMU driver, software events only.
    
    [    0.124976] Brought up 1 CPUs
    
    [    0.125181] NMI watchdog: disabled (cpu0): hardware events not enabled
    
    [    0.126021] NMI watchdog: Shutting down hard lockup detector on all cpus
    
    [    0.127247] devtmpfs: initialized
    
    [    0.131000] EVM: security.selinux
    
    [    0.132016] EVM: security.ima
    
    [    0.133019] EVM: security.capability
    
    [    0.135425] atomic64 test passed for x86-64 platform with CX8 and with SSE
    
    [    0.136038] pinctrl core: initialized pinctrl subsystem
    
    [    0.137076] xen:grant_table: Grant tables using version 1 layout
    
    [    0.138097] Grant table initialized
    
    [    0.158860] RTC time: 165:165:165, date: 165/165/65
    
    [    0.159193] NET: Registered protocol family 16
    
    [    0.161024] PCI: setting up Xen PCI frontend stub
    
    [    0.163416] ACPI: Interpreter disabled.
    
    [    0.164094] xen:balloon: Initialising balloon driver
    
    [    0.168063] xen_balloon: Initialising balloon driver
    
    [    0.169239] vgaarb: loaded
    
    [    0.170157] SCSI subsystem initialized
    
    [    0.171131] usbcore: registered new interface driver usbfs
    
    [    0.172071] usbcore: registered new interface driver hub
    
    [    0.173126] usbcore: registered new device driver usb
    
    [    0.174216] dmi: Firmware registration failed.
    
    [    0.175063] PCI: System does not support PCI
    
    [    0.176016] PCI: System does not support PCI
    
    [    0.177325] NetLabel: Initializing
    
    [    0.178020] NetLabel:  domain hash size = 128
    
    [    0.179025] NetLabel:  protocols = UNLABELED CIPSOv4
    
    [    0.180073] NetLabel:  unlabeled traffic allowed by default
    
    [    0.181114] amd_nb: Cannot enumerate AMD northbridges
    
    [    0.182087] Switched to clocksource xen
    
    [    0.199245] pnp: PnP ACPI: disabled
    
    [    0.208597] NET: Registered protocol family 2
    
    [    0.219064] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
    
    [    0.234639] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
    
    [    0.248032] TCP: Hash tables configured (established 16384 bind 16384)
    
    [    0.261014] TCP: reno registered
    
    [    0.267848] UDP hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.279841] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.292765] NET: Registered protocol family 1
    
    [    0.301892] Unpacking initramfs...
    
    [    0.315045] random: fast init done
    
    [    0.864749] Freeing initrd memory: 19404k freed
    
    [    0.882503] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 163840 ms ovfl timer
    
    [    0.898833] RAPL PMU: hw unit of domain pp0-core 2^-16 Joules
    
    [    0.910245] RAPL PMU: hw unit of domain package 2^-16 Joules
    
    [    0.921758] RAPL PMU: hw unit of domain dram 2^-16 Joules
    
    [    0.932642] platform rtc_cmos: registered platform RTC device (no PNP device found)
    
    [    0.948836] sha1_ssse3: Using SSSE3 optimized SHA-1 implementation
    
    [    0.961361] sha256_ssse3: Using SSSE3 optimized SHA-256 implementation
    
    [    0.975966] futex hash table entries: 256 (order: 2, 16384 bytes)
    
    [    0.988407] Initialise system trusted keyring
    
    [    0.997143] audit: initializing netlink socket (disabled)
    
    [    1.007983] type=2000 audit(1515851183.567:1): initialized
    
    [    1.054484] HugeTLB registered 2 MB page size, pre-allocated 0 pages
    
    [    1.068680] zpool: loaded
    
    [    1.074123] zbud: loaded
    
    [    1.079618] VFS: Disk quotas dquot_6.5.2
    
    [    1.087497] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
    
    [    1.100464] msgmni has been set to 3031
    
    [    1.108301] Key type big_key registered
    
    [    1.117363] NET: Registered protocol family 38
    
    [    1.126362] Key type asymmetric registered
    
    [    1.134694] Asymmetric key parser 'x509' registered
    
    [    1.144411] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
    
    [    1.159339] io scheduler noop registered
    
    [    1.167597] io scheduler deadline registered (default)
    
    [    1.177823] io scheduler cfq registered
    
    [    1.185634] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
    
    [    1.196439] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
    
    [    1.210179] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
    
    [    1.223251] Non-volatile memory driver v1.3
    
    [    1.231632] Linux agpgart interface v0.103
    
    [    1.240300] crash memory driver: version 1.1
    
    [    1.249463] rdac: device handler registered
    
    [    1.257886] hp_sw: device handler registered
    
    [    1.266558] emc: device handler registered
    
    [    1.274755] alua: device handler registered
    
    [    1.283045] libphy: Fixed MDIO Bus: probed
    
    [    1.291250] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    
    [    1.304124] ehci-pci: EHCI PCI platform driver
    
    [    1.312871] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
    
    [    1.325432] ohci-pci: OHCI PCI platform driver
    
    [    1.334448] uhci_hcd: USB Universal Host Controller Interface driver
    
    [    1.347205] usbcore: registered new interface driver usbserial
    
    [    1.358851] usbcore: registered new interface driver usbserial_generic
    
    [    1.371622] usbserial: USB Serial support registered for generic
    
    [    1.383781] i8042: PNP: No PS/2 controller found. Probing ports directly.
    
    [    3.416560] i8042: No controller found
    
    [    3.424758] mousedev: PS/2 mouse device common for all mice
    
    [    3.543888] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
    
    [    3.556864] rtc_cmos: probe of rtc_cmos failed with error -38
    
    [    3.568766] dmi-sysfs: dmi entry is absent.
    
    [    3.577438] hidraw: raw HID events driver (C) Jiri Kosina
    
    [    3.588561] usbcore: registered new interface driver usbhid
    
    [    3.599564] usbhid: USB HID core driver
    
    [    3.607238] drop_monitor: Initializing network drop monitor service
    
    [    3.620381] TCP: cubic registered
    
    [    3.627568] Initializing XFRM netlink socket
    
    [    3.636822] NET: Registered protocol family 10
    
    [    3.646443] NET: Registered protocol family 17
    
    [    3.655642] mce: Unable to init device /dev/mcelog (rc: -5)
    
    [    3.666897] microcode: CPU0 sig=0x206d7, pf=0x1, revision=0x712
    
    [    3.679130] microcode: Microcode Update Driver: v2.01 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
    
    [    3.697134] Loading compiled-in X.509 certificates
    
    [    3.707843] Loaded X.509 cert 'CentOS Linux kpatch signing key: ea0413152cde1d98ebdca3fe6f0230904c9ef717'
    
    [    3.726809] Loaded X.509 cert 'CentOS Linux Driver update signing key: 7f421ee0ab69461574bb358861dbe77762a4201b'
    
    [    3.748045] Loaded X.509 cert 'CentOS Linux kernel signing key: 2bade9ee54e382c6692955a95011589b663d6aef'
    
    [    3.767979] registered taskstats version 1
    
    Xen Minimal OS!
    
      start_info: 0xcba000(VA)
    
        nr_pages: 0x61d05
    
      shared_inf: 0x68e37000(MA)
    
         pt_base: 0xcbd000(VA)
    
    nr_pt_frames: 0xb
    
        mfn_list: 0x9ab000(VA)
    
       mod_start: 0x0(VA)
    
         mod_len: 0
    
           flags: 0x300
    
        cmd_line: root=/dev/sda1 ro console=hvc0 4
    
      stack:      0x96a100-0x98a100
    
    MM: Init
    
          _text: 0x0(VA)
    
         _etext: 0x7b824(VA)
    
       _erodata: 0x97000(VA)
    
         _edata: 0x9cce0(VA)
    
    stack start: 0x96a100(VA)
    
           _end: 0x9aa700(VA)
    
      start_pfn: ccb
    
        max_pfn: 61d05
    
    Mapping memory range 0x1000000 - 0x61d05000
    
    setting 0x0-0x97000 readonly
    
    skipped 0x1000
    
    MM: Initialise page allocator for fd3000(fd3000)-61d05000(61d05000)
    
    MM: done
    
    Demand map pfns at 61d06000-2061d06000.
    
    Heap resides at 2061d07000-4061d07000.
    
    Initialising timer interface
    
    Initialising console ... done.
    
    gnttab_table mapped at 0x61d06000.
    
    Initialising scheduler
    
    Thread "Idle": pointer: 0x2061d07050, stack: 0x619f0000
    
    Thread "xenstore": pointer: 0x2061d07800, stack: 0x61a00000
    
    xenbus initialised on irq 2 mfn 0xfeffc
    
    Thread "shutdown": pointer: 0x2061d07fb0, stack: 0x61a20000
    
    Dummy main: start_info=0x98a200
    
    Thread "main": pointer: 0x2061d08760, stack: 0x61a30000
    
    "main" "root=/dev/sda1" "ro" "console=hvc0" "4" 
    
    vbd 2049 is hd0
    
    ******************* BLKFRONT for device/vbd/2049 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21482/2049
    
    Failed to read /local/domain/0/backend/vbd/21482/2049/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21482/2049/feature-flush-cache.
    
    4194304 sectors of 512 bytes
    
    **************************
    
    vbd 2050 is hd1
    
    ******************* BLKFRONT for device/vbd/2050 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21482/2050
    
    Failed to read /local/domain/0/backend/vbd/21482/2050/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21482/2050/feature-flush-cache.
    
    312705024 sectors of 512 bytes
    
    **************************
    
    vbd 2051 is hd2
    
    ******************* BLKFRONT for device/vbd/2051 **********
    
    
    
    
    
    backend at /local/domain/0/backend/vbd/21482/2051
    
    Failed to read /local/domain/0/backend/vbd/21482/2051/feature-barrier.
    
    Failed to read /local/domain/0/backend/vbd/21482/2051/feature-flush-cache.
    
    1835008 sectors of 512 bytes
    
    **************************
    
      Booting 'CentOS Linux (3.10.0-693.11.6.el7.centos.plus.x86_64) 7 (Core)'
    
    
    
    
    
    root (hd0)
    
    
     Filesystem type is ext2fs, using whole disk
    
    
    kernel /boot/vmlinuz-3.10.0-693.11.6.el7.centos.plus.x86_64 ro root=UUID=ade108
    
    
    6f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    
    initrd /boot/initramfs-3.10.0-693.11.6.el7.centos.plus.x86_64.img
    
    
    
    
    
    ============= Init TPM Front ================
    
    Tpmfront:Error Unable to read device/vtpm/0/backend-id during tpmfront initialization! error = ENOENT
    
    Tpmfront:Info Shutting down tpmfront
    
    close blk: backend=/local/domain/0/backend/vbd/21482/2049 node=device/vbd/2049
    
    close blk: backend=/local/domain/0/backend/vbd/21482/2050 node=device/vbd/2050
    
    close blk: backend=/local/domain/0/backend/vbd/21482/2051 node=device/vbd/2051
    
    [    0.000000] Initializing cgroup subsys cpuset
    
    [    0.000000] Initializing cgroup subsys cpu
    
    [    0.000000] Initializing cgroup subsys cpuacct
    
    [    0.000000] Linux version 3.10.0-693.11.6.el7.centos.plus.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) ) #1 SMP Thu Jan 4 14:27:13 UTC 2018
    
    [    0.000000] Command line: ro root=UUID=ade1086f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    [    0.000000] ACPI in unprivileged domain disabled
    
    [    0.000000] e820: BIOS-provided physical RAM map:
    
    [    0.000000] Xen: [mem 0x0000000000000000-0x000000000009ffff] usable
    
    [    0.000000] Xen: [mem 0x00000000000a0000-0x00000000000fffff] reserved
    
    [    0.000000] Xen: [mem 0x0000000000100000-0x0000000062504fff] usable
    
    [    0.000000] NX (Execute Disable) protection: active
    
    [    0.000000] DMI not present or invalid.
    
    [    0.000000] e820: last_pfn = 0x62505 max_arch_pfn = 0x400000000
    
    [    0.000000] RAMDISK: [mem 0x0243e000-0x03730fff]
    
    [    0.000000] NUMA turned off
    
    [    0.000000] Faking a node at [mem 0x0000000000000000-0x0000000062504fff]
    
    [    0.000000] NODE_DATA(0) allocated [mem 0x61cde000-0x61d04fff]
    
    [    0.000000] Zone ranges:
    
    [    0.000000]   DMA      [mem 0x00001000-0x00ffffff]
    
    [    0.000000]   DMA32    [mem 0x01000000-0xffffffff]
    
    [    0.000000]   Normal   empty
    
    [    0.000000] Movable zone start for each node
    
    [    0.000000] Early memory node ranges
    
    [    0.000000]   node   0: [mem 0x00001000-0x0009ffff]
    
    [    0.000000]   node   0: [mem 0x00100000-0x62504fff]
    
    [    0.000000] Initmem setup node 0 [mem 0x00001000-0x62504fff]
    
    [    0.000000] SFI: Simple Firmware Interface v0.81 http://simplefirmware.org
    
    [    0.000000] No local APIC present
    
    [    0.000000] APIC: disable apic facility
    
    [    0.000000] APIC: switched to apic NOOP
    
    [    0.000000] smpboot: Allowing 1 CPUs, 0 hotplug CPUs
    
    [    0.000000] PM: Registered nosave memory: [mem 0x000a0000-0x000fffff]
    
    [    0.000000] e820: [mem 0x62505000-0xffffffff] available for PCI devices
    
    [    0.000000] Booting paravirtualized kernel on Xen
    
    [    0.000000] Xen version: 3.4.3.amazon (preserve-AD)
    
    [    0.000000] setup_percpu: NR_CPUS:5120 nr_cpumask_bits:1 nr_cpu_ids:1 nr_node_ids:1
    
    [    0.000000] PERCPU: Embedded 35 pages/cpu @ffff880061400000 s104600 r8192 d30568 u2097152
    
    [    0.000000] PV qspinlock hash table entries: 256 (order: 0, 4096 bytes)
    
    [    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 396282
    
    [    0.000000] Policy zone: DMA32
    
    [    0.000000] Kernel command line: ro root=UUID=ade1086f-b766-4089-9f8b-e3b0c28979f3 consoleblank=0 console=hvc0
    
    [    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.000000] Memory: 1532640k/1610772k available (6984k kernel code, 388k absent, 77744k reserved, 4581k data, 1800k init)
    
    [    0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=1, Nodes=1
    
    [    0.000000] x86/pti: Xen PV detected, disabling PTI protection
    
    [    0.000000] Hierarchical RCU implementation.
    
    [    0.000000] 	RCU restricting CPUs from NR_CPUS=5120 to nr_cpu_ids=1.
    
    [    0.000000] NR_IRQS:327936 nr_irqs:24 0
    
    [    0.000000] Console: colour dummy device 80x25
    
    [    0.000000] console [tty0] enabled
    
    [    0.000000] console [hvc0] enabled
    
    [    0.000000] allocated 6815744 bytes of page_cgroup
    
    [    0.000000] please try 'cgroup_disable=memory' option if you don't want memory cgroups
    
    [    0.000000] installing Xen timer for CPU 0
    
    [    0.000000] tsc: Fast TSC calibration using PIT
    
    [    0.000000] tsc: Detected 1795.806 MHz processor
    
    [    0.003000] Calibrating delay loop (skipped), value calculated using timer frequency.. 3591.34 BogoMIPS (lpj=1795672)
    
    [    0.004000] pid_max: default: 32768 minimum: 301
    
    [    0.004097] Security Framework initialized
    
    [    0.005028] SELinux:  Initializing.
    
    [    0.006053] Yama: becoming mindful.
    
    [    0.007650] Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
    
    [    0.008825] Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
    
    [    0.009272] Mount-cache hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.010050] Mountpoint-cache hash table entries: 4096 (order: 3, 32768 bytes)
    
    [    0.011423] Initializing cgroup subsys memory
    
    [    0.012037] Initializing cgroup subsys devices
    
    [    0.013023] Initializing cgroup subsys freezer
    
    [    0.014031] Initializing cgroup subsys net_cls
    
    [    0.015034] Initializing cgroup subsys blkio
    
    [    0.016020] Initializing cgroup subsys perf_event
    
    [    0.017020] Initializing cgroup subsys hugetlb
    
    [    0.018020] Initializing cgroup subsys pids
    
    [    0.019028] Initializing cgroup subsys net_prio
    
    [    0.020173] FEATURE SPEC_CTRL Not Present
    
    [    0.021022] FEATURE IBPB_SUPPORT Not Present
    
    [    0.022023] CPU: Physical Processor ID: 0
    
    [    0.023019] CPU: Processor Core ID: 0
    
    [    0.024031] Last level iTLB entries: 4KB 512, 2MB 0, 4MB 0
    
    [    0.025033] Last level dTLB entries: 4KB 512, 2MB 32, 4MB 32
    
    [    0.026016] tlb_flushall_shift: 6
    
    [    0.064237] Freeing SMP alternatives: 24k freed
    
    [    0.070363] ftrace: allocating 26850 entries in 105 pages
    
    [    0.076160] cpu 0 spinlock event irq 17
    
    [    0.077018] smpboot: Max logical packages: 1
    
    [    0.078083] Performance Events: unsupported p6 CPU model 45 no PMU driver, software events only.
    
    [    0.082401] Brought up 1 CPUs
    
    [    0.083223] NMI watchdog: disabled (cpu0): hardware events not enabled
    
    [    0.084013] NMI watchdog: Shutting down hard lockup detector on all cpus
    
    [    0.085233] devtmpfs: initialized
    
    [    0.088000] EVM: security.selinux
    
    [    0.088026] EVM: security.ima
    
    [    0.089018] EVM: security.capability
    
    [    0.091000] atomic64 test passed for x86-64 platform with CX8 and with SSE
    
    [    0.091040] pinctrl core: initialized pinctrl subsystem
    
    [    0.092123] xen:grant_table: Grant tables using version 1 layout
    
    [    0.093114] Grant table initialized
    
    [    0.105228] RTC time: 165:165:165, date: 165/165/65
    
    [    0.106214] NET: Registered protocol family 16
    
    [    0.107909] PCI: setting up Xen PCI frontend stub
    
    [    0.109000] ACPI: Interpreter disabled.
    
    [    0.109082] xen:balloon: Initialising balloon driver
    
    [    0.112029] xen_balloon: Initialising balloon driver
    
    [    0.113210] vgaarb: loaded
    
    [    0.114140] SCSI subsystem initialized
    
    [    0.115106] usbcore: registered new interface driver usbfs
    
    [    0.116041] usbcore: registered new interface driver hub
    
    [    0.117099] usbcore: registered new device driver usb
    
    [    0.118153] dmi: Firmware registration failed.
    
    [    0.119052] PCI: System does not support PCI
    
    [    0.120016] PCI: System does not support PCI
    
    [    0.121242] NetLabel: Initializing
    
    [    0.122018] NetLabel:  domain hash size = 128
    
    [    0.123009] NetLabel:  protocols = UNLABELED CIPSOv4
    
    [    0.124060] NetLabel:  unlabeled traffic allowed by default
    
    [    0.125092] amd_nb: Cannot enumerate AMD northbridges
    
    [    0.126043] Switched to clocksource xen
    
    [    0.140743] pnp: PnP ACPI: disabled
    
    [    0.149606] NET: Registered protocol family 2
    
    [    0.158530] TCP established hash table entries: 16384 (order: 5, 131072 bytes)
    
    [    0.172639] TCP bind hash table entries: 16384 (order: 6, 262144 bytes)
    
    [    0.185290] TCP: Hash tables configured (established 16384 bind 16384)
    
    [    0.197702] TCP: reno registered
    
    [    0.204071] UDP hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.216075] UDP-Lite hash table entries: 1024 (order: 3, 32768 bytes)
    
    [    0.228633] NET: Registered protocol family 1
    
    [    0.237750] Unpacking initramfs...
    
    [    0.369479] random: fast init done
    
    [    0.812227] Freeing initrd memory: 19404k freed
    
    [    0.829207] RAPL PMU: API unit is 2^-32 Joules, 3 fixed counters, 163840 ms ovfl timer
    
    [    0.847981] RAPL PMU: hw unit of domain pp0-core 2^-16 Joules
    
    [    0.858970] RAPL PMU: hw unit of domain package 2^-16 Joules
    
    [    0.870339] RAPL PMU: hw unit of domain dram 2^-16 Joules
    
    [    0.881408] platform rtc_cmos: registered platform RTC device (no PNP device found)
    
    [    0.896268] sha1_ssse3: Using SSSE3 optimized SHA-1 implementation
    
    [    0.908284] sha256_ssse3: Using SSSE3 optimized SHA-256 implementation
    
    [    0.921239] futex hash table entries: 256 (order: 2, 16384 bytes)
    
    [    0.932918] Initialise system trusted keyring
    
    [    0.941410] audit: initializing netlink socket (disabled)
    
    [    0.951788] type=2000 audit(1515851217.858:1): initialized
    
    [    1.021248] HugeTLB registered 2 MB page size, pre-allocated 0 pages
    
    [    1.035196] zpool: loaded
    
    [    1.035196] zbud: loaded
    
    [    1.045980] VFS: Disk quotas dquot_6.5.2
    
    [    1.053654] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
    
    [    1.066576] msgmni has been set to 3031
    
    [    1.067244] Key type big_key registered
    
    [    1.082965] NET: Registered protocol family 38
    
    [    1.091844] Key type asymmetric registered
    
    [    1.099826] Asymmetric key parser 'x509' registered
    
    [    1.109567] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 250)
    
    [    1.123911] io scheduler noop registered
    
    [    1.131564] io scheduler deadline registered (default)
    
    [    1.141809] io scheduler cfq registered
    
    [    1.149393] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
    
    [    1.160247] pciehp: PCI Express Hot Plug Controller Driver version: 0.4
    
    [    1.173417] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
    
    [    1.186569] Non-volatile memory driver v1.3
    
    [    1.194905] Linux agpgart interface v0.103
    
    [    1.203037] crash memory driver: version 1.1
    
    [    1.211751] rdac: device handler registered
    
    [    1.220166] hp_sw: device handler registered
    
    [    1.229060] emc: device handler registered
    
    [    1.237808] alua: device handler registered
    
    [    1.246499] libphy: Fixed MDIO Bus: probed
    
    [    1.254925] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
    
    [    1.268099] ehci-pci: EHCI PCI platform driver
    
    [    1.269023] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
    
    [    1.289595] ohci-pci: OHCI PCI platform driver
    
    [    1.290550] uhci_hcd: USB Universal Host Controller Interface driver
    
    [    1.311474] usbcore: registered new interface driver usbserial
    
    [    1.322858] usbcore: registered new interface driver usbserial_generic
    
    [    1.335999] usbserial: USB Serial support registered for generic
    
    [    1.349056] i8042: PNP: No PS/2 controller found. Probing ports directly.
    
    [    3.400514] i8042: No controller found
    
    [    3.408389] mousedev: PS/2 mouse device common for all mice
    
    [    3.527531] rtc_cmos rtc_cmos: rtc core: registered rtc_cmos as rtc0
    
    [    3.540338] rtc_cmos: probe of rtc_cmos failed with error -38
    
    [    3.551716] dmi-sysfs: dmi entry is absent.
    
    [    3.559914] hidraw: raw HID events driver (C) Jiri Kosina
    
    [    3.570512] usbcore: registered new interface driver usbhid
    
    [    3.581218] usbhid: USB HID core driver
    
    [    3.588925] drop_monitor: Initializing network drop monitor service
    
    [    3.601194] TCP: cubic registered
    
    [    3.608001] Initializing XFRM netlink socket
    
    [    3.616800] NET: Registered protocol family 10
    
    [    3.625903] NET: Registered protocol family 17
    
    [    3.634501] mce: Unable to init device /dev/mcelog (rc: -5)
    
    [    3.645168] microcode: CPU0 sig=0x206d7, pf=0x1, revision=0x712
    
    [    3.656568] microcode: Microcode Update Driver: v2.01 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
    
    [    3.674563] Loading compiled-in X.509 certificates
    
    [    3.683010] Loaded X.509 cert 'CentOS Linux kpatch signing key: ea0413152cde1d98ebdca3fe6f0230904c9ef717'
    
    [    3.700538] Loaded X.509 cert 'CentOS Linux Driver update signing key: 7f421ee0ab69461574bb358861dbe77762a4201b'
    
    [    3.720636] Loaded X.509 cert 'CentOS Linux kernel signing key: 2bade9ee54e382c6692955a95011589b663d6aef'
    
    [    3.738829] registered taskstats version 1
    
    	2018-01-13T13:47:04.000Z
    

-Relationships
+Relationships

-Notes

~0029977

kstange (reporter)

Running xen version on host:

xen-4.4.4-26
kernel-4.9.44-29

~0029978

kstange (reporter)

Created a bug at RH bugzilla:

https://bugzilla.redhat.com/show_bug.cgi?id=1487754

Anyone who wants a CC to the bug to comment or view, please let me know the email to add... you can ping me on IRC (kstange) if you would rather.

~0029979

arrfab (administrator)

Confirmed through CentOS QA jobs and kernel panic so unable to setup a new PV domU guest with the 7.4.1708 tree

~0030014

kstange (reporter)

Things I have done since filing this bug:

- I tested booting my CentOS virt kernel 4.9.44 and it boots correctly.

- I tested an official RHEL 7.4 PV guest and it does not boot.

- From the Red Hat bug, it was suggested that I rebuild with CONFIG_RANDOMIZE_BASE=n and this allowed the kernel to boot.

- I did some digging based on this change and found this thread:

https://lists.xen.org/archives/html/xen-devel/2014-08/msg01024.html

The patch proposed is here in its final form:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?h=v4.13&id=0b5a50635fc916cf46e3de0b819a61fc3f17e7ee

It doesn't apply cleanly to 3.10.0, but I backported it to the 693.1.1 kernel and applied it, and I can now boot my PV guest properly with CONFIG_RANDOMIZE_BASE=y.

- I have asked, but I am doubtful that Red Hat will accept this patch. We could potentially apply it to CentOS Plus kernels, but that doesn't solve the installer problem unless we have an alternate installer for PV guests.

~0030021

kstange (reporter)

With many thanks to hughesjr and toracat, the patch indicated in my previous comment is now in CentOS Plus kernel 3.10.0-693.2.1. I've asked Red Hat to apply it to some future kernel update, but that is only a dream for now.

In the meantime, if anyone who has been experiencing the issue with PV domains can try out the CentOS Plus kernel here and provide feedback, I'd appreciate it!

https://buildlogs.centos.org/c7-plus/kernel-plus/20170907163005/3.10.0-693.2.1.el7.centos.plus.x86_64/

~0030077

avij (manager)

For the record, those 3.10.0-693.* CentOS Plus kernels are now available via the regular centosplus repository.

https://wiki.centos.org/AdditionalResources/Repositories/CentOSPlus
http://mirror.centos.org/centos/7.4.1708/centosplus/x86_64/Packages/

~0030095

bill_mcgonigle (reporter)

Kevin, thanks for managing this bug for us - saved my bacon. I ran into it when rebooting a database server and have proactively switched to kernel-plus on a dozen other VM's, both in-house and cloud-hosted (on XenPV). They have been fully updated and rebooted successfully. I've started a blog entry/howto on this issue, which is really major IMO <https://www.bfccomputing.com/2017/09/15/centos-plus-kernel-for-xen.html> - if you can add me to the RHBZ, I'll verify that my claims are accurate. RHBZ id is bill-bugzilla.redhat.com@bfccomputing.com .

~0030169

PryMar56 (reporter)

In order to launch any Xen pv install we need another pxeboot pair, vmlinuz/initrd.img, that have the desired patch.
The tool of choice to make these is `lorax`, or pylorax.

It turns out that kernel-plus can be used in lorax by changing the first kernel -> kernel-plus in:
/usr/share/lorax/runtime-install.tmpl

The resulting boot.iso, pxeboot/{vmlinuz,initrd.img} can be used launch a Xen pv netinstall. I've done it.

~0030194

afletdinov (reporter)

confirmed, for 693.2.2 kernel:
[ 1.750052] ------------[ cut here ]------------
[ 1.750067] WARNING: CPU: 0 PID: 207 at mm/vmalloc.c:131 vmap_page_range_noflush+0x2c1/0x350
[ 1.750072] Modules linked in:
[ 1.750077] CPU: 0 PID: 207 Comm: systemd-udevd Not tainted 3.10.0-693.2.2.el7.x86_64 #1
[ 1.750082] 0000000000000000 0000000045c2d9dc ffff880003153bd8 ffffffff816a3db1
[ 1.750088] ffff880003153c18 ffffffff810879c8 00000083811c14e8 ffff8800036b5000
[ 1.750094] 0000000000000001 ffff8800030f1e80 ffffffffc0000000 0000000000000000
[ 1.750099] Call Trace:
[ 1.750108] [<ffffffff816a3db1>] dump_stack+0x19/0x1b
[ 1.750115] [<ffffffff810879c8>] __warn+0xd8/0x100
[ 1.750119] [<ffffffff81087b0d>] warn_slowpath_null+0x1d/0x20
[ 1.750123] [<ffffffff811c0781>] vmap_page_range_noflush+0x2c1/0x350
[ 1.750127] [<ffffffff811c083e>] map_vm_area+0x2e/0x40
[ 1.750131] [<ffffffff811c1590>] __vmalloc_node_range+0x170/0x270
[ 1.750137] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.750141] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.750147] [<ffffffff8105f143>] module_alloc+0x73/0xd0
[ 1.750150] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.750154] [<ffffffff810fe754>] module_alloc_update_bounds+0x14/0x70
[ 1.750158] [<ffffffff810ff2d2>] load_module+0xb02/0x29e0
[ 1.750161] [<ffffffff811c0717>] ? vmap_page_range_noflush+0x257/0x350
[ 1.750165] [<ffffffff811c083e>] ? map_vm_area+0x2e/0x40
[ 1.750168] [<ffffffff811c1590>] ? __vmalloc_node_range+0x170/0x270
[ 1.750172] [<ffffffff81101249>] ? SyS_init_module+0x99/0x110
[ 1.750175] [<ffffffff81101275>] SyS_init_module+0xc5/0x110
[ 1.750181] [<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
[ 1.750185] ---[ end trace 5fc1bf96e70088cb ]---
[ 1.750188] vmalloc: allocation failure, allocated 24576 of 28672 bytes
[ 1.750192] systemd-udevd: page allocation failure: order:0, mode:0xd2
[ 1.750195] CPU: 0 PID: 207 Comm: systemd-udevd Tainted: G W ------------ 3.10.0-693.2.2.el7.x86_64 #1
[ 1.750200] 00000000000000d2 0000000045c2d9dc ffff880003153c48 ffffffff816a3db1
[ 1.750205] ffff880003153cd8 ffffffff81188810 ffffffff8190ea38 ffff880003153c68
[ 1.750211] ffffffff00000018 ffff880003153ce8 ffff880003153c88 0000000045c2d9dc
[ 1.750216] Call Trace:
[ 1.750219] [<ffffffff816a3db1>] dump_stack+0x19/0x1b
[ 1.750227] [<ffffffff81188810>] warn_alloc_failed+0x110/0x180
[ 1.750231] [<ffffffff811c1654>] __vmalloc_node_range+0x234/0x270
[ 1.750236] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.750239] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.750243] [<ffffffff8105f143>] module_alloc+0x73/0xd0
[ 1.750246] [<ffffffff810fe754>] ? module_alloc_update_bounds+0x14/0x70
[ 1.750250] [<ffffffff810fe754>] module_alloc_update_bounds+0x14/0x70
[ 1.750253] [<ffffffff810ff2d2>] load_module+0xb02/0x29e0
[ 1.750256] [<ffffffff811c0717>] ? vmap_page_range_noflush+0x257/0x350
[ 1.750260] [<ffffffff811c083e>] ? map_vm_area+0x2e/0x40
[ 1.750263] [<ffffffff811c1590>] ? __vmalloc_node_range+0x170/0x270
[ 1.750267] [<ffffffff81101249>] ? SyS_init_module+0x99/0x110
[ 1.750270] [<ffffffff81101275>] SyS_init_module+0xc5/0x110
[ 1.750274] [<ffffffff816b5009>] system_call_fastpath+0x16/0x1b
[ 1.750278] Mem-Info:
[ 1.750282] active_anon:908 inactive_anon:1583 isolated_anon:0
[ 1.750282] active_file:1848 inactive_file:7699 isolated_file:0
[ 1.750282] unevictable:0 dirty:0 writeback:0 unstable:0
[ 1.750282] slab_reclaimable:1677 slab_unreclaimable:1124
[ 1.750282] mapped:874 shmem:1586 pagetables:313 bounce:0
[ 1.750282] free:235835 free_pcp:52 free_cma:0
[ 1.750296] Node 0 DMA free:15536kB min:60kB low:72kB high:88kB active_anon:128kB inactive_anon:168kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15996kB managed:15912kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:168kB slab_reclaimable:24kB slab_unreclaimable:16kB kernel_stack:0kB pagetables:32kB unstable:0kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[ 1.750315] lowmem_reserve[]: 0 971 971 971
[ 1.750320] Node 0 DMA32 free:927804kB min:3956kB low:4944kB high:5932kB active_anon:3504kB inactive_anon:6164kB active_file:7392kB inactive_file:30796kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:1032192kB managed:996556kB mlocked:0kB dirty:0kB writeback:0kB mapped:3496kB shmem:6176kB slab_reclaimable:6684kB slab_unreclaimable:4480kB kernel_stack:720kB pagetables:1220kB unstable:0kB bounce:0kB free_pcp:208kB local_pcp:208kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[ 1.750337] lowmem_reserve[]: 0 0 0 0
[ 1.750341] Node 0 DMA: 3*4kB (UM) 4*8kB (UEM) 0*16kB 2*32kB (UE) 3*64kB (UEM) 3*128kB (UEM) 2*256kB (UE) 2*512kB (EM) 3*1024kB (UEM) 1*2048kB (E) 2*4096kB (M) = 15532kB
[ 1.750358] Node 0 DMA32: 1*4kB (E) 1*8kB (U) 1*16kB (E) 3*32kB (UEM) 3*64kB (UEM) 2*128kB (UM) 2*256kB (UE) 2*512kB (UM) 4*1024kB (EM) 2*2048kB (M) 224*4096kB (M) = 927804kB
[ 1.750375] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[ 1.750379] 11133 total pagecache pages
[ 1.750382] 0 pages in swap cache
[ 1.750385] Swap cache stats: add 0, delete 0, find 0/0
[ 1.750387] Free swap = 0kB
[ 1.750390] Total swap = 0kB
[ 1.750392] 262047 pages RAM
[ 1.750394] 0 pages HighMem/MovableOnly
[ 1.750396] 8930 pages reserved

~0030281

kstange (reporter)

I have some updated information from the Red Hat bug!

Red Hat has committed a version of the patch and produced it in test build 3.10.0-720.el7, which puts the commit ON_QA now. This signals that the fix most likely will not appear until RHEL 7.5 unless there is an additional push to put the patch into the 693/EL7.4 kernel series.

I'm trying to get a copy of the 720 kernel to verify that the fix resolves the issue and will report back.

~0030314

Gene (reporter)

I got hit with this on the 3.10.0-693 kernel. Switched back to the previous kernel (3.10.0-514) to get the system to boot.

There was a ten month wait between the 7.3 and 7.4 releases, and 7.4 just came out last month. Waiting ~9 months for a fix is unreasonable. Can't the CentOS project put in a fix for this in the meantime?

~0030315

kstange (reporter)

There is a fix. Switch to using the kernel-plus package from the centos repo. CentOS policy doesn't allow modifying the main kernel package to deviate from the Red Hat sources.

~0030333

kjohnson (reporter)

I tried the kernel-plus and had this kernel panic on first boot. I rebooted and the second boot was fine. Any idea?

3.10.0-693.2.2.el7.centos.plus.x86_64 in PV mode:

[ 13.372417] ------------[ cut here ]------------
[ 13.372434] kernel BUG at arch/x86/kernel/paravirt.c:252!
[ 13.372441] invalid opcode: 0000 [#1] SMP
[ 13.372450] Modules linked in: xt_owner nf_nat_ftp xt_REDIRECT nf_nat_redirect xt_conntrack iptable_mangle nf_conntrack_ftp xt_LOG xt_limit xt_multiport iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack ipt_REJECT nf_reject_ipv4 iptable_filter vfat fat isofs xfs libcrc32c loop sb_edac edac_core coretemp intel_rapl iosf_mbi crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper ablk_helper cryptd pcspkr ip_tables ext4 mbcache jbd2 xen_blkfront xen_netfront crct10dif_pclmul crct10dif_common crc32c_intel
[ 13.372545] CPU: 0 PID: 1138 Comm: mysqld Not tainted 3.10.0-693.2.2.el7.centos.plus.x86_64 #1
[ 13.372555] task: ffff8800fb1a8fd0 ti: ffff8801e9e20000 task.ti: ffff8801e9e20000
[ 13.372561] RIP: e030:[<ffffffff816ad7fe>] [<ffffffff816ad7fe>] enter_lazy.part.0+0x4/0x6
[ 13.372579] RSP: e02b:ffff8801fea03a80 EFLAGS: 00010002
[ 13.372584] RAX: 0000000000000001 RBX: ffff88017d05b280 RCX: ffffffff810215a0
[ 13.372593] RDX: ffff880000000280 RSI: 00003ffffffff000 RDI: 0000000000000001
[ 13.372599] RBP: ffff8801fea03a80 R08: ffff8801fea03b50 R09: ffffffff813f6559
[ 13.372605] R10: ffff88017fc01d00 R11: ffffea00000c0380 R12: ffffc90000c52000
[ 13.372616] R13: ffffc90000c50000 R14: 0000000000000000 R15: ffff8801e3770e00
[ 13.372632] FS: 00007febcc67a900(0000) GS:ffff8801fea00000(0000) knlGS:ffff8801fea00000
[ 13.372644] CS: e033 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 13.372654] CR2: 000056038881bd60 CR3: 00000000f996d000 CR4: 0000000000042660
[ 13.372663] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 13.372672] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[ 13.372680] Stack:
[ 13.372685] ffff8801fea03a90 ffffffff81063b05 ffff8801fea03b38 ffffffff811b2a80
[ 13.372701] ffffc90000c51fff ffffc90000c52000 ffffffff81a0ac90 ffffc90000c51fff
[ 13.372718] ffffc90000c52000 ffff88017fc50000 ffffffff810215a0 00000000fead6d00
[ 13.372733] Call Trace:
[ 13.372740] <IRQ>
[ 13.372744] [<ffffffff81063b05>] paravirt_enter_lazy_mmu+0x25/0x30
[ 13.372771] [<ffffffff811b2a80>] apply_to_page_range+0x260/0x430
[ 13.372784] [<ffffffff810215a0>] ? map_pte_fn+0x60/0x60
[ 13.372794] [<ffffffff810216eb>] arch_gnttab_map_status+0x3b/0x70
[ 13.372809] [<ffffffff813f65d6>] gnttab_map_frames_v2+0xd6/0x150
[ 13.372820] [<ffffffff813f66f1>] gnttab_map+0xa1/0x140
[ 13.372831] [<ffffffff813f6890>] get_free_entries+0x100/0x2e0

~0030334

kjohnson (reporter)

For my note above, it seems to be this bug:

https://lists.centos.org/pipermail/centos-virt/2017-May/005540.html

~0030925

adrian27 (reporter)

I had the same issue booting a CentOS 7 PV instance-store AMI in AWS EC2. I started with the latest CentOS 7.4 kernel 3.10.0-693.11.6.el7.x86_64, and after a few days of troubleshooting I managed to boot the instance using older 3.10.0-514* kernels from CentOS 7.3.

Then I found this issue reported here, so I start using CentOS Plus kernels. The CentOS 7 PV instance-store AMI boots up properly with CentOS Plus kernel versions kernel-plus-3.10.0-693.2.1 to 693.11.1, however the latest CentOS Plus kernel kernel-plus-3.10.0-693.11.6 stops working, and the boot is interrupted after the following message, then the VM reboots:

(...)
[ 3.570512] usbcore: registered new interface driver usbhid
[ 3.581218] usbhid: USB HID core driver
[ 3.588925] drop_monitor: Initializing network drop monitor service
[ 3.601194] TCP: cubic registered
[ 3.608001] Initializing XFRM netlink socket
[ 3.616800] NET: Registered protocol family 10
[ 3.625903] NET: Registered protocol family 17
[ 3.634501] mce: Unable to init device /dev/mcelog (rc: -5)
[ 3.645168] microcode: CPU0 sig=0x206d7, pf=0x1, revision=0x712
[ 3.656568] microcode: Microcode Update Driver: v2.01 <tigran@aivazian.fsnet.co.uk>, Peter Oruba
[ 3.674563] Loading compiled-in X.509 certificates
[ 3.683010] Loaded X.509 cert 'CentOS Linux kpatch signing key: ea0413152cde1d98ebdca3fe6f0230904c9ef717'
[ 3.700538] Loaded X.509 cert 'CentOS Linux Driver update signing key: 7f421ee0ab69461574bb358861dbe77762a4201b'
[ 3.720636] Loaded X.509 cert 'CentOS Linux kernel signing key: 2bade9ee54e382c6692955a95011589b663d6aef'
[ 3.738829] registered taskstats version 1

The full kernel boot log as retrieved from AWS EC2 is attached.

~0030926

adrian27 (reporter)

As a side note, the same issue reported above with the latest CentOS Plus kernel 3.10.0-693.11.6 happens with the latest CentOS 7.4 standard kernel 3.10.0-693.11.6 as well. So this might be another issue introduced by RHEL linux kernels.

~0030927

toracat (manager)

@adrian27

See https://lists.centos.org/pipermail/centos-virt/2018-January/005716.html for a report on the centos-virt mailing list. In that thread, a possible solution is provided by @JohnnyHughes:

https://lists.centos.org/pipermail/centos-virt/2018-January/005721.html

~0030928

adrian27 (reporter)

As another side node, I made a diff between the config files of the CentOS Plus kernels 693.11.1 and 693.11.6, and the only difference is the CONFIG_KAISER=y that appears in the 693.11.6 kernel only ( and which I think causes the boot to hang at message [ 3.738829] registered taskstats version 1)

# diff 11.1/boot/config-3.10.0-693.11.1.el7.centos.plus.x86_64 11.6/boot/config-3.10.0-693.11.6.el7.centos.plus.x86_64
3c3
< # Linux/x86_64 3.10.0-693.11.1.el7.centos.plus.x86_64 Kernel Configuration
---
> # Linux/x86_64 3.10.0-693.11.6.el7.centos.plus.x86_64 Kernel Configuration
5735a5736
> CONFIG_KAISER=y

~0030930

bill_mcgonigle (reporter)

The KAISER bug (seems to be a new problem, also causing a Xen boot failure) is being tracked here:

https://bugs.centos.org/view.php?id=14347
+Notes

-Issue History
Date Modified Username Field Change
2017-09-01 20:11 kstange New Issue
2017-09-01 20:12 kstange Note Added: 0029977
2017-09-01 22:15 kstange Note Added: 0029978
2017-09-01 23:09 kstange File Added: kernel-3.10.0-693.1.1-boot.log
2017-09-02 09:49 arrfab Status new => acknowledged
2017-09-02 09:49 arrfab Note Added: 0029979
2017-09-06 19:06 kstange File Added: xen-dont-copy-bogus-duplicate-entries-into-kernel-page-tables.patch
2017-09-06 19:06 kstange Note Added: 0030014
2017-09-07 20:21 kstange Note Added: 0030021
2017-09-14 14:39 avij Note Added: 0030077
2017-09-16 15:38 bill_mcgonigle Note Added: 0030095
2017-09-22 03:21 PryMar56 File Added: run_lorax
2017-09-22 03:21 PryMar56 Note Added: 0030169
2017-09-25 02:15 afletdinov Note Added: 0030194
2017-10-02 19:04 kstange Note Added: 0030281
2017-10-05 18:35 Gene Note Added: 0030314
2017-10-05 18:37 kstange Note Added: 0030315
2017-10-09 02:01 kjohnson Note Added: 0030333
2017-10-09 02:04 kjohnson Note Added: 0030334
2018-01-13 14:29 adrian27 File Added: aws-kernel-plus-3.10.0-693.11.6.el7.centos.plus.x86_64.log.txt
2018-01-13 14:29 adrian27 Note Added: 0030925
2018-01-13 14:36 adrian27 Note Added: 0030926
2018-01-13 14:47 toracat Note Added: 0030927
2018-01-13 14:48 adrian27 Note Added: 0030928
2018-01-13 17:01 bill_mcgonigle Note Added: 0030930
+Issue History