View Issue Details

IDProjectCategoryView StatusLast Update
0015172CentOS-7kernelpublic2018-08-22 02:55
Reporterpperminov 
PriorityurgentSeveritycrashReproducibilityalways
Status newResolutionopen 
Platformx86_64OSCentosOS VersionCore
Product Version7.5.1804 
Target VersionFixed in Version 
Summary0015172: Kernel panic at ElasticSearch reindexing shards in Docker
DescriptionFirst part: https://github.com/opencontainers/runc/issues/1857

So. I tried to this moment different configurations and different stress tests in docker containers. With(out) SElinux/swap. In almost all cases all works (almost) well saving cases with ElasticSearch.

Last one crash.

KERNEL: /usr/lib/debug/lib/modules/3.10.0-862.9.1.el7.x86_64/vmlinux
DUMPFILE: vmcore[PARTIAL DUMP]
CPUS: 40
DATE: Tue Aug 14 13:14:10 2018
UPTIME: 19:29:41
LOAD AVERAGE: 11.02, 63.47, 131.28
TASKS: 1528
NODENAME: tech-node3-dcdp
RELEASE: 3.10.0-862.9.1.el7.x86_64
VERSION: #1 SMP Mon Jul 16 16:29:36 UTC 2018
MACHINE: x86_64(2200 Mhz)
MEMORY: 126.6 GB
PANIC: "kernel BUG at mm/slub.c:3601!"
PID: 199160
COMMAND: "runc:[2:INIT]"
TASK: ffff9af3d8b30fd0[THREAD_INFO: ffff9af09ba0c000]
CPU: 5
STATE: TASK_RUNNING (PANIC)
crash> bt
PID: 199160 TASK: ffff9af3d8b30fd0 CPU: 5 COMMAND: "runc:[2:INIT]"
 #0 [ffff9af09ba0f9f0] machine_kexec at ffffffff9746178a
 #1 [ffff9af09ba0fa50] __crash_kexec at ffffffff97513bf2
 #2 [ffff9af09ba0fb20] crash_kexec at ffffffff97513ce0
 #3 [ffff9af09ba0fb38] oops_end at ffffffff97b18738
 #4 [ffff9af09ba0fb60] die at ffffffff9742e96b
 #5 [ffff9af09ba0fb90] do_trap at ffffffff97b17eb0
 #6 [ffff9af09ba0fbe0] do_invalid_op at ffffffff9742b284
 #7 [ffff9af09ba0fc90] invalid_op at ffffffff97b23aee
    [exception RIP: kfree+316]
    RIP: ffffffff975f73dc RSP: ffff9af09ba0fd40 RFLAGS: 00010246
    RAX: 006fffff0000083c RBX: ffff9af500000001 RCX: 0000000000000045
    RDX: 006fffff0000003c RSI: 0000000000000022 RDI: ffff9af500000001
    RBP: ffff9af09ba0fd58 R8: 0000000000000000 R9: 0000000000000000
    R10: ffff9af5bdf78200 R11: ffffdb61fd000000 R12: 0000000000000001
    R13: ffffffff976dbc5d R14: ffff9af500000012 R15: 0000000000000001
    ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
 #8 [ffff9af09ba0fd60] selinux_sb_show_options at ffffffff976dbc5d
 #9 [ffff9af09ba0fdc0] security_sb_show_options at ffffffff976d0c2c
#10 [ffff9af09ba0fdd0] show_sb_opts at ffffffff9766077c
#11 [ffff9af09ba0fdf8] show_mountinfo at ffffffff97660d3b
#12 [ffff9af09ba0fe58] m_show at ffffffff9763c219
#13 [ffff9af09ba0fe68] seq_read at ffffffff97641b80
#14 [ffff9af09ba0fed8] vfs_read at ffffffff9761b2ff
#15 [ffff9af09ba0ff08] sys_read at ffffffff9761c1cf
#16 [ffff9af09ba0ff50] system_call_fastpath at ffffffff97b20795
    RIP: 000055b622891710 RSP: 000000c4200a3c58 RFLAGS: 00000216
    RAX: 0000000000000000 RBX: 0000000000000000 RCX: ffffffffffffffff
    RDX: 0000000000001000 RSI: 000000c42011b000 RDI: 0000000000000004
    RBP: 000000c4200a3a68 R8: 0000000000000000 R9: 0000000000000000
    R10: 0000000000000000 R11: 0000000000000202 R12: ffffffffffffffff
    R13: 0000000000000002 R14: 0000000000000001 R15: 0000000000000002
    ORIG_RAX: 0000000000000000 CS: 0033 SS: 002b

Last log entries:

[69772.598979] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598981] cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598983] node 0: slabs: 31, objs: 1879, free: 0
[69772.598984] node 1: slabs: 41, objs: 2545, free: 0
[69772.599033] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.599035] cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.599037] node 0: slabs: 31, objs: 1879, free: 0
[69772.599039] node 1: slabs: 41, objs: 2545, free: 0
[69989.770831] ___slab_alloc: 7 callbacks suppressed
[69989.770834] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69989.770836] cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69989.770838] node 0: slabs: 31, objs: 1879, free: 0
[69989.770839] node 1: slabs: 44, objs: 2764, free: 0
[70016.648155] ------------[ cut here ]------------
[70016.648180] kernel BUG at mm/slub.c:3601!
[70016.648193] invalid opcode: 0000 [#1] SMP
[70016.648209] Modules linked in: ip_vs_rr xt_ipvs ip_vs xt_nat veth vxlan ip6_udp_tunnel udp_tunnel xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype xt_conntrack br_netfilter overlay(T) ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dcdbas iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel vfat kvm fat xfs irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper libcrc32c ablk_helper cryptd ipmi_ssif pcspkr
[70016.648487] sg mei_me mei lpc_ich i2c_i801 shpchp ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ahci ixgbe drm libahci tg3 megaraid_sas libata mdio dca ptp i2c_core pps_core
[70016.648635] CPU: 5 PID: 199160 Comm: runc:[2:INIT] Kdump: loaded Tainted: G ------------ T 3.10.0-862.9.1.el7.x86_64 #1
[70016.648669] Hardware name: Dell Inc. PowerEdge R440/0N28XX, BIOS 1.2.11 10/20/2017
[70016.648692] task: ffff9af3d8b30fd0 ti: ffff9af09ba0c000 task.ti: ffff9af09ba0c000
[70016.648714] RIP: 0010:[<ffffffff975f73dc>] [<ffffffff975f73dc>] kfree+0x13c/0x140
[70016.648742] RSP: 0018:ffff9af09ba0fd40 EFLAGS: 00010246
[70016.648758] RAX: 006fffff0000083c RBX: ffff9af500000001 RCX: 0000000000000045
[70016.648780] RDX: 006fffff0000003c RSI: 0000000000000022 RDI: ffff9af500000001
[70016.648801] RBP: ffff9af09ba0fd58 R08: 0000000000000000 R09: 0000000000000000
[70016.648822] R10: ffff9af5bdf78200 R11: ffffdb61fd000000 R12: 0000000000000001
[70016.648843] R13: ffffffff976dbc5d R14: ffff9af500000012 R15: 0000000000000001
[70016.649785] FS: 00007fd4679ca740(0000) GS:ffff9af5bbe80000(0000) knlGS:0000000000000000
[70016.650701] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[70016.651606] CR2: 000000c420120008 CR3: 0000001fe8a20000 CR4: 00000000007607e0
[70016.652518] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[70016.653413] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[70016.654292] PKRU: 55555554
[70016.655197] Call Trace:
[70016.656127] [<ffffffff976dbc5d>] selinux_sb_show_options+0x1dd/0x470
[70016.656961] [<ffffffff976d0c2c>] security_sb_show_options+0x1c/0x20
[70016.657786] [<ffffffff9766077c>] show_sb_opts+0x5c/0x70
[70016.658601] [<ffffffff97660d3b>] show_mountinfo+0x20b/0x2d0
[70016.659453] [<ffffffff9763c219>] m_show+0x19/0x20
[70016.660345] [<ffffffff97641b80>] seq_read+0x110/0x3f0
[70016.661213] [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[70016.661988] [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[70016.662753] [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[70016.663508] Code: 49 8b 03 31 f6 f6 c4 40 74 04 41 8b 73 68 4c 89 df e8 89 63 fa ff eb 84 4c 8b 58 30 48 8b 10 80 e6 80 4c 0f 44 d8 e9 28 ff ff ff <0f> 0b 66 90 0f 1f 44 00 00 55 89 f1 48 89 e5 41 57 41 56 41 55
[70016.665126] RIP [<ffffffff975f73dc>] kfree+0x13c/0x140
[70016.665905] RSP <ffff9af09ba0fd40>
Steps To Reproduce1. Use kernel 3.10.0-862.9.1.el7.x86_64
2. Use Docker higher than 17.06.1
3. You will need around 600GB of shards to reindex with elasticsearch
4. Elasticsearch docker image. We using custom builded from centos:centos7 and https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.1.1.tar.gz
5. Docker compose file in attachments
6. Run it!
Tagsdocker, java
abrt_hash
URL

Activities

pperminov

pperminov

2018-08-14 12:51

reporter  

elastic.yml (493 bytes)
log (56,198 bytes)
[64579.406601] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64579.406605]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64579.406607]   node 0: slabs: 30, objs: 1920, free: 0
[64579.406609]   node 1: slabs: 44, objs: 2816, free: 0
[64579.406643] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64579.406645]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64579.406646]   node 0: slabs: 30, objs: 1920, free: 0
[64579.406648]   node 1: slabs: 44, objs: 2816, free: 0
[64579.406679] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64579.406680]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64579.406682]   node 0: slabs: 30, objs: 1920, free: 0
[64579.406683]   node 1: slabs: 44, objs: 2816, free: 0
[64579.406714] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64579.406715]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64579.406717]   node 0: slabs: 30, objs: 1920, free: 0
[64579.406718]   node 1: slabs: 44, objs: 2816, free: 0
[64579.406750] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64579.406751]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64579.406752]   node 0: slabs: 30, objs: 1920, free: 0
[64579.406754]   node 1: slabs: 44, objs: 2816, free: 0
[64740.903372] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[64740.903377]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[64740.903380]   node 0: slabs: 20, objs: 1460, free: 0
[64740.903381]   node 1: slabs: 32, objs: 2080, free: 0
[64740.903421] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[64740.903423]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[64740.903425]   node 0: slabs: 20, objs: 1460, free: 0
[64740.903426]   node 1: slabs: 32, objs: 2080, free: 0
[64740.903459] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[64740.903460]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[64740.903462]   node 0: slabs: 20, objs: 1460, free: 0
[64740.903463]   node 1: slabs: 32, objs: 2080, free: 0
[64745.563088] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64745.563092]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64745.563094]   node 0: slabs: 31, objs: 1984, free: 0
[64745.563095]   node 1: slabs: 51, objs: 3264, free: 0
[64745.563139] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64745.563140]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64745.563141]   node 0: slabs: 31, objs: 1984, free: 0
[64745.563143]   node 1: slabs: 51, objs: 3264, free: 0
[64745.563181] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64745.563182]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64745.563183]   node 0: slabs: 31, objs: 1984, free: 0
[64745.563185]   node 1: slabs: 51, objs: 3264, free: 0
[64745.563219] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64745.563221]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64745.563222]   node 0: slabs: 31, objs: 1984, free: 0
[64745.563223]   node 1: slabs: 51, objs: 3264, free: 0
[64745.563259] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64745.563260]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64745.563261]   node 0: slabs: 31, objs: 1984, free: 0
[64745.563263]   node 1: slabs: 51, objs: 3264, free: 0
[64745.563300] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64745.563301]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64745.563302]   node 0: slabs: 31, objs: 1984, free: 0
[64745.563304]   node 1: slabs: 51, objs: 3264, free: 0
[64745.563342] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64745.563343]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64745.563345]   node 0: slabs: 31, objs: 1984, free: 0
[64745.563346]   node 1: slabs: 51, objs: 3264, free: 0
[64883.699742] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.699746]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.699749]   node 0: slabs: 32, objs: 2048, free: 0
[64883.699750]   node 1: slabs: 54, objs: 3456, free: 0
[64883.699793] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.699795]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.699796]   node 0: slabs: 32, objs: 2048, free: 0
[64883.699797]   node 1: slabs: 54, objs: 3456, free: 0
[64883.699832] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.699834]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.699835]   node 0: slabs: 32, objs: 2048, free: 0
[64883.699836]   node 1: slabs: 54, objs: 3456, free: 0
[64883.699873] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.699874]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.699875]   node 0: slabs: 32, objs: 2048, free: 0
[64883.699877]   node 1: slabs: 54, objs: 3456, free: 0
[64883.699911] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.699913]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.699914]   node 0: slabs: 32, objs: 2048, free: 0
[64883.699916]   node 1: slabs: 54, objs: 3456, free: 0
[64883.699952] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.699953]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.699955]   node 0: slabs: 32, objs: 2048, free: 0
[64883.699956]   node 1: slabs: 54, objs: 3456, free: 0
[64883.699991] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.699992]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.699994]   node 0: slabs: 32, objs: 2048, free: 0
[64883.699995]   node 1: slabs: 54, objs: 3456, free: 0
[64883.700029] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.700031]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.700032]   node 0: slabs: 32, objs: 2048, free: 0
[64883.700034]   node 1: slabs: 54, objs: 3456, free: 0
[64883.700098] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.700100]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.700101]   node 0: slabs: 32, objs: 2048, free: 0
[64883.700103]   node 1: slabs: 54, objs: 3456, free: 0
[64883.700137] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64883.700139]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64883.700140]   node 0: slabs: 32, objs: 2048, free: 0
[64883.700142]   node 1: slabs: 54, objs: 3456, free: 0
[64901.530694] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64901.530699]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64901.530701]   node 0: slabs: 32, objs: 2048, free: 0
[64901.530703]   node 1: slabs: 54, objs: 3456, free: 0
[64901.530744] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64901.530746]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64901.530748]   node 0: slabs: 32, objs: 2048, free: 0
[64901.530749]   node 1: slabs: 54, objs: 3456, free: 0
[64901.530784] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64901.530786]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64901.530788]   node 0: slabs: 32, objs: 2048, free: 0
[64901.530789]   node 1: slabs: 54, objs: 3456, free: 0
[64901.530825] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64901.530826]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64901.530828]   node 0: slabs: 32, objs: 2048, free: 0
[64901.530829]   node 1: slabs: 54, objs: 3456, free: 0
[64901.530863] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64901.530865]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64901.530866]   node 0: slabs: 32, objs: 2048, free: 0
[64901.530867]   node 1: slabs: 54, objs: 3456, free: 0
[64901.530902] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64901.530904]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64901.530905]   node 0: slabs: 32, objs: 2048, free: 0
[64901.530906]   node 1: slabs: 54, objs: 3456, free: 0
[64901.530923] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64901.530924]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64901.530926]   node 0: slabs: 32, objs: 2048, free: 0
[64901.530927]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741143] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741148]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741150]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741151]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741195] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741196]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741198]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741199]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741235] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741236]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741238]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741239]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741274] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741276]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741277]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741278]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741312] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741314]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741315]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741317]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741352] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741354]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741355]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741356]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741393] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741395]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741396]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741398]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741445] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741447]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741448]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741450]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741485] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741486]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741488]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741489]   node 1: slabs: 54, objs: 3456, free: 0
[64916.741524] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[64916.741525]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[64916.741527]   node 0: slabs: 32, objs: 2048, free: 0
[64916.741528]   node 1: slabs: 54, objs: 3456, free: 0
[65011.009787] ___slab_alloc: 14 callbacks suppressed
[65011.009791] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65011.009793]   cache: kmalloc-256(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 256, buffer size: 256, default order: 2, min order: 0
[65011.009796]   node 0: slabs: 103, objs: 6592, free: 41
[65011.009797]   node 1: slabs: 414, objs: 26256, free: 0
[65011.009844] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65011.009846]   cache: kmalloc-256(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 256, buffer size: 256, default order: 2, min order: 0
[65011.009848]   node 0: slabs: 103, objs: 6592, free: 41
[65011.009850]   node 1: slabs: 414, objs: 26256, free: 0
[65011.009887] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65011.009889]   cache: kmalloc-256(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 256, buffer size: 256, default order: 2, min order: 0
[65011.009891]   node 0: slabs: 103, objs: 6592, free: 41
[65011.009892]   node 1: slabs: 414, objs: 26256, free: 0
[65011.009950] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65011.009952]   cache: kmalloc-256(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 256, buffer size: 256, default order: 2, min order: 0
[65011.009954]   node 0: slabs: 103, objs: 6592, free: 41
[65011.009955]   node 1: slabs: 414, objs: 26256, free: 0
[65023.705019] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705023]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705025]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705026]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705065] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705067]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705068]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705070]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705110] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705111]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705113]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705114]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705150] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705151]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705153]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705154]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705186] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705187]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705189]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705190]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705223] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705225]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705226]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705227]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705260] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705262]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705263]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705264]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705296] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705298]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705299]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705300]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705331] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705332]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705334]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705335]   node 1: slabs: 58, objs: 3712, free: 0
[65023.705368] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65023.705369]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65023.705370]   node 0: slabs: 32, objs: 2048, free: 0
[65023.705372]   node 1: slabs: 58, objs: 3712, free: 0
[65128.700487] ___slab_alloc: 2 callbacks suppressed
[65128.700490] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65128.700492]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65128.700495]   node 0: slabs: 21, objs: 1469, free: 0
[65128.700496]   node 1: slabs: 34, objs: 2226, free: 0
[65288.140173] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65288.140177]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65288.140179]   node 0: slabs: 35, objs: 2208, free: 0
[65288.140181]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081020] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081025]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081027]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081029]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081074] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081076]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081077]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081078]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081117] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081118]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081120]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081121]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081170] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081172]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081174]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081175]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081214] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081215]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081217]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081218]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081254] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081256]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081258]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081259]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081295] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081296]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081298]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081299]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081336] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081337]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081339]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081340]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081378] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081379]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081381]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081382]   node 1: slabs: 59, objs: 3776, free: 0
[65316.081419] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65316.081420]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65316.081422]   node 0: slabs: 36, objs: 2272, free: 0
[65316.081423]   node 1: slabs: 59, objs: 3776, free: 0
[65473.502673] ___slab_alloc: 36 callbacks suppressed
[65473.502678] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65473.502680]   cache: kmalloc-64(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 64, buffer size: 64, default order: 0, min order: 0
[65473.502683]   node 0: slabs: 2369, objs: 151616, free: 0
[65473.502684]   node 1: slabs: 2436, objs: 155904, free: 0
[65551.085068] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65551.085072]   cache: kmalloc-64(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 64, buffer size: 64, default order: 0, min order: 0
[65551.085075]   node 0: slabs: 2370, objs: 151680, free: 2
[65551.085076]   node 1: slabs: 2491, objs: 159424, free: 0
[65627.630852] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65627.630856]   cache: kmalloc-64(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 64, buffer size: 64, default order: 0, min order: 0
[65627.630859]   node 0: slabs: 2370, objs: 151680, free: 1
[65627.630860]   node 1: slabs: 2558, objs: 163712, free: 0
[65706.065557] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65706.065562]   cache: kmalloc-256(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 256, buffer size: 256, default order: 2, min order: 0
[65706.065565]   node 0: slabs: 103, objs: 6592, free: 50
[65706.065567]   node 1: slabs: 483, objs: 30192, free: 0
[65706.065627] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65706.065630]   cache: kmalloc-256(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 256, buffer size: 256, default order: 2, min order: 0
[65706.065631]   node 0: slabs: 103, objs: 6592, free: 50
[65706.065633]   node 1: slabs: 483, objs: 30192, free: 0
[65706.065687] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65706.065689]   cache: kmalloc-256(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 256, buffer size: 256, default order: 2, min order: 0
[65706.065690]   node 0: slabs: 103, objs: 6592, free: 50
[65706.065692]   node 1: slabs: 483, objs: 30192, free: 0
[65706.065749] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65706.065751]   cache: kmalloc-256(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 256, buffer size: 256, default order: 2, min order: 0
[65706.065753]   node 0: slabs: 103, objs: 6592, free: 50
[65706.065754]   node 1: slabs: 483, objs: 30192, free: 0
[65793.467621] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65793.467625]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65793.467627]   node 0: slabs: 36, objs: 2272, free: 0
[65793.467629]   node 1: slabs: 72, objs: 4608, free: 0
[65793.787988] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65793.787998]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65793.788005]   node 0: slabs: 36, objs: 2272, free: 0
[65793.788010]   node 1: slabs: 72, objs: 4608, free: 0
[65793.788194] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65793.788201]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65793.788206]   node 0: slabs: 36, objs: 2272, free: 0
[65793.788211]   node 1: slabs: 72, objs: 4608, free: 0
[65793.987709] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65793.987720]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65793.987726]   node 0: slabs: 36, objs: 2272, free: 0
[65793.987732]   node 1: slabs: 72, objs: 4608, free: 0
[65793.987903] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65793.987918]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65793.987920]   node 0: slabs: 36, objs: 2272, free: 0
[65793.987921]   node 1: slabs: 72, objs: 4608, free: 0
[65900.001821] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.001825]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.001828]   node 0: slabs: 39, objs: 2400, free: 0
[65900.001829]   node 1: slabs: 72, objs: 4608, free: 0
[65900.001876] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.001878]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.001880]   node 0: slabs: 39, objs: 2400, free: 0
[65900.001881]   node 1: slabs: 72, objs: 4608, free: 0
[65900.001918] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.001920]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.001921]   node 0: slabs: 39, objs: 2400, free: 0
[65900.001923]   node 1: slabs: 72, objs: 4608, free: 0
[65900.001959] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.001961]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.001962]   node 0: slabs: 39, objs: 2400, free: 0
[65900.001963]   node 1: slabs: 72, objs: 4608, free: 0
[65900.002001] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.002003]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.002004]   node 0: slabs: 39, objs: 2400, free: 0
[65900.002006]   node 1: slabs: 72, objs: 4608, free: 0
[65900.002043] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.002044]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.002046]   node 0: slabs: 39, objs: 2400, free: 0
[65900.002047]   node 1: slabs: 72, objs: 4608, free: 0
[65900.002083] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.002085]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.002086]   node 0: slabs: 39, objs: 2400, free: 0
[65900.002088]   node 1: slabs: 72, objs: 4608, free: 0
[65900.002128] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.002130]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.002132]   node 0: slabs: 39, objs: 2400, free: 0
[65900.002133]   node 1: slabs: 72, objs: 4608, free: 0
[65900.002172] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.002174]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.002176]   node 0: slabs: 39, objs: 2400, free: 0
[65900.002177]   node 1: slabs: 72, objs: 4608, free: 0
[65900.002237] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[65900.002239]   cache: kmalloc-128(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 128, buffer size: 128, default order: 1, min order: 0
[65900.002240]   node 0: slabs: 39, objs: 2400, free: 0
[65900.002242]   node 1: slabs: 72, objs: 4608, free: 0
[65994.736707] ___slab_alloc: 8 callbacks suppressed
[65994.736711] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.736714]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.736716]   node 0: slabs: 26, objs: 1642, free: 0
[65994.736717]   node 1: slabs: 36, objs: 2244, free: 0
[65994.736759] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.736760]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.736762]   node 0: slabs: 26, objs: 1642, free: 0
[65994.736763]   node 1: slabs: 36, objs: 2244, free: 0
[65994.736796] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.736797]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.736799]   node 0: slabs: 26, objs: 1642, free: 0
[65994.736800]   node 1: slabs: 36, objs: 2244, free: 0
[65994.736836] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.736837]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.736839]   node 0: slabs: 26, objs: 1642, free: 0
[65994.736840]   node 1: slabs: 36, objs: 2244, free: 0
[65994.736876] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.736877]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.736879]   node 0: slabs: 26, objs: 1642, free: 0
[65994.736880]   node 1: slabs: 36, objs: 2244, free: 0
[65994.736915] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.736917]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.736918]   node 0: slabs: 26, objs: 1642, free: 0
[65994.736920]   node 1: slabs: 36, objs: 2244, free: 0
[65994.736954] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.736956]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.736957]   node 0: slabs: 26, objs: 1642, free: 0
[65994.736959]   node 1: slabs: 36, objs: 2244, free: 0
[65994.736991] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.736992]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.736994]   node 0: slabs: 26, objs: 1642, free: 0
[65994.736995]   node 1: slabs: 36, objs: 2244, free: 0
[65994.737028] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.737030]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.737031]   node 0: slabs: 26, objs: 1642, free: 0
[65994.737033]   node 1: slabs: 36, objs: 2244, free: 0
[65994.737067] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[65994.737068]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[65994.737070]   node 0: slabs: 26, objs: 1642, free: 0
[65994.737071]   node 1: slabs: 36, objs: 2244, free: 0
[66035.340660] ___slab_alloc: 12 callbacks suppressed
[66035.340664] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.340666]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.340668]   node 0: slabs: 28, objs: 1724, free: 0
[66035.340670]   node 1: slabs: 36, objs: 2244, free: 0
[66035.340726] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.340728]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.340729]   node 0: slabs: 28, objs: 1724, free: 0
[66035.340731]   node 1: slabs: 36, objs: 2244, free: 0
[66035.340780] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.340781]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.340783]   node 0: slabs: 28, objs: 1724, free: 0
[66035.340784]   node 1: slabs: 36, objs: 2244, free: 0
[66035.340832] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.340834]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.340835]   node 0: slabs: 28, objs: 1724, free: 0
[66035.340837]   node 1: slabs: 36, objs: 2244, free: 0
[66035.340884] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.340886]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.340888]   node 0: slabs: 28, objs: 1724, free: 0
[66035.340889]   node 1: slabs: 36, objs: 2244, free: 0
[66035.340941] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.340943]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.340944]   node 0: slabs: 28, objs: 1724, free: 0
[66035.340946]   node 1: slabs: 36, objs: 2244, free: 0
[66035.340993] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.340994]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.340996]   node 0: slabs: 28, objs: 1724, free: 0
[66035.340997]   node 1: slabs: 36, objs: 2244, free: 0
[66035.341046] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.341048]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.341050]   node 0: slabs: 28, objs: 1724, free: 0
[66035.341051]   node 1: slabs: 36, objs: 2244, free: 0
[66035.341100] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.341102]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.341103]   node 0: slabs: 28, objs: 1724, free: 0
[66035.341105]   node 1: slabs: 36, objs: 2244, free: 0
[66035.341156] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66035.341157]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66035.341159]   node 0: slabs: 28, objs: 1724, free: 0
[66035.341160]   node 1: slabs: 36, objs: 2244, free: 0
[66083.997777] ___slab_alloc: 24 callbacks suppressed
[66083.997781] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[66083.997783]   cache: kmalloc-64(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 64, buffer size: 64, default order: 0, min order: 0
[66083.997787]   node 0: slabs: 2378, objs: 152192, free: 9
[66083.997788]   node 1: slabs: 2677, objs: 171328, free: 0
[66375.205716] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[66375.205721]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[66375.205723]   node 0: slabs: 29, objs: 1797, free: 0
[66375.205724]   node 1: slabs: 39, objs: 2399, free: 0
[67167.407527] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
[67167.407531]   cache: kmalloc-64(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 64, buffer size: 64, default order: 0, min order: 0
[67167.407535]   node 0: slabs: 2382, objs: 152448, free: 23
[67167.407536]   node 1: slabs: 2776, objs: 177664, free: 0
[67403.474931] INFO: task systemd:1 blocked for more than 120 seconds.
[67403.475003] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475077] systemd         D ffff9ad729fb8000     0     1      0 0x00000000
[67403.475087] Call Trace:
[67403.475104]  [<ffffffff975cc9ff>] ? vma_set_page_prot+0x2f/0x50
[67403.475116]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475125]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475133]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475142]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475152]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475162]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475169]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475181]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[67403.475223] INFO: task khugepaged:232 blocked for more than 120 seconds.
[67403.475265] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475286] khugepaged      D ffff9ae5bf0eeeb0     0   232      2 0x00000000
[67403.475288] Call Trace:
[67403.475292]  [<ffffffff97b14029>] schedule+0x29/0x70
[67403.475295]  [<ffffffff97b1565d>] rwsem_down_read_failed+0x10d/0x1a0
[67403.475299]  [<ffffffff9775b238>] call_rwsem_down_read_failed+0x18/0x30
[67403.475301]  [<ffffffff97b13150>] down_read+0x20/0x40
[67403.475306]  [<ffffffff97603787>] khugepaged_scan_mm_slot+0x67/0xe30
[67403.475310]  [<ffffffff974a3750>] ? internal_add_timer+0x70/0x70
[67403.475313]  [<ffffffff9760468b>] khugepaged+0x13b/0x480
[67403.475316]  [<ffffffff974bc610>] ? wake_up_atomic_t+0x30/0x30
[67403.475318]  [<ffffffff97604550>] ? khugepaged_scan_mm_slot+0xe30/0xe30
[67403.475320]  [<ffffffff974bb621>] kthread+0xd1/0xe0
[67403.475322]  [<ffffffff974bb550>] ? insert_kthread_work+0x40/0x40
[67403.475325]  [<ffffffff97b205dd>] ret_from_fork_nospec_begin+0x7/0x21
[67403.475327]  [<ffffffff974bb550>] ? insert_kthread_work+0x40/0x40
[67403.475346] INFO: task systemd-journal:1296 blocked for more than 120 seconds.
[67403.475366] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475385] systemd-journal D ffff9af5be3f2f70     0  1296      1 0x00000080
[67403.475388] Call Trace:
[67403.475392]  [<ffffffff975cc9ff>] ? vma_set_page_prot+0x2f/0x50
[67403.475395]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475397]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475399]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475402]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475406]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475409]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475411]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475413]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[67403.475434] INFO: task dockerd:63461 blocked for more than 120 seconds.
[67403.475452] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475471] dockerd         D ffff9ae29a11dee0     0 63461      1 0x00000080
[67403.475473] Call Trace:
[67403.475477]  [<ffffffff9763dde4>] ? mntput+0x24/0x40
[67403.475480]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475481]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475484]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475486]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475488]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475490]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475492]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475495]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[67403.475497] INFO: task dockerd:63503 blocked for more than 120 seconds.
[67403.475516] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475536] dockerd         D ffff9ae5b8fb8fd0     0 63503      1 0x00000080
[67403.475538] Call Trace:
[67403.475540]  [<ffffffff9763dde4>] ? mntput+0x24/0x40
[67403.475543]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475545]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475547]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475549]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475551]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475553]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475555]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475557]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[67403.475561] INFO: task dockerd:63518 blocked for more than 120 seconds.
[67403.475579] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475598] dockerd         D ffff9ae5b77e4f10     0 63518      1 0x00000080
[67403.475601] Call Trace:
[67403.475603]  [<ffffffff9763dde4>] ? mntput+0x24/0x40
[67403.475605]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475607]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475609]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475611]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475613]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475616]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475617]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475620]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[67403.475622] INFO: task dockerd:63535 blocked for more than 120 seconds.
[67403.475641] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475661] dockerd         D ffff9af5b4fa2f70     0 63535      1 0x00000080
[67403.475663] Call Trace:
[67403.475665]  [<ffffffff9763dde4>] ? mntput+0x24/0x40
[67403.475667]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475670]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475672]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475674]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475676]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475678]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475686]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475690]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[67403.475701] INFO: task dockerd:63600 blocked for more than 120 seconds.
[67403.475719] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475740] dockerd         D ffff9af5b911bf40     0 63600      1 0x00000080
[67403.475742] Call Trace:
[67403.475744]  [<ffffffff9763dde4>] ? mntput+0x24/0x40
[67403.475746]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475749]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475751]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475753]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475755]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475757]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475759]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475761]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[67403.475784] INFO: task dockerd:63738 blocked for more than 120 seconds.
[67403.475802] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475822] dockerd         D ffff9ae5882fdee0     0 63738      1 0x00000080
[67403.475824] Call Trace:
[67403.475826]  [<ffffffff9763dde4>] ? mntput+0x24/0x40
[67403.475828]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475830]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475832]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475835]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475837]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475839]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475841]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475843]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[67403.475846] INFO: task dockerd:63747 blocked for more than 120 seconds.
[67403.475864] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[67403.475884] dockerd         D ffff9ae58827eeb0     0 63747      1 0x00000080
[67403.475886] Call Trace:
[67403.475888]  [<ffffffff9763dde4>] ? mntput+0x24/0x40
[67403.475890]  [<ffffffff97b14f49>] schedule_preempt_disabled+0x29/0x70
[67403.475892]  [<ffffffff97b12d07>] __mutex_lock_slowpath+0xc7/0x1d0
[67403.475894]  [<ffffffff97b120ef>] mutex_lock+0x1f/0x2f
[67403.475896]  [<ffffffff9751ec76>] proc_cgroup_show+0x66/0x1e0
[67403.475898]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[67403.475901]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[67403.475903]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[67403.475905]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[69772.598581] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598586]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598589]   node 0: slabs: 31, objs: 1879, free: 0
[69772.598591]   node 1: slabs: 41, objs: 2545, free: 0
[69772.598646] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598648]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598650]   node 0: slabs: 31, objs: 1879, free: 0
[69772.598652]   node 1: slabs: 41, objs: 2545, free: 0
[69772.598698] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598700]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598702]   node 0: slabs: 31, objs: 1879, free: 0
[69772.598703]   node 1: slabs: 41, objs: 2545, free: 0
[69772.598754] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598756]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598758]   node 0: slabs: 31, objs: 1879, free: 0
[69772.598759]   node 1: slabs: 41, objs: 2545, free: 0
[69772.598807] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598809]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598811]   node 0: slabs: 31, objs: 1879, free: 0
[69772.598812]   node 1: slabs: 41, objs: 2545, free: 0
[69772.598866] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598868]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598870]   node 0: slabs: 31, objs: 1879, free: 0
[69772.598872]   node 1: slabs: 41, objs: 2545, free: 0
[69772.598925] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598927]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598929]   node 0: slabs: 31, objs: 1879, free: 0
[69772.598930]   node 1: slabs: 41, objs: 2545, free: 0
[69772.598979] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.598981]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.598983]   node 0: slabs: 31, objs: 1879, free: 0
[69772.598984]   node 1: slabs: 41, objs: 2545, free: 0
[69772.599033] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69772.599035]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69772.599037]   node 0: slabs: 31, objs: 1879, free: 0
[69772.599039]   node 1: slabs: 41, objs: 2545, free: 0
[69989.770831] ___slab_alloc: 7 callbacks suppressed
[69989.770834] SLUB: Unable to allocate memory on node -1 (gfp=0x8020)
[69989.770836]   cache: ip6_dst_cache(2:5230bed7f0cfd2c0747205e06e39ce095caefee20b309e10da5395cea0e4093d), object size: 448, buffer size: 448, default order: 3, min order: 0
[69989.770838]   node 0: slabs: 31, objs: 1879, free: 0
[69989.770839]   node 1: slabs: 44, objs: 2764, free: 0
[70016.648155] ------------[ cut here ]------------
[70016.648180] kernel BUG at mm/slub.c:3601!
[70016.648193] invalid opcode: 0000 [#1] SMP 
[70016.648209] Modules linked in: ip_vs_rr xt_ipvs ip_vs xt_nat veth vxlan ip6_udp_tunnel udp_tunnel xt_mark ipt_MASQUERADE nf_nat_masquerade_ipv4 nf_conntrack_netlink xt_addrtype xt_conntrack br_netfilter overlay(T) ip_set nfnetlink ebtable_nat ebtable_broute bridge stp llc ip6table_nat nf_conntrack_ipv6 nf_defrag_ipv6 nf_nat_ipv6 ip6table_mangle ip6table_security ip6table_raw iptable_nat nf_conntrack_ipv4 nf_defrag_ipv4 nf_nat_ipv4 nf_nat nf_conntrack iptable_mangle iptable_security iptable_raw ebtable_filter ebtables ip6table_filter ip6_tables iptable_filter dcdbas iTCO_wdt iTCO_vendor_support skx_edac intel_powerclamp coretemp intel_rapl iosf_mbi kvm_intel vfat kvm fat xfs irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel lrw gf128mul glue_helper libcrc32c ablk_helper cryptd ipmi_ssif pcspkr
[70016.648487]  sg mei_me mei lpc_ich i2c_i801 shpchp ipmi_si ipmi_devintf ipmi_msghandler nfit libnvdimm acpi_power_meter acpi_pad ip_tables ext4 mbcache jbd2 sd_mod crc_t10dif crct10dif_generic mgag200 i2c_algo_bit drm_kms_helper syscopyarea sysfillrect sysimgblt fb_sys_fops ttm crct10dif_pclmul crct10dif_common crc32c_intel ahci ixgbe drm libahci tg3 megaraid_sas libata mdio dca ptp i2c_core pps_core
[70016.648635] CPU: 5 PID: 199160 Comm: runc:[2:INIT] Kdump: loaded Tainted: G               ------------ T 3.10.0-862.9.1.el7.x86_64 #1
[70016.648669] Hardware name: Dell Inc. PowerEdge R440/0N28XX, BIOS 1.2.11 10/20/2017
[70016.648692] task: ffff9af3d8b30fd0 ti: ffff9af09ba0c000 task.ti: ffff9af09ba0c000
[70016.648714] RIP: 0010:[<ffffffff975f73dc>]  [<ffffffff975f73dc>] kfree+0x13c/0x140
[70016.648742] RSP: 0018:ffff9af09ba0fd40  EFLAGS: 00010246
[70016.648758] RAX: 006fffff0000083c RBX: ffff9af500000001 RCX: 0000000000000045
[70016.648780] RDX: 006fffff0000003c RSI: 0000000000000022 RDI: ffff9af500000001
[70016.648801] RBP: ffff9af09ba0fd58 R08: 0000000000000000 R09: 0000000000000000
[70016.648822] R10: ffff9af5bdf78200 R11: ffffdb61fd000000 R12: 0000000000000001
[70016.648843] R13: ffffffff976dbc5d R14: ffff9af500000012 R15: 0000000000000001
[70016.649785] FS:  00007fd4679ca740(0000) GS:ffff9af5bbe80000(0000) knlGS:0000000000000000
[70016.650701] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[70016.651606] CR2: 000000c420120008 CR3: 0000001fe8a20000 CR4: 00000000007607e0
[70016.652518] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[70016.653413] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[70016.654292] PKRU: 55555554
[70016.655197] Call Trace:
[70016.656127]  [<ffffffff976dbc5d>] selinux_sb_show_options+0x1dd/0x470
[70016.656961]  [<ffffffff976d0c2c>] security_sb_show_options+0x1c/0x20
[70016.657786]  [<ffffffff9766077c>] show_sb_opts+0x5c/0x70
[70016.658601]  [<ffffffff97660d3b>] show_mountinfo+0x20b/0x2d0
[70016.659453]  [<ffffffff9763c219>] m_show+0x19/0x20
[70016.660345]  [<ffffffff97641b80>] seq_read+0x110/0x3f0
[70016.661213]  [<ffffffff9761b2ff>] vfs_read+0x9f/0x170
[70016.661988]  [<ffffffff9761c1cf>] SyS_read+0x7f/0xf0
[70016.662753]  [<ffffffff97b20795>] system_call_fastpath+0x1c/0x21
[70016.663508] Code: 49 8b 03 31 f6 f6 c4 40 74 04 41 8b 73 68 4c 89 df e8 89 63 fa ff eb 84 4c 8b 58 30 48 8b 10 80 e6 80 4c 0f 44 d8 e9 28 ff ff ff <0f> 0b 66 90 0f 1f 44 00 00 55 89 f1 48 89 e5 41 57 41 56 41 55 
[70016.665126] RIP  [<ffffffff975f73dc>] kfree+0x13c/0x140
[70016.665905]  RSP <ffff9af09ba0fd40>
log (56,198 bytes)
pperminov

pperminov

2018-08-14 12:56

reporter   ~0032481

Forgot to mention
Before the end it will be not able to start htop, top, ps aux. Also I can't get access to /proc/PID/maps of containered jvm.
pperminov

pperminov

2018-08-22 02:55

reporter   ~0032566

Confirmed on kernel 3.10.0-862.11.6

Issue History

Date Modified Username Field Change
2018-08-14 12:51 pperminov New Issue
2018-08-14 12:51 pperminov File Added: elastic.yml
2018-08-14 12:51 pperminov File Added: log
2018-08-14 12:51 pperminov Tag Attached: docker
2018-08-14 12:51 pperminov Tag Attached: java
2018-08-14 12:56 pperminov Note Added: 0032481
2018-08-22 02:55 pperminov Note Added: 0032566