View Issue Details

IDProjectCategoryView StatusLast Update
0014597CentOS-7kernel-pluspublic2018-04-20 00:45
Reporterkabe 
PrioritynormalSeverityblockReproducibilityalways
Status resolvedResolutionfixed 
Platformi686OS7OS Version7.4
Product Version7.4.1708 
Target VersionFixed in Version 
Summary0014597: kernel-3.10.0-693.21.1.el7 doesn't compile on i686
Descriptiongcc -Wp,-MD,arch/x86/kvm/.vmx.o.d -nostdinc -isystem /usr/lib/gcc/x86_64-redhat-linux/4.8.5/include -I./arch/x86/include -Iarch/x86/include/generated -Iinclude -I./arch/x86/include/uapi -Iarch/x86/include/generated/uapi -I./include/uapi -Iinclude/generated/uapi -include ./include/linux/kconfig.h -D__KERNEL__ -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Werror-implicit-function-declaration -Wno-format-security -fno-delete-null-pointer-checks -std=gnu89 -Werror -O2 -m32 -msoft-float -mregparm=3 -freg-struct-return -mno-mmx -mno-sse -fno-pic -mpreferred-stack-boundary=2 -march=i686 -maccumulate-outgoing-args -Wa,-mtune=generic32 -ffreestanding -Wframe-larger-than=2048 -DCONFIG_AS_CFI=1 -DCONFIG_AS_CFI_SIGNAL_FRAME=1 -DCONFIG_AS_CFI_SECTIONS=1 -DCONFIG_AS_AVX=1 -DCONFIG_AS_AVX2=1 -DCONFIG_AS_AVX512=1 -DCONFIG_AS_SHA1_NI=1 -DCONFIG_AS_SHA256_NI=1 -pipe -Wno-sign-compare -fno-asynchronous-unwind-tables -mno-sse -mno-mmx -mno-sse2 -mno-3dnow -mno-avx -mindirect-branch=thunk-extern -mindirect-branch-register -DRETPOLINE -Wframe-larger-than=2048 -fstack-protector-strong -Wno-unused-but-set-variable -fno-omit-frame-pointer -fno-optimize-sibling-calls -g -pg -fno-inline-functions-called-once -Wdeclaration-after-statement -Wno-pointer-sign -fno-strict-overflow -fconserve-stack -DCC_HAVE_ASM_GOTO -Ivirt/kvm -Iarch/x86/kvm -I. -DMODULE -D"KBUILD_STR(s)=#s" -D"KBUILD_BASENAME=KBUILD_STR(vmx)" -D"KBUILD_MODNAME=KBUILD_STR(kvm_intel)" -c -o arch/x86/kvm/.tmp_vmx.o arch/x86/kvm/vmx.c
2018-03-20 16:24:31 arch/x86/kvm/vmx.c: Assembler messages:
2018-03-20 16:24:31 arch/x86/kvm/vmx.c:8374: Error: value of 306 too large for field of 1 bytes at 0

The error message isn't very helpful here;
the problem lies in that ALTERNATIVE() macro which provides
different instructions depending on CPU flags
doesn't consider that alternative assembly code is longer
than the original code.
Additional InformationFrom assembly code, the error is:

================= vmx.s
#APP
# 8359 "arch/x86/kvm/19vmx.c" 1
        pushf
        orl $0x200, (%esp)
         pushl $96
        661:
        call *%eax << "original" code

662:
.pushsection .altinstructions,"a"
 .long 661b - .
 .long 6631f - .
 .word X86_FEATURE_RETPOLINE
 .byte 662b-661b
 .byte 6641f-6631f
.popsection
.pushsection .discard,"aw",@progbits
 .byte 0xff + (6641f-6631f) - (662b-661b) <<<<<error here
.popsection
.pushsection .altinstr_replacement, "ax"
6631:
               jmp 904f; << "alternate" code from here to .popsection
       .align 16
901: call 903f;
902: pause;
        lfence;
       jmp 902b;
       .align 16
903: addl $4, %esp;
       pushl %eax;
       ret;
       .align 16
904: call 901b;

6641:
        .popsection
=================

arch/x86/include/asm/alternative.h assumes that
#define DISCARD_ENTRY(number) /* rlen <= slen */ \
        " .byte 0xff + (" alt_rlen(number) ") - (" alt_slen ")\n"

that is, replacement assembly (rlen) is shorter than
source assembly length (slen), but this is not the case for
CONFIG_RETPOLINE code in arch/x86/include/asm/nospec-branch.h:

=================
/*
 * For i386 we use the original ret-equivalent retpoline, because
 * otherwise we'll run out of registers. We don't care about CET
 * here, anyway.
 */
# define CALL_NOSPEC ALTERNATIVE("call *%[thunk_target]\n", \ <<original
        " jmp 904f;\n" \ <<replacements follow
        " .align 16\n" \
        "901: call 903f;\n" \
        "902: pause;\n" \
        " lfence;\n" \
        " jmp 902b;\n" \
        " .align 16\n" \
        "903: addl $4, %%esp;\n" \
        " pushl %[thunk_target];\n" \
        " ret;\n" \
        " .align 16\n" \
        "904: call 901b;\n", \
        X86_FEATURE_RETPOLINE)
=================


The mainline kernel fixes this by considering the replacement code
could be longer than the original code:

=======3.10.0-693.21.1.el7 arch/x86/include/asm/alternative.h
struct alt_instr {
        s32 instr_offset; /* original instruction */
        s32 repl_offset; /* offset to replacement instruction */
        u16 cpuid; /* cpuid bit set for replacement */
        u8 instrlen; /* length of original instruction */
        u8 replacementlen; /* length of new instruction, <= instrlen */
};


=======linux-4.14.y mainline arch/x86/include/asm/alternative.h
struct alt_instr {
        s32 instr_offset; /* original instruction */
        s32 repl_offset; /* offset to replacement instruction */
        u16 cpuid; /* cpuid bit set for replacement */
        u8 instrlen; /* length of original instruction */
        u8 replacementlen; /* length of new instruction */
        u8 padlen; /* length of build-time padding */
} __packed;


so, fixing 3.10.0-693.21.1.el7 to be compilable with CONFIG_RETPOLINE
seems to involve major rework of ALTERNATIVE mechanism.

Easy way to let it compile on i686 is to disable CONFIG_RETPOLINE
but I think noone would agree with that.
Tagsi386
abrt_hash
URL

Activities

toracat

toracat

2018-03-20 13:51

manager   ~0031470

Thank you for the detailed analysis.

If no one can resolve this issue, one other option will be to use a newer [LTS] kernel (such as 4.4 which has a long life).
pgreco

pgreco

2018-03-20 13:59

developer   ~0031472

kabe, we are working to fix this atm, there are some patches to make it build, but it still doesn't work as we want to.
Disabling retpoline also generates a build failure, so that is not an option either (but as you said, we don't want that).
pgreco

pgreco

2018-03-20 19:08

developer   ~0031475

These are the patches we are applying on top of the ones that are currently in the src.rpm. It doesn't fully support retpoline, but it builds correctly. I hope to be able to backport everything correctly by 7.5

i386-retpoline-support.patch (851 bytes)
diff -aurp a/arch/x86/kernel/i386_ksyms_32.c b/arch/x86/kernel/i386_ksyms_32.c
--- a/arch/x86/kernel/i386_ksyms_32.c	2018-02-23 14:18:23.000000000 -0300
+++ b/arch/x86/kernel/i386_ksyms_32.c	2018-03-14 11:57:46.239599959 -0300
@@ -4,6 +4,7 @@
 #include <asm/pgtable.h>
 #include <asm/desc.h>
 #include <asm/ftrace.h>
+#include <asm/asm.h>
 
 #ifdef CONFIG_FUNCTION_TRACER
 /* mcount is defined in assembly */
@@ -37,3 +38,17 @@ EXPORT_SYMBOL(strstr);
 
 EXPORT_SYMBOL(csum_partial);
 EXPORT_SYMBOL(empty_zero_page);
+
+#ifdef CONFIG_RETPOLINE
+#define EXPORT_THUNK(reg)						\
+	extern void __x86_indirect_thunk_ ## reg(void);			\
+	EXPORT_SYMBOL(__x86_indirect_thunk_ ## reg)
+
+EXPORT_THUNK(eax);
+EXPORT_THUNK(ebx);
+EXPORT_THUNK(ecx);
+EXPORT_THUNK(edx);
+EXPORT_THUNK(esi);
+EXPORT_THUNK(edi);
+EXPORT_THUNK(ebp);
+#endif /* CONFIG_RETPOLINE */

patch-i686-fpu-issues.patch (2,071 bytes)
diff -Naurp a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
--- a/arch/x86/kernel/cpu/bugs.c	2018-02-23 14:18:23.000000000 -0300
+++ b/arch/x86/kernel/cpu/bugs.c	2018-03-13 11:04:58.923951923 -0300
@@ -24,6 +24,70 @@
 #include <asm/cacheflush.h>
 #include <asm/spec_ctrl.h>
 
+static int __init no_387(char *s)
+{
+	boot_cpu_data.hard_math = 0;
+	write_cr0(X86_CR0_TS | X86_CR0_EM | X86_CR0_MP | read_cr0());
+	return 1;
+}
+
+__setup("no387", no_387);
+
+static double __initdata x = 4195835.0;
+static double __initdata y = 3145727.0;
+
+/*
+ * This used to check for exceptions..
+ * However, it turns out that to support that,
+ * the XMM trap handlers basically had to
+ * be buggy. So let's have a correct XMM trap
+ * handler, and forget about printing out
+ * some status at boot.
+ *
+ * We should really only care about bugs here
+ * anyway. Not features.
+ */
+static void __init check_fpu(void)
+{
+	s32 fdiv_bug;
+
+	if (!boot_cpu_data.hard_math) {
+#ifndef CONFIG_MATH_EMULATION
+		pr_emerg("No coprocessor found and no math emulation present\n");
+		pr_emerg("Giving up\n");
+		for (;;) ;
+#endif
+		return;
+	}
+
+	kernel_fpu_begin();
+
+	/*
+	 * trap_init() enabled FXSR and company _before_ testing for FP
+	 * problems here.
+	 *
+	 * Test for the divl bug: http://en.wikipedia.org/wiki/Fdiv_bug
+	 */
+	__asm__("fninit\n\t"
+		"fldl %1\n\t"
+		"fdivl %2\n\t"
+		"fmull %2\n\t"
+		"fldl %1\n\t"
+		"fsubp %%st,%%st(1)\n\t"
+		"fistpl %0\n\t"
+		"fwait\n\t"
+		"fninit"
+		: "=m" (*&fdiv_bug)
+		: "m" (*&x), "m" (*&y));
+
+	kernel_fpu_end();
+
+	if (fdiv_bug) {
+		set_cpu_bug(&boot_cpu_data, X86_BUG_FDIV);
+		pr_warn("Hmm, FPU with FDIV bug\n");
+	}
+}
+
 static void __init spectre_v2_select_mitigation(void);
 
 void __init check_bugs(void)
@@ -206,8 +270,10 @@ ssize_t cpu_show_meltdown(struct device
 {
 	if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
 		return sprintf(buf, "Not affected\n");
+#ifdef CONFIG_KAISER
 	if (kaiser_enabled)
 		return sprintf(buf, "Mitigation: PTI\n");
+#endif
 	return sprintf(buf, "Vulnerable\n");
 }
 
kabe

kabe

2018-03-23 04:51

reporter   ~0031491

This is the /sys/devices/system/cpu/vulnerabilities/* running on real Pentium 4 with pgreco's patch;
I heard CPUs this old aren't vulnerable, but just FYI:

[kabe@capricorn vulnerabilities]$ cpuid |tail -1
   (synth) = Intel Pentium 4 (Prescott E0), 90nm
[kabe@capricorn vulnerabilities]$ uname -a
Linux capricorn.five.ten 3.10.0-693.21.1.el7.centos.plus.i686 #1 SMP Wed Mar 21 18:11:21 JST 2018 i686 i686 i386 GNU/Linux
[kabe@capricorn vulnerabilities]$ grep RETPOLINE /boot/config-3.10.0-693.21.1.el7.centos.plus.i686
CONFIG_RETPOLINE=y
[kabe@capricorn vulnerabilities]$ dmesg|grep FEATURE
[ 0.005005] FEATURE SPEC_CTRL Not Present
[ 0.005087] FEATURE IBPB_SUPPORT Not Present
[kabe@capricorn vulnerabilities]$ LANG=C sudo more /sys/devices/system/cpu/vulnerabilities/*
::::::::::::::
/sys/devices/system/cpu/vulnerabilities/meltdown
::::::::::::::
Vulnerable
::::::::::::::
/sys/devices/system/cpu/vulnerabilities/spectre_v1
::::::::::::::
Mitigation: Load fences
::::::::::::::
/sys/devices/system/cpu/vulnerabilities/spectre_v2
::::::::::::::
Vulnerable: Retpoline without IBPB
JohnnyHughes

JohnnyHughes

2018-03-23 10:42

administrator   ~0031495

Last edited: 2018-03-23 10:52

View 3 revisions

I have created a 4.14.x LTS kernel for i386 and x86_64 for other things .. I wonder if this one might be better at this point with Red Hat not backporting i386 code into their kernels.

https://buildlogs.centos.org/c7.1708.exp.x86_64/

The latest i386 version (right now) is here:

https://buildlogs.centos.org/c7.1708.exp.x86_64/kernel/20180319184005/4.14.28-201.el7.centos.i386/

I have tested THIS kernel on a KVM VM (I do not have any hardware i386 installs) using this script:

https://raw.githubusercontent.com/speed47/spectre-meltdown-checker/master/spectre-meltdown-checker.sh

Here is the output of the script:

Checking for vulnerabilities on current system
Kernel is Linux 4.14.28-201.el7.centos.i686 #1 SMP Mon Mar 19 20:19:46 UTC 2018 i686
CPU is AMD Opteron 63xx class CPU

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available: NO
    * CPU indicates IBRS capability: NO
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available: NO
    * CPU indicates IBPB capability: NO
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available: NO
    * CPU indicates STIBP capability: NO
  * Enhanced IBRS (IBRS_ALL)
    * CPU indicates ARCH_CAPABILITIES MSR availability: NO
    * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability: NO
  * CPU explicitly indicates not being vulnerable to Meltdown (RDCL_NO): NO
  * CPU microcode is known to cause stability problems: NO
* CPU vulnerability to the three speculative execution attacks variants
  * Vulnerable to Variant 1: YES
  * Vulnerable to Variant 2: YES
  * Vulnerable to Variant 3: NO

CVE-2017-5753 [bounds check bypass] aka 'Spectre Variant 1'
* Mitigated according to the /sys interface: YES (kernel confirms that the mitigation is active)
* Kernel has array_index_mask_nospec: YES (1 occurence(s) found of 32 bits array_index_mask_nospec())
* Kernel has the Red Hat/Ubuntu patch: NO
> STATUS: NOT VULNERABLE (Mitigation: __user pointer sanitization)

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface: YES (kernel confirms that the mitigation is active)
* Mitigation 1
  * Kernel is compiled with IBRS/IBPB support: NO
  * Currently enabled features
    * IBRS enabled for Kernel space: NO
    * IBRS enabled for User space: NO
    * IBPB enabled: NO
* Mitigation 2
  * Kernel compiled with retpoline option: YES
  * Kernel compiled with a retpoline-aware compiler: YES (kernel reports full retpoline compilation)
> STATUS: NOT VULNERABLE (Mitigation: Full generic retpoline)
CVE-2017-5754 [rogue data cache load] aka 'Meltdown' aka 'Variant 3'
* Mitigated according to the /sys interface: YES (kernel confirms that your CPU is unaffected)
* Kernel supports Page Table Isolation (PTI): NO
* PTI enabled and active: NO
* Running as a Xen PV DomU: NO
> STATUS: NOT VULNERABLE (your CPU vendor reported your CPU model as not vulnerable)

JohnnyHughes

JohnnyHughes

2018-03-23 10:58

administrator   ~0031496

Note: that script reports that the Plus kernel is vulnerable to:

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'

where the 4.14.28 kernel says it is NOT.
kabe

kabe

2018-03-26 04:07

reporter   ~0031504

This is gettine more off topic, but FYI:

The spectre-meltdown-checker.sh uses /sys/devices/system/cpu/vulnerabilities/*
for primary authoritative information source, so just checking these /sys entries
is generally enough unless you need more detailed information.

For CentOS kernel-3.10.0-693.21.1.el7.i686, vulnerabilities/* returns:
/meltdown:
    Always returns "Vulnerable", unless CONFIG_KAISER is defined,
    which is only available on x86_64.
/spectre_v1:
    Always shows "Mitigation: Load fences"
/spectre_v2:
    Will be "Vulnerable: Retpoline without IBPB" unless CPU has
    X86_FEATURE_IBPB_SUPPORT ("FEATURE IBPB_SUPPORT Present" in dmesg).

For mainline kernel-4.15.y:
/meltdown:
    Returns "Vulnerable", unless CPU has X86_FEATURE_PTI, which is enabled
    with CONFIG_PAGE_TABLE_ISOLATION, which is only available on x86_64.
/spectre_v1:
    Always shows "Mitigation: __user pointer sanitization"
/spectre_v2:
    Shows "Mitigation: Full generic retpoline" if compiled properly with
    CONFIG_RETPOLINE. I guess these wordarounds could be backported.

Summary:
    Currently, /meltdown will be always "Vulnerable" on i686.
kabe

kabe

2018-03-28 12:34

reporter   ~0031518

I've been trying to implement CALL_NOSPEC retpoline-aware, with no success.
Attached patch will compile, but causes hv_vmbus.ko (Hyper-V guest driver)
to panic.
drivers/hv/hv.c (eventually will be in hv_vmbus.ko) is the only
kernel module using CALL_NOSPEC inline assembly macro.

The difference between kernel-3.10.0-693.21.1 and kernel.org upstream code
is that kernel-3.10.0-693.21.1 has retpoline as runtime tunable
(/sys/kernel/debug/x86/retp_enabled), but upstream kernel.org has
this as static boot time option (spectre_v2=on, X86_FEATURE_RETPOLINE).

kernel-3.10.0-693.21.1 DOESN'T define X86_FEATURE_RETPOLINE, so
initial pgreco's patch didn't compile anyway.

kernel-3.10.0-693.21.1 implements retpoline tunable with
STATIC_JUMP, but given how STATIC_JUMP is implemented,
I concluded that this won't work in dynamically loaded kernel module.
Is this correct?
I couldn't understand why this causes kernel panic though.

To mitigate Spectre variant 2, we may have to resort to make
CALL_NOSPEC unconditionally retpolined, not tunable.
Implementing "spectre_v2=on" kernel option will deviate too much from
x86_64 counterpart, so this IMHO is not an option.

And all this cruft may not apply to future RHEL kernel code...
How does RHEL 7.5 code look like?

retp-plus2.patch (2,655 bytes)
WARNING
WARNING This patch doesn't work; it causes hv_vmbus.ko (Hyper-V guest driver)
WARNING to panic on early boot.
WARNING

diff -up ./arch/x86/include/asm/nospec-branch.h.retp ./arch/x86/include/asm/nospec-branch.h
--- ./arch/x86/include/asm/nospec-branch.h.retp	2018-03-25 18:26:17.000000000 +0900
+++ ./arch/x86/include/asm/nospec-branch.h	2018-03-27 23:21:44.000000000 +0900
@@ -160,6 +160,41 @@
 	"call __x86_indirect_thunk_%V[thunk_target]\n"
 #define THUNK_TARGET(addr) [thunk_target] "r" (addr)
 
+#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
+/*
+ * Redefinition of STATIC_JUMP in arch/x86/include/asm/jump_label.h
+ * as inline asm code.
+ */
+# define STATIC_JUMP(target, key) \
+	"911: .byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t"	\
+	".pushsection __jump_table, \"aw\" \n\t"		\
+	_ASM_ALIGN "\n\t"					\
+	_ASM_PTR "911b, " target ", " #key "\n\t" /*jump_entry*/\
+	".popsection\n"
+/*
+ * For i386 we use the original ret-equivalent retpoline, because
+ * otherwise we'll run out of registers. We don't care about CET
+ * here, anyway.
+ */
+# define CALL_NOSPEC STATIC_JUMP("912f", retp_enabled_key)	\
+	"	call *%[thunk_target]\n"			\
+	"	jmp    913f\n"					\
+	"912:	\n"						\
+	"       jmp    904f;\n"					\
+	"       .align 16\n"					\
+	"901:	call   903f;\n"					\
+	"902:	pause;\n"					\
+	"    	lfence;\n"					\
+	"       jmp    902b;\n"					\
+	"       .align 16\n"					\
+	"903:	addl   $4, %%esp;\n"				\
+	"       pushl  %[thunk_target];\n"			\
+	"       ret;\n"						\
+	"       .align 16\n"					\
+	"904:	call   901b;\n"					\
+	"913:	\n"					
+
+# define THUNK_TARGET(addr) [thunk_target] "rm" (addr)
 #else /* No retpoline for C / inline asm */
 # define CALL_NOSPEC "call *%[thunk_target]\n"
 # define THUNK_TARGET(addr) [thunk_target] "rm" (addr)
diff -up ./arch/x86/kernel/i386_ksyms_32.c.retp ./arch/x86/kernel/i386_ksyms_32.c
--- ./arch/x86/kernel/i386_ksyms_32.c.retp	2018-03-27 22:42:06.000000000 +0900
+++ ./arch/x86/kernel/i386_ksyms_32.c	2018-03-23 07:28:39.000000000 +0900
@@ -4,6 +4,7 @@
 #include <asm/pgtable.h>
 #include <asm/desc.h>
 #include <asm/ftrace.h>
+#include <asm/asm.h>
 
 #ifdef CONFIG_FUNCTION_TRACER
 /* mcount is defined in assembly */
@@ -37,3 +38,17 @@ EXPORT_SYMBOL(strstr);
 
 EXPORT_SYMBOL(csum_partial);
 EXPORT_SYMBOL(empty_zero_page);
+
+#ifdef CONFIG_RETPOLINE
+#define EXPORT_THUNK(reg)						\
+	extern void __x86_indirect_thunk_ ## reg(void);			\
+	EXPORT_SYMBOL(__x86_indirect_thunk_ ## reg)
+
+EXPORT_THUNK(eax);
+EXPORT_THUNK(ebx);
+EXPORT_THUNK(ecx);
+EXPORT_THUNK(edx);
+EXPORT_THUNK(esi);
+EXPORT_THUNK(edi);
+EXPORT_THUNK(ebp);
+#endif /* CONFIG_RETPOLINE */
retp-plus2.patch (2,655 bytes)
kabe

kabe

2018-03-30 08:50

reporter   ~0031534

I lied about CALL_NOSPEC usage;
besides drivers/hv/hv.c, arch/x86/kvm/vmx.c and arch/x86/kvm/emulate.c
uses inline asm using CALL_NOSPEC.
Also STATIC_JUMP does work inside kernel modules too.


The problem in drivers/hv/hv.c:hv_do_hypercall() was that
inline assembly allocates up all the generic registers,
so the indirect call for %[thunk_target] was compiled as

    call *(%esp)

which normal CALL_NOSPEC can't handle.
Retpoline plays around with %esp, so the retpolined call was going nowhere.
That will surely cause panic.


I think I've came up with a solution patch;
Confirmed that it boots on Hyper-V, and
retpoline could be switched on-off by
/sys/kernel/debug/x86/retp_enabled, in hv_vmbus.ko:hv_do_hypercall().

i386-retpoline-support-v5.patch (4,108 bytes)
diff -up ./arch/x86/include/asm/nospec-branch.h.retp ./arch/x86/include/asm/nospec-branch.h
--- ./arch/x86/include/asm/nospec-branch.h.retp	2018-02-24 02:18:23.000000000 +0900
+++ ./arch/x86/include/asm/nospec-branch.h	2018-03-30 17:27:19.019359336 +0900
@@ -160,6 +160,67 @@
 	"call __x86_indirect_thunk_%V[thunk_target]\n"
 #define THUNK_TARGET(addr) [thunk_target] "r" (addr)
 
+#elif defined(CONFIG_X86_32) && defined(CONFIG_RETPOLINE)
+/*
+ * Redefinition of STATIC_JUMP in arch/x86/include/asm/jump_label.h
+ * as inline asm code.
+ */
+# define STATIC_JUMP(target, key) \
+	"911: .byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t"	\
+	".pushsection __jump_table, \"aw\" \n\t"		\
+	_ASM_ALIGN "\n\t"					\
+	_ASM_PTR "911b, " target ", " #key "\n\t" /*jump_entry*/\
+	".popsection\n"
+/*
+ * For i386 we use the original ret-equivalent retpoline, because
+ * otherwise we'll run out of registers. We don't care about CET
+ * here, anyway.
+ */
+# define CALL_NOSPEC STATIC_JUMP("912f", retp_enabled_key)	\
+	"	call *%[thunk_target]\n"			\
+	"	jmp    913f\n"					\
+	"912:	\n"						\
+	"       jmp    904f;\n"					\
+	"       .align 16\n"					\
+	"901:	call   903f;\n"					\
+	"902:	pause;\n"					\
+	"    	lfence;\n"					\
+	"       jmp    902b;\n"					\
+	"       .align 16\n"					\
+	"903:	addl   $4, %%esp;\n"				\
+	"       pushl  %[thunk_target];\n"			\
+	"       ret;\n"						\
+	"       .align 16\n"					\
+	"904:	call   901b;\n"					\
+	"913:	\n"					
+
+# define THUNK_TARGET(addr) [thunk_target] "rm" (addr)
+/*
+ * CALL_NOSPEC doesn't work if [thunk_target] was *(%esp),
+ * when all registers except %ebp is used up, as in
+ * drivers/hs/hv.c:hv_do_hypercall().
+ */
+# define CALL_NOSPEC_FULL STATIC_JUMP("912f", retp_enabled_key)	\
+	"	call *%[thunk_target]\n"			\
+	"	jmp    913f\n"					\
+	"912:	\n"						\
+	"	pushl  %[thunk_target];\n"			\
+	"       jmp    904f;\n"					\
+	"       .align 16\n"					\
+	"901:	call   903f;\n"					\
+	"902:	pause;\n"					\
+	"    	lfence;\n"					\
+	"       jmp    902b;\n"					\
+	"       .align 16\n"					\
+	"903:	addl   $4, %%esp;\n"				\
+	"       pushl  4(%%esp);\n"				\
+	"       ret;\n"						\
+	"       .align 16\n"					\
+	"904:	call   901b;\n"					\
+	"	addl   $4, %%esp;\n"				\
+	"913:	\n"					
+
+# define THUNK_TARGET_FULL(addr) [thunk_target] "rm" (addr)
 #else /* No retpoline for C / inline asm */
 # define CALL_NOSPEC "call *%[thunk_target]\n"
 # define THUNK_TARGET(addr) [thunk_target] "rm" (addr)
diff -up ./arch/x86/kernel/i386_ksyms_32.c.retp ./arch/x86/kernel/i386_ksyms_32.c
--- ./arch/x86/kernel/i386_ksyms_32.c.retp	2018-02-24 02:18:23.000000000 +0900
+++ ./arch/x86/kernel/i386_ksyms_32.c	2018-03-30 15:44:33.683189525 +0900
@@ -4,6 +4,7 @@
 #include <asm/pgtable.h>
 #include <asm/desc.h>
 #include <asm/ftrace.h>
+#include <asm/asm.h>
 
 #ifdef CONFIG_FUNCTION_TRACER
 /* mcount is defined in assembly */
@@ -37,3 +38,17 @@ EXPORT_SYMBOL(strstr);
 
 EXPORT_SYMBOL(csum_partial);
 EXPORT_SYMBOL(empty_zero_page);
+
+#ifdef CONFIG_RETPOLINE
+#define EXPORT_THUNK(reg)						\
+	extern void __x86_indirect_thunk_ ## reg(void);			\
+	EXPORT_SYMBOL(__x86_indirect_thunk_ ## reg)
+
+EXPORT_THUNK(eax);
+EXPORT_THUNK(ebx);
+EXPORT_THUNK(ecx);
+EXPORT_THUNK(edx);
+EXPORT_THUNK(esi);
+EXPORT_THUNK(edi);
+EXPORT_THUNK(ebp);
+#endif /* CONFIG_RETPOLINE */
diff -up ./drivers/hv/hv.c.retp ./drivers/hv/hv.c
--- ./drivers/hv/hv.c.retp	2018-02-24 02:18:23.000000000 +0900
+++ ./drivers/hv/hv.c	2018-03-30 17:27:47.590429188 +0900
@@ -124,11 +124,11 @@ u64 hv_do_hypercall(u64 control, void *i
 	if (!hypercall_page)
 		return (u64)ULLONG_MAX;
 
-	__asm__ __volatile__ (CALL_NOSPEC : "=d"(hv_status_hi),
+	__asm__ __volatile__ (CALL_NOSPEC_FULL : "=d"(hv_status_hi),
 			      "=a"(hv_status_lo) : "d" (control_hi),
 			      "a" (control_lo), "b" (input_address_hi),
 			      "c" (input_address_lo), "D"(output_address_hi),
-			      "S"(output_address_lo), THUNK_TARGET(hypercall_page));
+			      "S"(output_address_lo), THUNK_TARGET_FULL(hypercall_page));
 
 	return hv_status_lo | ((u64)hv_status_hi << 32);
 #endif /* !x86_64 */
toracat

toracat

2018-04-01 06:02

manager   ~0031537

Looking good.

@JohnnyHughes, @pgreco, I think we should go ahead and release the i686 kernel if there is no objection.
JohnnyHughes

JohnnyHughes

2018-04-03 09:45

administrator   ~0031543

I have built this and am releasing it now.
toracat

toracat

2018-04-03 15:46

manager   ~0031548

Confirmed the packages are on the mirrors now. Closing the ticket.

Thank you, @kabe. Thank you, @pgreco. See you at the 7.5 release time. :-)
toracat

toracat

2018-04-20 00:45

manager   ~0031642

7.5 is out. The plus kernel fails to build. A new ticket has been opened here:

https://bugs.centos.org/view.php?id=14703

Issue History

Date Modified Username Field Change
2018-03-20 10:34 kabe New Issue
2018-03-20 10:34 kabe Tag Attached: i386
2018-03-20 13:27 toracat Status new => assigned
2018-03-20 13:51 toracat Note Added: 0031470
2018-03-20 13:59 pgreco Note Added: 0031472
2018-03-20 19:08 pgreco File Added: i386-retpoline-support.patch
2018-03-20 19:08 pgreco File Added: patch-i686-fpu-issues.patch
2018-03-20 19:08 pgreco Note Added: 0031475
2018-03-23 04:51 kabe Note Added: 0031491
2018-03-23 10:42 JohnnyHughes Note Added: 0031495
2018-03-23 10:43 JohnnyHughes Note Edited: 0031495 View Revisions
2018-03-23 10:52 JohnnyHughes Note Edited: 0031495 View Revisions
2018-03-23 10:58 JohnnyHughes Note Added: 0031496
2018-03-26 04:07 kabe Note Added: 0031504
2018-03-28 12:34 kabe File Added: retp-plus2.patch
2018-03-28 12:34 kabe Note Added: 0031518
2018-03-30 08:50 kabe File Added: i386-retpoline-support-v5.patch
2018-03-30 08:50 kabe Note Added: 0031534
2018-04-01 06:02 toracat Note Added: 0031537
2018-04-03 09:45 JohnnyHughes Note Added: 0031543
2018-04-03 15:46 toracat Status assigned => resolved
2018-04-03 15:46 toracat Resolution open => fixed
2018-04-03 15:46 toracat Note Added: 0031548
2018-04-20 00:45 toracat Note Added: 0031642