0002076: Kernel panic on module ppp_mppe

(0005580)
sebycarta (reporter)
2007-07-16 06:24

I have the same problem also with kernel 2.6.18-8.1.6.el5.centos.plus x86_64.
Using pptpd, after few minutes of use the machine freezes. I tried to install mppe module throws dkms but it did not compile on this kernel.
There is a way to resolve (temporarily) this, waiting for a kernel fix?
Thank you.

(0005604)
arrfab (updater)
2007-07-18 12:31

sebycarta : you don't need to build mppe module since it's already included in the kernel starting from 2.6.15 (see http://pptpclient.sourceforge.net/ [^] for informations) and centos 5 has kernel 2.6.18 ....
I've installed the pptpclient (that i rebuilt for centos 5) and i've used it several times without any problems (connecting to linux pptpd-poptop, watchguard Firewall pptp and also Microsoft PPTPD) ... but i admit i've never tried to setup the server part (pptpd) on centos 5 though ... and that's something i want to avoid if possible ...

(0005607)
sebycarta (reporter)
2007-07-18 13:41

arrfab:
i know. i tried to build other mppe module because the module of the kernel freeze the machine.

i tried also to install centos 5 x86_64 inside vmware virtual machine, and i obtain the same freeze when i connect from other clients.
Strange thing that the version x86 of centos works very well and not freeze.
Anyway, after talking on #centos irc channel , i filed this bug to redhat.

pptpd can be very important, we use it to provide to our customers ( windows clients ) the access at our network.

(0005609)
arrfab (updater)
2007-07-18 14:30

ok, so does that kernel panic occurs on both i386 and x86_64 ? i'll try to install pptpd on a centos 5 in a lab to confirm that bug .... btw, it never froze my machine (centos 5 i386) when i used it (mppe) on a client side ... and if it freezes only with pptpd (the server part), that will probably be a good thing to see with poptop people what they are thinking of that bug ...
BTW i don't think you'll have an official answer of RHEL (and which bugzilla number is it ?) because they don't provide neither pptp nor pptpd in EL5 ...

(0005610)
sebycarta (reporter)
2007-07-18 14:38

arrfab:
No, kernel panic only on x86_64 machine. The i386 kernel does not freeze, it run very well.

Bug on redhat is this: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=248716 [^]

(0005860)
kingherowu (reporter)
2007-08-15 10:07

kernel-2.6.18-8.1.8.el5 x86_64 have the same problem.

kernel-xen-2.6.18-8.1.8.el5 x86_64 worked good.

(0005969)
Martynas (reporter)
2007-09-10 12:44

Hello,

I have 2 machines with Centos 5, and I can confirm, that I have no problem with i386 and ppp+pptpd, and it freezes on x86_64 platform!

So, could Centos team fix that problem ASAP?!

Martynas

(0005972)
arrfab (updater)
2007-09-10 21:06

Don't forget that CentOS uses only kernels released upstream ... and support for pptp/pptpd is not even included in the base product from which CentOS is based on ...
See the bugzilla entry on upstream's website and you'll see that they didn't even react on this (not a rh problem anyway) ...
My idea is that you can contact people from the pptpclient.sf.net project on their mailing-list and retrieve useful informations from there ...
BTW, have you tried with the centosplus x86_64 kernel to see if this is reproducible with this one ?

(0005973)
Martynas (reporter)
2007-09-11 06:09

Hello,

What is Centosplus? What is difference between Centos and Centosplus? And where coudl I find packages of centosplus?
HAve anybody tried to just load ppp_mppe module without pptpd? like modprobe ppp_mppe? If so, x86_64 machine freezes or not?

regards,
Martynas

(0005975)
toracat (developer)
2007-09-11 16:02

Info on centosplus is here:

http://wiki.centos.org/Repositories/CentOSPlus [^]

I just ran 'modprobe ppp_mppe' on a system running 2.6.18-8.1.8.el5.centos.plus x86_64. What am I supposed to do to trigger the freeze reported here?

Akemi

(0005977)
sdv (reporter)
2007-09-11 17:35

Hi guys!

4 months is a too long term for serious bug, isn't it? :(

Akemi, thanks to your team for job, BUT.... Could you be so kind to try to read bug description? "Using pptpd, after few minutes of use the machine freezes." Did you try to use VPN connection or just "modprobe..."?

I'm afraid, that we should remove centos from our security servers and migrate to debian. :(

(0005978)
lanconnected (reporter)
2007-09-11 17:37

To arrfab:
pptp support might no be included in CentOS, but ppp_mppe module is. Kernel freeze is caused by the module, not by userspace programs using it (pppd, actually). pptpclient.sf.net project has nothing to do with this issue, vanilla kernel works just fine, problem is caused by some patches added to CentOS kernel (or RedHat kernel). Plesase, read the first post carefully.
To toracat:
Loading ppp_mppe does not trigger kernel crush, but using it does. Create an encrypted ppp connection if you want to reproduce kernel panic. Please, read the first post carefully.
To sebycarta: <There is a way to resolve (temporarily) this, waiting for a kernel fix?> ---- There is a good working fix: use vanilla kernel from kernel.org without RedHat patchset.

(0005979)
toracat (developer)
2007-09-11 17:47

My response is to this question: "HAve anybody tried to just load ppp_mppe module without pptpd? like modprobe ppp_mppe? " I just did what it asked for.

(0005981)
toracat (developer)
2007-09-11 18:04
edited on: 2007-09-12 01:55

"Akemi, thanks to your team for job"

I am not a centos dev. Just a user like yourself. So I am free to say good luck with debian.

I also had a very nasty kernel crash problem which was introduced in the cifs module of kernel 2.6.18. The system was totally useless (FC5 and FC6). Luckily, I was able to get a fix with a lot of help from cifs programmers. Then when CentOS 5 came out (which uses 2.6.18), the CentOS team kindly offered the fixed version of the cifs module for users who were affected by the bug. RedHat does not even offer this kind of help.

In principle, CentOS does not repair the kernel. It reproduces everything from upstream including bugs. In the above example, I also worked with RedHat to get the bug fix applied to their kernel because this is the only way to have that fix in CentOS kernels. As a result, the cifs bug is now supposed to be included in RHEL 5.1 as well as 4.6. I think you should do the same. Push upstream and do it very hard. That's the lesson I learned.

Akemi

(0005986)
sebycarta (reporter)
2007-09-12 06:19

The bug opened on redhat ( https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=248716 [^] ) has been marked as "assigned", but with "low priority" ( i cannot imagine high priority redhat bugs!).
I suggest all "bug friend" to post on redhat bugzilla, as i realize, centos cannot fix the kernel, the solution is in upstream.

(0005990)
sdv (reporter)
2007-09-12 09:33

> I am not a centos dev. Just a user like yourself. So I am free to say good luck with debian.

Ok, thank you.

Another question: does anybody compile mppe module with dkms in CentOS5?

(0005992)
toracat (developer)
2007-09-12 12:13

Again, my answer is as a user...

dkms is not a preferred method by the centos developers. But if you are interested in building kernel modules, please take a look at:

http://wiki.centos.org/HowTos/BuildingKernelModules [^]

Akemi

(0005993)
toracat (developer)
2007-09-12 14:38

Speaking of modules, there was one more note I forgot to mention. When I saw the description that says the vanilla kernel 2.6.18.4 works fine, I downloaded the source for that version and compiled the ppp_mppe module. But it did not load in CentOS 5. Not surprising given the fact the RHEL kernel is heavily customized. It would require some (or a lot of) hacking to get it to work.

Akemi

(0005995)
sdv (reporter)
2007-09-13 09:34

Vanilla 2.6.18.4 compiled using 2.6.18.1.8 CentOS5 kernel .config works fine.

(0005996)
toracat (developer)
2007-09-13 11:36

Yes, my attempt to use the module from the mainline kernel was based on that description. I did use the .config from CentOS-5 2.6.18-8.1.8.el5 and compilation went through. But the module built that way did not load in the CentOS kernel. If this worked, it would have been much easier to provide a fix -- just load the patched module to the running kernel. A person with good knowledge on the RH kernel might be able to get it to work.

Akemi

(0006002)
sdv (reporter)
2007-09-13 15:42
edited on: 2007-09-13 15:42

> A person with good knowledge on the RH kernel might be able to get it to work.

May be. I'm mot novice in Linux and already have set of my own patches to some old kernels and loop-aes, but only one server was affected and now it works, so, i don't want to spend my time more. Even if somebody will have a patch, i will wait for official solution. May be, vanilla kernel is a bad choice, but affected server isn't production.

Thanks for help.

(0006033)
sebycarta (reporter)
2007-09-19 20:06

Yeeeee! there is a patch from redhat! Look at https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=248716 [^]
the kernel developer issued a patch.
I have to try that.
Bye.

(0006049)
sebycarta (reporter)
2007-09-24 10:02

Only for information:
Redhat has been uploaded a patched kernel, you can download it on the redhat bugzilla ( see notes above for the link).
Bug resolved, go to work! ;-)

(0006665)
jhughes@hughesjr.com (administrator)
2008-01-08 10:19

just to be clear here ... CentOS is a rebuild of upstream items and we provide a product (warts and all) that behaves like the upstream product. It is the WHOLE point and main purpose of CentOS.

While we want to help identify and help fix upstream problems (see bugs 0002189 and 0001776 for examples), our main CentOS products will not be updated until a stable release is done upstream.

We are not about making a fork, but a predictable OS that performs exactly the same as the products we clone.

Notes
(0005580) sebycarta (reporter) 2007-07-16 06:24	I have the same problem also with kernel 2.6.18-8.1.6.el5.centos.plus x86_64. Using pptpd, after few minutes of use the machine freezes. I tried to install mppe module throws dkms but it did not compile on this kernel. There is a way to resolve (temporarily) this, waiting for a kernel fix? Thank you.

(0005604) arrfab (updater) 2007-07-18 12:31	sebycarta : you don't need to build mppe module since it's already included in the kernel starting from 2.6.15 (see http://pptpclient.sourceforge.net/ [^] for informations) and centos 5 has kernel 2.6.18 .... I've installed the pptpclient (that i rebuilt for centos 5) and i've used it several times without any problems (connecting to linux pptpd-poptop, watchguard Firewall pptp and also Microsoft PPTPD) ... but i admit i've never tried to setup the server part (pptpd) on centos 5 though ... and that's something i want to avoid if possible ...

(0005607) sebycarta (reporter) 2007-07-18 13:41	arrfab: i know. i tried to build other mppe module because the module of the kernel freeze the machine. i tried also to install centos 5 x86_64 inside vmware virtual machine, and i obtain the same freeze when i connect from other clients. Strange thing that the version x86 of centos works very well and not freeze. Anyway, after talking on #centos irc channel , i filed this bug to redhat. pptpd can be very important, we use it to provide to our customers ( windows clients ) the access at our network.

(0005609) arrfab (updater) 2007-07-18 14:30	ok, so does that kernel panic occurs on both i386 and x86_64 ? i'll try to install pptpd on a centos 5 in a lab to confirm that bug .... btw, it never froze my machine (centos 5 i386) when i used it (mppe) on a client side ... and if it freezes only with pptpd (the server part), that will probably be a good thing to see with poptop people what they are thinking of that bug ... BTW i don't think you'll have an official answer of RHEL (and which bugzilla number is it ?) because they don't provide neither pptp nor pptpd in EL5 ...

(0005610) sebycarta (reporter) 2007-07-18 14:38	arrfab: No, kernel panic only on x86_64 machine. The i386 kernel does not freeze, it run very well. Bug on redhat is this: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=248716 [^]

(0005860) kingherowu (reporter) 2007-08-15 10:07	kernel-2.6.18-8.1.8.el5 x86_64 have the same problem. kernel-xen-2.6.18-8.1.8.el5 x86_64 worked good.

(0005969) Martynas (reporter) 2007-09-10 12:44	Hello, I have 2 machines with Centos 5, and I can confirm, that I have no problem with i386 and ppp+pptpd, and it freezes on x86_64 platform! So, could Centos team fix that problem ASAP?! Martynas

(0005972) arrfab (updater) 2007-09-10 21:06	Don't forget that CentOS uses only kernels released upstream ... and support for pptp/pptpd is not even included in the base product from which CentOS is based on ... See the bugzilla entry on upstream's website and you'll see that they didn't even react on this (not a rh problem anyway) ... My idea is that you can contact people from the pptpclient.sf.net project on their mailing-list and retrieve useful informations from there ... BTW, have you tried with the centosplus x86_64 kernel to see if this is reproducible with this one ?

(0005973) Martynas (reporter) 2007-09-11 06:09	Hello, What is Centosplus? What is difference between Centos and Centosplus? And where coudl I find packages of centosplus? HAve anybody tried to just load ppp_mppe module without pptpd? like modprobe ppp_mppe? If so, x86_64 machine freezes or not? regards, Martynas

(0005975) toracat (developer) 2007-09-11 16:02	Info on centosplus is here: http://wiki.centos.org/Repositories/CentOSPlus [^] I just ran 'modprobe ppp_mppe' on a system running 2.6.18-8.1.8.el5.centos.plus x86_64. What am I supposed to do to trigger the freeze reported here? Akemi

(0005977) sdv (reporter) 2007-09-11 17:35	Hi guys! 4 months is a too long term for serious bug, isn't it? :( Akemi, thanks to your team for job, BUT.... Could you be so kind to try to read bug description? "Using pptpd, after few minutes of use the machine freezes." Did you try to use VPN connection or just "modprobe..."? I'm afraid, that we should remove centos from our security servers and migrate to debian. :(

(0005978) lanconnected (reporter) 2007-09-11 17:37	To arrfab: pptp support might no be included in CentOS, but ppp_mppe module is. Kernel freeze is caused by the module, not by userspace programs using it (pppd, actually). pptpclient.sf.net project has nothing to do with this issue, vanilla kernel works just fine, problem is caused by some patches added to CentOS kernel (or RedHat kernel). Plesase, read the first post carefully. To toracat: Loading ppp_mppe does not trigger kernel crush, but using it does. Create an encrypted ppp connection if you want to reproduce kernel panic. Please, read the first post carefully. To sebycarta: <There is a way to resolve (temporarily) this, waiting for a kernel fix?> ---- There is a good working fix: use vanilla kernel from kernel.org without RedHat patchset.

(0005979) toracat (developer) 2007-09-11 17:47	My response is to this question: "HAve anybody tried to just load ppp_mppe module without pptpd? like modprobe ppp_mppe? " I just did what it asked for.

(0005981) toracat (developer) 2007-09-11 18:04 edited on: 2007-09-12 01:55	"Akemi, thanks to your team for job" I am not a centos dev. Just a user like yourself. So I am free to say good luck with debian. I also had a very nasty kernel crash problem which was introduced in the cifs module of kernel 2.6.18. The system was totally useless (FC5 and FC6). Luckily, I was able to get a fix with a lot of help from cifs programmers. Then when CentOS 5 came out (which uses 2.6.18), the CentOS team kindly offered the fixed version of the cifs module for users who were affected by the bug. RedHat does not even offer this kind of help. In principle, CentOS does not repair the kernel. It reproduces everything from upstream including bugs. In the above example, I also worked with RedHat to get the bug fix applied to their kernel because this is the only way to have that fix in CentOS kernels. As a result, the cifs bug is now supposed to be included in RHEL 5.1 as well as 4.6. I think you should do the same. Push upstream and do it very hard. That's the lesson I learned. Akemi

(0005986) sebycarta (reporter) 2007-09-12 06:19	The bug opened on redhat ( https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=248716 [^] ) has been marked as "assigned", but with "low priority" ( i cannot imagine high priority redhat bugs!). I suggest all "bug friend" to post on redhat bugzilla, as i realize, centos cannot fix the kernel, the solution is in upstream.

(0005990) sdv (reporter) 2007-09-12 09:33	> I am not a centos dev. Just a user like yourself. So I am free to say good luck with debian. Ok, thank you. Another question: does anybody compile mppe module with dkms in CentOS5?

(0005992) toracat (developer) 2007-09-12 12:13	Again, my answer is as a user... dkms is not a preferred method by the centos developers. But if you are interested in building kernel modules, please take a look at: http://wiki.centos.org/HowTos/BuildingKernelModules [^] Akemi

(0005993) toracat (developer) 2007-09-12 14:38	Speaking of modules, there was one more note I forgot to mention. When I saw the description that says the vanilla kernel 2.6.18.4 works fine, I downloaded the source for that version and compiled the ppp_mppe module. But it did not load in CentOS 5. Not surprising given the fact the RHEL kernel is heavily customized. It would require some (or a lot of) hacking to get it to work. Akemi

(0005995) sdv (reporter) 2007-09-13 09:34	Vanilla 2.6.18.4 compiled using 2.6.18.1.8 CentOS5 kernel .config works fine.

(0005996) toracat (developer) 2007-09-13 11:36	Yes, my attempt to use the module from the mainline kernel was based on that description. I did use the .config from CentOS-5 2.6.18-8.1.8.el5 and compilation went through. But the module built that way did not load in the CentOS kernel. If this worked, it would have been much easier to provide a fix -- just load the patched module to the running kernel. A person with good knowledge on the RH kernel might be able to get it to work. Akemi

(0006002) sdv (reporter) 2007-09-13 15:42 edited on: 2007-09-13 15:42	> A person with good knowledge on the RH kernel might be able to get it to work. May be. I'm mot novice in Linux and already have set of my own patches to some old kernels and loop-aes, but only one server was affected and now it works, so, i don't want to spend my time more. Even if somebody will have a patch, i will wait for official solution. May be, vanilla kernel is a bad choice, but affected server isn't production. Thanks for help.

(0006033) sebycarta (reporter) 2007-09-19 20:06	Yeeeee! there is a patch from redhat! Look at https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=248716 [^] the kernel developer issued a patch. I have to try that. Bye.

(0006049) sebycarta (reporter) 2007-09-24 10:02	Only for information: Redhat has been uploaded a patched kernel, you can download it on the redhat bugzilla ( see notes above for the link). Bug resolved, go to work! ;-)

(0006665) jhughes@hughesjr.com (administrator) 2008-01-08 10:19	just to be clear here ... CentOS is a rebuild of upstream items and we provide a product (warts and all) that behaves like the upstream product. It is the WHOLE point and main purpose of CentOS. While we want to help identify and help fix upstream problems (see bugs 0002189 and 0001776 for examples), our main CentOS products will not be updated until a stable release is done upstream. We are not about making a fork, but a predictable OS that performs exactly the same as the products we clone.

Issue History
Date Modified	Username	Field	Change
2007-05-23 17:00	lanconnected	New Issue
2007-05-23 17:00	lanconnected	Status	new => assigned
2007-05-23 17:00	lanconnected	Assigned To	=> kbsingh@karan.org
2007-07-16 06:24	sebycarta	Note Added: 0005580
2007-07-16 06:25	sebycarta	Issue Monitored: sebycarta
2007-07-18 12:31	arrfab	Note Added: 0005604
2007-07-18 12:32	arrfab	Issue Monitored: arrfab
2007-07-18 13:41	sebycarta	Note Added: 0005607
2007-07-18 14:30	arrfab	Note Added: 0005609
2007-07-18 14:38	sebycarta	Note Added: 0005610
2007-08-15 10:07	kingherowu	Note Added: 0005860
2007-09-10 12:44	Martynas	Note Added: 0005969
2007-09-10 21:06	arrfab	Note Added: 0005972
2007-09-11 06:09	Martynas	Note Added: 0005973
2007-09-11 16:02	toracat	Note Added: 0005975
2007-09-11 17:35	sdv	Note Added: 0005977
2007-09-11 17:37	lanconnected	Note Added: 0005978
2007-09-11 17:47	toracat	Note Added: 0005979
2007-09-11 18:04	toracat	Note Added: 0005981
2007-09-12 01:55	toracat	Note Edited: 0005981
2007-09-12 06:19	sebycarta	Note Added: 0005986
2007-09-12 09:33	sdv	Note Added: 0005990
2007-09-12 12:13	toracat	Note Added: 0005992
2007-09-12 14:38	toracat	Note Added: 0005993
2007-09-13 09:34	sdv	Note Added: 0005995
2007-09-13 11:36	toracat	Note Added: 0005996
2007-09-13 15:42	sdv	Note Added: 0006002
2007-09-13 15:42	sdv	Note Edited: 0006002
2007-09-19 20:06	sebycarta	Note Added: 0006033
2007-09-24 10:02	sebycarta	Note Added: 0006049
2008-01-08 10:19	jhughes@hughesjr.com	Note Added: 0006665

Relationships