CentOS Bug Tracker
CentOS Website
Main | My View | View Issues | Roadmap | Docs
  

Viewing Issue Simple Details [ Jump to Notes ] [ View Advanced ] [ Issue History ] [ Print ]
ID Category Severity Reproducibility Date Submitted Last Update
0003897 [CentOS-5] kernel major always 2009-10-08 17:09 2009-10-26 18:52
Reporter TheMule View Status public  
Assigned To
Priority normal Resolution open  
Status resolved   Product Version 5.3
Summary 0003897: kernel 2.6.18-164 breaks bridging with kvm guest
Description I have 3 kvm guests hosted by a server running 5.3 with kernel 2.6.18-128.2.1.

Their network setup (on the host) is as follows:

br1 8000.001d0908e910 yes vnet0
                                                        vnet2
                                                        vnet1
                                                        eth1

6: br1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue
    link/ether 00:1d:09:08:e9:10 brd ff:ff:ff:ff:ff:ff
    inet xxx.xxx.xxx.139/27 brd xxx.xxx.xxx.159 scope global br1
    inet xxx.xxx.xxx.130/32 brd xxx.xxx.xxx.130 scope global br1:0
    inet xxx.xxx.xxx.131/32 brd xxx.xxx.xxx.131 scope global br1:1
    inet xxx.xxx.xxx.134/32 brd xxx.xxx.xxx.134 scope global br1:2

that is, the three quests are attached to a bridge, and the physical interface (eth1) of the host is attached to that bridge, too.
One example of the guest addresses would be xxx.xxx.xxx.150/27.

On the same ethernet segment (which physically is a single switch) there are other hosts (say, xxx.xxx.xxx.138).

After I upgraded to 2.6.18-164, the guests stopped being reachable from the outside would, that is, xxx.xxx.xxx.138 won't ping xxx.xxx.xxx.150, although it still can be pinged from the host.

After some investigation, I found out that the packets are received by the host (tcpdump -i eth1), they are forwarded to the guest (tcpdump on the guest), the guest replies to the ping, the reply packets are visible on the host (both tcpdump -i vnet1 and tcpdump -i br1 show them), but they are NOT forwarded to eth1 (neither tcpdump -i eth1 on the host nor tcpdump on the ping originator shows them). That's also true for packets originated by the guest.

Rebooting 2.6.18-128.7 solves it (i.e. I can't be a trivial configuration problem).
Additional Information The host is PE SC1435 by Dell, with a dual-core Opteron CPU. Investigation has been carried out on a test system, same configuration, with only one guest. I've also tried disabling iptables at all (iptables -F, all policies to ACCEPT), with no effect.
Attached a guest configuration example. In case you wonder /usr/local/bin/qemu-kvm just invokes the system qemu-kvm with '-serial telnet:127.0.0.1:$port,server,nowait,nodelay' added to the command line (something you can't do with the libvirt xml file yet).
Tags No tags attached.
Attached Files xml file icon guest.xml [^] (714 bytes) 2009-10-08 17:09

- Relationships
has duplicate 0003899closedkbsingh@karan.org kernel upgrade to 2.6.18-164 breaks network in kvm guests 

-  Notes
(0010044)
toracat (developer)
2009-10-08 19:42

 Which version of kvm is this? I would suggest you wait for CentOS 5.4 which should be out soon. 5.4 comes with kvm-83 and kmod-kvm-83. It is possible that older versions of kvm (or the kvm kernel module) have issues with the -164 kernel (part of 5.4).
(0010050)
TheMule (reporter)
2009-10-08 23:29

 kvm is kvm-36-1, the module kmod-kvm-36-3.

I've been directed here from the general support forum. See https://www.centos.org/modules/newbb/viewtopic.php?topic_id=22004&viewmode=flat&order=ASC&start=71 [^]

BTW, "It is possible that older versions of kvm (or the kvm kernel module) have issues with the -164 kernel (part of 5.4)" summarizes pretty well what I've already written there. I do agree the problem most likely lies in the choice of including the 5.4 kernel in the updates for 5.3, but apparently you have no right of speech until you open a bug. So here we go.
(0010127)
toracat (developer)
2009-10-23 18:00

 > TheMule wrote:
> "the problem most likely lies in the choice of including the 5.4 kernel in the updates for 5.3"

Could you please confirm this was indeed the case now that 5.4 is out? Did kvm work normally after the system update to 5.4?
(0010167)
TheMule (reporter)
2009-10-26 11:03

 I've done only one test, but it's positive. I had to change the configuration a bit, it won't run right after the upgrade. In 5.3 I was forced to use a wrapper script around qemu-kvm (BTW, the oath used to be /usr/bin/qemu-kvm now it's /usr/libexec/qemu-kvm) to add all the options I wasn't able to control with the libvirt xml config file. It seems that now libvirt is more sensitive about the executable, it complains about my script ('internal error cannot parse QEMU version number').

Anyway, the new libvirt now supports all the options I need (serial console via telnet) so I just moved the specs into the xml and changed the 'emulator' element back to point to the system qemu-kvm, and it works. The guest can be pinged from the outside, that is.

You may close the bug, as far as I'm concerned (it seems I can't do it directly). Should I find other issues, I'll reopen or open another, whichever is fit.

Thank you.

P.S.
Oh, and please take note to never include alien packages in the updates repository again. The 5.4 kernel is definitely 'alien' to the 5.3 tree. It's ok to 'jump' from 5.3 to 5.4 via updates, it's the partial upgrade the problem. Just don't let your left foot take a jump before the rest of your body is ready to follow: the outcome usually hurts. :)
(0010170)
toracat (developer)
2009-10-26 14:49

 First of all, I believe that the developers are planning to make security updates in a place different from the current release updates. This way, people who want to get the update can do so conscientiously.

Second, this is asking. Could you provide help with updating the Wiki kvm page at:

http://wiki.centos.org/HowTos/KVM [^]

What you reported here will definitely help others who are updating their existing kvm from the CentOS extras.
(0010172)
TheMule (reporter)
2009-10-26 18:09

 That WikiPage doesn't even mention libvirt stuff. It's plain old style qemu-kvm (which btw was affected by the same 'bug', anyway, I tried and manually ran it to factor out libvirt issues).

My setup is based on libvirt.org documentation, which is another world. Actually, I should modify this page

http://wiki.centos.org/HowTos/Xen/InstallingHVMDomU [^]

which is much closer to what I do. It's almost the same, with a different hypervisor. Since I knew this was the direction things were evolving to, I decided to go the libvirt way with KVM even if the support was very limited in 5.3, and it forced me to implement some workarounds.

I can't make promises, but the setup here needs to be documented anyway, and I don't mind publishing the docs. Maybe a new WikiPage is in order, something like 'Installing and using libvirt and KVM on Centos 5.4'.
(0010173)
toracat (developer)
2009-10-26 18:51

 Yes, in fact Fabian Arrotin is/was planning to rewrite the KVM page. I also was thinking of adding the libvirt stuff to the wiki. Your participation would be greatly appreciated.

I would like to ask you to send a message to the centos-docs mailing list so that you get the edit access to those pages you referenced (please mention this bug tracker # 3897).

Thanks in advance.

- Issue History
Date Modified Username Field Change
2009-10-08 17:09 TheMule New Issue
2009-10-08 17:09 TheMule Assigned To => kbsingh@karan.org
2009-10-08 17:09 TheMule File Added: guest.xml
2009-10-08 19:42 toracat Note Added: 0010044
2009-10-08 23:29 TheMule Note Added: 0010050
2009-10-09 14:38 toracat Relationship added has duplicate 0003899
2009-10-23 18:00 toracat Note Added: 0010127
2009-10-23 18:00 toracat Status new => feedback
2009-10-26 11:03 TheMule Note Added: 0010167
2009-10-26 14:49 toracat Note Added: 0010170
2009-10-26 18:09 TheMule Note Added: 0010172
2009-10-26 18:51 toracat Note Added: 0010173
2009-10-26 18:52 toracat Status feedback => resolved
2009-10-26 18:52 toracat Fixed in Version => 5.4

Copyright © 2000 - 2009 Mantis Group
Powered by Mantis Bugtracker