FirstServed Tech Blog - FirstServed and the Art of Server Tuning

While XenServer and DRBD do play together nicely in various setups, problems can be encountered when using a setup similar to this:
 

sda3 -> LVM -> DRBD -> LVM -> LVM

This occured in a complex XenServer setup we were building.
When booting the (paravirtualised) guest we saw this message in the systems log:

kernel: drbd6: bio would need to, but cannot, be split: (vcnt=4,idx=0,size=4096,sector=4423615)

Simultaniously the guest would complain about unreadable sectors on the drive.

The issue is actually rather simple, and documented on the internet.
But while some xensource users switch over to file-based VM’s as a work-around, this is not an option in XenServer.

So, how can this really be solved?
By simply adding an extra parameter to the drbd.conf: 

disk {
    max-bio-bvecs 1;
}

This simple switch effectively cures the whole situation.
If you want more information about this switch, please read the man files on drbd.conf

man drbd.conf

The switch and what it does is fairly well documented there.

Hi,

Using DRBD on Xenserver is fairly simple.
Just download the Driver Development kit from Citrix for XenServer 4.1.
Import the DDK to your server.
Download your DRBD release (source code) of choice (we used version 8.2.5).
Put it inside the DDK, extract, run the magical command "make rpm" and off you go!

The packages will be ready soon. Just SCP them out of your DDK to any Xenserver you want to install them on.
No additional dependencies are needed to be resolved.

BEWARE: Citrix doesn’t support custom modules (and most likely they won’t ever support DRBD).

Kind Regards,

Koen

You want to use DRBD for mirroring your disks over the network, and put LVM on top of your DRBD disk for flexible disk management?  Try it like this:

1. Setting up DRDB
2. Setting up LVM

However, LVM is rather greedy when it comes to hogging block level devices at boot time.  So, after restarting, it will happily discover your scsi disk, md device, or whatever, and DRBD will spawn some nasty output like:

Starting DRBD resources: [ d0 d1 ioctl(,SET_DISK_CONFIG,) failed:
Invalid argument
Lower device is already mounted.

cmd /sbin/drbdsetup /dev/drbd1 disk /dev/md3 internal -1 failed!

The solution is to edit your lvm.conf to filter out the block devices you've assigned as DRBD devices.  Try modifying your LCM configuration like this ( found in /etc/lvm/lvm.conf on Fedora and Red Hat distros ):

# By default we accept every block device:
# Comment this out:
# filter = [ "a/.*/" ]

# Filter out your DRBD block devices, accept the rest:
filter = [ "r/md3", "a/.*/" ]

After this, rescan your LVM configuration by executing:

pvscan
vgscan
vgchange -a y

And finally start your DRBD daemon:

/etc/init.d drbd start