Skip to main content

MooseFS on XCP-NG

This guide provides a step-by-step instruction for installing MooseFS Pro on XCP-NG machines, allowing it to be set up as a storage provider for XCP-NG in a multi-node cluster.

1. Add MooseFS repository to all XCP-NG nodes​

curl "https://repository.moosefs.com/RPM-GPG-KEY-MooseFS" > /etc/pki/rpm-gpg/RPM-GPG-KEY-MooseFS
curl "https://repository.moosefs.com/MooseFS-4-el7.repo" > /etc/yum.repos.d/MooseFS.repo

2. Install MooseFS Master Server package on selected XCP-NG nodes​

yum install moosefs-pro-master

3. Make the domain name of the Master Server resolvable on all XCP-NG nodes​

The domain name of the Master Server (mfsmaster or mfsmaster.vm.lan in this example) has to be resolvable on all XCP-NG nodes, either via DNS, or via /etc/hosts file. If you decide to use the hosts file, make sure that it is consistent across all the nodes.

For example:

192.168.0.11		mfsmaster.vm.lan	mfsmaster
192.168.0.12		mfsmaster.vm.lan	mfsmaster


192.168.0.11		node1.vm.lan		node1
192.168.0.12		node2.vm.lan		node2
192.168.0.13		node3.vm.lan		node3
192.168.0.14		node4.vm.lan		node4
192.168.0.15		node5.vm.lan		node5

127.0.0.1 localhost.localdomain localhost

4. Enable and start MooseFS Master Server​

systemctl enable --now moosefs-pro-master

5. Install Chunk Servers on all XCP-NG nodes​

A Chunk Server must be present on all XCP-NG nodes that will be used as data storage providers. Install moosefs-pro-chunkserver package on these.

yum install moosefs-pro-chunkserver

6. Prepare disks for Chunk Servers​

Do not mount the disks for storage in XCP-NG. Instead, prepare them for Chunk Servers.

For each storage provider node:

  1. Format all disks designated as Chunk Server disks with a file system of your choice. We recommend using XFS for HDD and EXT4 for SSD/NVMe disks.
  2. Add them to /etc/fstab
  3. Reload systemd:
systemctl daemon-reload
  1. Mount all fstab mountpoint:
mount -a
  1. Change the ownership of Chunk Server mountpoints to mfs:mfs, for example:
chown -R mfs:mfs /mnt/chunks*
  1. Add the mountpoints to /etc/mfs/mfshdd.cfg file. For example:
# cat /etc/mfs/mfshdd.cfg
/mnt/chunks1
/mnt/chunks2
/mnt/chunks3
/mnt/chunks4

7. Enable and start the Chunk Server service​

systemctl enable --now moosefs-pro-chunkserver

8. Install MooseFS Client on each XCP-NG node​

yum install moosefs-pro-client
yum install fuse
yum install fuse-libs

9. Enable MooseFS plugin on all XCP-NG nodes​

By default, the moosefs storage driver is not enabled and must be whitelisted in XAPI's configuration.

The list of accepted storage drivers is defined in /etc/xapi.conf but we must never modify this file directly. Instead, create dedicated file with the sm-plugins definition:

echo "sm-plugins=ext nfs iscsi lvmoiscsi dummy file hba rawhba udev iso lvm lvmohba lvmofcoe shm smb cephfs glusterfs moosefs xfs zfs largeblock" > /etc/xapi.conf.d/99-enable-moosefs.conf

⚠️ XAPI only takes the last definition of sm-plugins it reads into account. Files are read in alphabetical order. If there's already a configuration file in /etc/xapi.conf.d (to enable another additional storage driver), take it into consideration when you write your new definition of sm-plugins.

Restart xen tools stack

Xe-toolstack-restart

10. Connect to an existing MooseFS cluster​

Now when the MooseFS client is installed and enabled you can connect to an existing MooseFS cluster and create a shared SR for all hosts in the pool.

xe sr-create type=moosefs name-label=MooseFS-SR content-type=user shared=True device-config:masterhost=mfsmaster.vm.lan device-config:masterport=9421 device-config:rootpath=/xcp-ng

We also suggest using a folder on the MooseFS cluster as a root path rather than using the direct path of the cluster.