GlusterFS Installation

Installation on Proxmox VE #

Logical Volume and File System Setup #

To start with GlusterFS, we need to install the server dependency:

$ apt-get install glusterfs-server -y

After we’ve done that we need to create the Logical Volume where we’ll setup Gluster.

$ pvcreate /dev/sdb
$ vgcreate vg_glusterfs /dev/sdb
$ lvcreate -l100%FREE -n lv_gluster vg_glusterfs

Make a Filesystem for Gluster to sync.

$ mkfs.xfs /dev/vg_glusterfs/lv_gluster

Now we can start the glusterd service.

$ systemctl enable glusterd
$ systemctl start glusterd

And create the data directory for glusterfs:

$ mkdir -p /data/gluster

# Create an FSTAB entry for the volume:

$ echo /dev/vg_glusterfs/lv_gluster /data/gluster xfs defaults 0 0 >> /etc/fstab

$ mount /data/gluster
$ mkdir -p "/data/gluster/brick{1}"

Once we’ve verified that the local LV (for example with df -h) is correctly mounted, and we’ve done this on every node that will have GlusterFS we can start adding peers!

$ gluster peer probe pve01.local.home
$ gluster peer probe pve02.local.home
$ gluster peer probe arbiter-a.local.home

#Output should be the following:
$ gluster peer probe pve02
Probe successful
$ gluster peer probe arbiter-a
Probe successful

Gluster Peer Configuration #

Two Storage Nodes and an Arbiter #

In this configuration scheme there will be 2 storage nodes and an arbiter with less space (to avoid split-brains).

After all nodes have been probed we can create the gluster volume:

$ gluster volume create gluster_vol replica 3 arbiter 1 pve01:/data/gluster/brick1 pve02:/data/gluster/brick2 arbiter-a:/data/gluster/arbitera

Now we need to start the volume so that all data in the volume gets replicated to each storage node.

$ gluster volume start gluster_vol

Three Storage Nodes or More #

To set it up with more nodes and no arbiter, it’s basically the same process as above, but easier because you don’t have to calculate the size of the arbiter block device:

$ gluster volume create gluster_vol replica 4 pve01:/data/gluster/brick1 pve02:/data/gluster/brick2 pve03:/data/gluster/brick3 pve04:/data/gluster/brick4

And now you can start the volume.

$ gluster volume start gluster_vol

Done!