Skip to main content

Setting Up Multilocations

Upgrading from MooseFS 4.x​

The upgrade procedure for MooseFS Pro is the same as for any version upgrade. A freshly upgraded MooseFS 5 instance looks and behaves exactly like MooseFS 4 – the tools for working with locations are installed, but the feature itself is not activated. Enabling Multilocations requires a conscious decision after upgrading.

Configuring a New Instance with Multilocations​

A new MooseFS 5 instance can be set up with Multilocations straightforwardly. Decide on the number of locations, choose short but meaningful names, and follow these steps:

  1. Identify your locations – which IP addresses belong to which locations and what will the locations be named?
  2. Create your locations – add the second (and any further) locations, then rename the initial default location to the desired name.
  3. Turn all locations ON.
  4. Map IPs to locations.
  5. Create storage classes.
  6. Start recording data.

Example: Content Provider​

A MooseFS instance serving static content (e.g. movies) to users, with modules distributed across two data centers: one in Chicago and one in New York. Both locations serve content actively. Data changes very rarely after it is recorded, and the desired redundancy level (RL) is 2.

Setup information:

  • Locations: chicago (default) and newyork
  • Chicago modules: 192.168.1.x network
  • New York modules: 10.10.1.x network

Step 2 – Create locations​

mfslocadmin create newyork
mfslocadmin rename default chicago

Step 3 – Turn locations ON​

The default location (chicago) is already ON. Only the newly created one needs to be enabled:

mfslocadmin state -s ON newyork

Step 4 – Map IPs​

mfslocadmin map -i 192.168.1.0/24 chicago
mfslocadmin map -i 10.10.1.0/24 newyork

Step 5 – Create storage class​

mfsscadmin create -o f -l chicago -K 3 -A @8+2 -l newyork -K 3 -A @8+2 content

This class stores 3 copies of content in each location. Once created, chunks are converted to EC 8+2 format as quickly as possible (-o f, fast mode) to save space. Each location requires at least 12 chunkservers (13 recommended).

tip

If fewer chunkservers are available, use EC 4+2 instead of EC 8+2 (replace @8+2 with @4+2). This requires only 8 chunkservers per location (9 recommended) at the cost of slightly higher storage usage.

Step 6 – Notes​

chicago is set as the default location. Because this setup is symmetric, it does not matter much which location is default – unless one is more prone to failures (e.g. network or power), in which case the more stable one should be the default. To switch the default location off for maintenance, first promote the other location:

mfslocadmin setdefault newyork
mfslocadmin state -s OFF chicago

Example: Research Facility​

A MooseFS instance with 3 locations: one main location where clients perform research on data, and two backup locations. New files are created only in the main location; existing data is kept in all locations. Data is considered cold after 24 hours without modification. Desired RL: 2 in the main location, 1 in each backup.

Setup information:

  • Locations: main (default), backup1, backup2
  • main: 2 masters at 10.10.1.1–10.10.1.2, 16 chunkservers at 10.10.1.11–10.10.1.26
  • backup1: 1 master + 8 chunkservers at 10.10.1.31–10.10.1.39
  • backup2: 1 master + 8 chunkservers at 10.10.1.41–10.10.1.49

Step 2 – Create locations​

mfslocadmin create backup1
mfslocadmin create backup2
mfslocadmin rename default main

Step 3 – Turn locations ON​

mfslocadmin state -s ON backup1
mfslocadmin state -s ON backup2

Step 4 – Map IPs​

mfslocadmin map -i 10.10.1.1-10.10.1.2 main       # masters
mfslocadmin map -i 10.10.1.11-10.10.1.26 main     # chunkservers
mfslocadmin map -i 10.10.1.31-10.10.1.39 backup1  # master + chunkservers
mfslocadmin map -i 10.10.1.41-10.10.1.49 backup2  # master + chunkservers

Step 5 – Create storage class​

mfsscadmin create -o m -d 24 -l main -K 3 -A @8+2 -l backup1 -C 0 -K 2 -A @4+1 -l backup2 -C 0 -K 2 -A @4+1 data

What this does:

  • New chunks are created only in main (3 copies, inherited from KEEP). The backup locations have an explicit -C 0, so no new chunks are recorded there.
  • Once recorded, chunks replicate to 2 copies in each backup location (-K 2).
  • After 24 hours of inactivity (-o m -d 24), the archive flag is set and chunks convert to their ARCHIVE format: EC 8+2 in main (16 chunkservers available), EC 4+1 in each backup (only 8 chunkservers – EC 8+2 would require at least 10).

Step 6 – Notes​

main is the default location. With 3 locations, a leader is elected whenever at least 2 are active — this follows strict majority voting (more than half). Since 3 is odd, an exactly-half tie cannot occur, so the default location plays no tie-breaking role in leader election for this setup.