Storage Classes

Storage Classes extend MooseFS's classic numeric goal system with a flexible, label-based mechanism for controlling where and how chunks are stored. They allow administrators to specify placement policies per file or directory — choosing not just how many copies to keep, but on which Chunkservers they land.

For backward compatibility, MooseFS ships with nine predefined Storage Classes (named 1 through 9) that behave like goals from earlier versions. Goal tools operate only on these predefined classes. Any file that had goal N before upgrading to MooseFS 3.0 will have Storage Class N assigned automatically.

Labels

Labels are single letters (A–Z) assigned to Chunkservers. Each Chunkserver can carry up to 26 labels simultaneously. Labels are defined in /etc/mfs/mfschunkserver.cfg:

# labels string (default is empty - no labels)
LABELS = A

Multiple labels can be written as XYZ, X,Y,Z, or X Y Z — all three forms are equivalent. After editing the configuration, reload the Chunkserver to apply the change without interrupting service:

mfschunkserver reload

or using the service manager:

service moosefs-pro-chunkserver reload

Once labels are applied, the CGI monitor's Servers tab shows which labels each Chunkserver carries. If no labels are configured, the system balances data across all Chunkservers the same way MooseFS 2.x did.

Label expressions

A label expression defines the placement policy for a Storage Class. It is a comma-separated list of subexpressions, where each subexpression specifies the placement of one copy:

• A single letter matches Chunkservers carrying that label.
• A multiplication (e.g. AB) matches Chunkservers that carry all listed labels (logical AND).
• A sum (e.g. A+B) matches Chunkservers that carry any of the listed labels (logical OR).
• Brackets combine operations: B[X+Y] matches Chunkservers that have label B and either X or Y.
• An asterisk (*) matches any Chunkserver.
• A numeric prefix is shorthand for repetition: 2A is equivalent to A,A.

Examples:

Expression	Meaning
A,B	Two copies: one on label-A servers, one on label-B servers
A,*	Two copies: one on label-A servers, one on any server
2* or 2	Two copies on any servers (equivalent to goal 2)
AB,C+D	Two copies: one on servers with both A and B (AND); one on servers with C or D (OR)
A,B[X+Y],C[X+Y]	Three copies: one on A servers; one on servers with B and (X or Y); one on servers with C and (X or Y)
A,A	Equivalent to 2A
A,3BC	Equivalent to A,BC,BC,BC

Creation, keep, and archive labels

Each Storage Class can define up to three placement phases that chunks move through over their lifetime:

• Creation labels (-C CREATION_LABELS): where chunks are written at the moment of file creation. This is an optional parameter; if omitted, the keep labels are used for creation as well. Creation labels are useful when clients are physically closer to a specific set of servers and write performance matters.
• Keep labels (-K KEEP_LABELS): the long-term home for chunks, maintained at all times outside of the creation and archive phases. This is the only mandatory parameter for a Storage Class.
• Archive labels (-A ARCH_LABELS with -d ARCH_DELAY): where chunks are moved once a file has not been modified for more than the number of days specified with -d. The system determines archive eligibility based on the file's mtime.

This lets you design full data lifecycle policies in a single Storage Class definition: write fast to nearby SSDs, replicate to cross-site redundant servers for ongoing availability, then automatically migrate to capacity-optimized HDD storage after a configurable idle period — all without manual intervention.

Chunk creation modes

When there is no free space on the Chunkservers targeted by a Storage Class, or those servers are temporarily overloaded, MooseFS behavior is governed by the creation mode assigned to the class:

Mode	Chunkserver is full	Chunkserver is overloaded
Loose (-m L)	Use servers with other labels	Use servers with other labels
Default (-m D)	Use servers with other labels	Wait for available Chunkserver
Strict (-m S)	Return ENOSPC	Wait for available Chunkserver

The default mode is a balanced choice: it keeps writes flowing when space runs out on labeled servers by temporarily using others, but it does not silently bypass overloaded servers — it waits for them to recover, which preserves placement intent for time-sensitive operations. Strict mode is appropriate when policy compliance is more important than availability (for example, when data must never leave a labeled set of servers). Loose mode prioritizes availability above all.

You can observe current Chunkserver states in the Resources tab of the CGI monitor.

Preferred labels for read/write

Clients can specify preferred Chunkservers for I/O using the mfspreflabels mount option:

-o mfspreflabels=LABELEXPR

LABELEXPR is an ordered list of up to nine label expressions, separated by commas (e.g. E1,E2,E3). Each mountpoint can have its own preferred labels, allowing clients in different locations or with different roles to favor different sets of Chunkservers.

How priority is calculated

When a Client performs a read, the Master returns a list of chunk locations in random order. For each Chunkserver in the list, the Client calculates a priority as the lowest index y for which that Chunkserver's labels match expression Ey. If no expression matches, the priority is set to the number of expressions plus one (lowest priority).

The Client then sorts the list by priority — lower numbers win — and reads from the highest-priority Chunkserver. If multiple Chunkservers share the same priority, the one that has handled the fewest operations from this Client is preferred, which distributes load. If a read fails, the Client falls back to the next-best copy.

For write operations the same priority sorting applies, but when two Chunkservers share the same priority the ordering returned by the Master is used rather than operation count. If no mfspreflabels option is set, the Master's list order is used directly with no modifications.

This mechanism allows, for example, a Client in datacenter A to read from local Chunkservers labeled A while transparently falling back to datacenter B Chunkservers on failure, without any changes to Storage Class definitions.

Chunkserver states

Chunkservers operate in three states visible in the CGI monitor's Servers tab:

• Normal — standard operating state; load shown as a plain number (e.g. 7).
• Internal rebalance — activated when a new disk is added to a Chunkserver; the server redistributes chunks across all of its drives to equalize utilization. Load shown in round brackets (e.g. (7)). Data remains accessible during this process.
• Overloaded — the Chunkserver cannot accept additional operations and notifies the Master. Once the load drops, the Master is informed and normal scheduling resumes. Load shown in square brackets (e.g. [77]). Chunk creation behavior when a Chunkserver is overloaded depends on the active creation mode.

Managing Storage Classes

mfsscadmin — Storage Class administration

mfsscadmin creates and manages Storage Class definitions. It can be run from any directory — the /MOUNTPOINT prefix tells it where MooseFS is mounted:

mfsscadmin /mnt/mfs create 2B sclass2

If you are already inside the MooseFS mount, the mountpoint prefix can be omitted.

Synopsis

mfsscadmin [/MOUNTPOINT] create|make [-a admin_only] [-m creation_mode]
    [-C CREATION_LABELS] -K KEEP_LABELS [-A ARCH_LABELS -d ARCH_DELAY] SCLASS_NAME...

mfsscadmin [/MOUNTPOINT] create|make [-a admin_only] [-m creation_mode]
    LABELS SCLASS_NAME...

mfsscadmin [/MOUNTPOINT] change|modify [-f] [-a admin_only] [-m creation_mode]
    [-C CREATION_LABELS] [-K KEEP_LABELS] [-A ARCH_LABELS] [-d ARCH_DELAY] SCLASS_NAME...

mfsscadmin [/MOUNTPOINT] delete|remove SCLASS_NAME...
mfsscadmin [/MOUNTPOINT] copy|duplicate SRC_SCLASS_NAME DST_SCLASS_NAME...
mfsscadmin [/MOUNTPOINT] rename SRC_SCLASS_NAME DST_SCLASS_NAME
mfsscadmin [/MOUNTPOINT] list [-l]

Commands

• create|make — creates a new Storage Class; providing multiple names creates multiple identical classes in one command.
• change|modify — updates one or more options on an existing class without affecting unspecified options.
• delete|remove — removes a class; if the class is still assigned to any objects, the command fails with an error. Empty classes are removed regardless.
• copy|duplicate — copies an existing class definition under a new name.
• rename — changes the name of a class without altering its definition.
• list — lists all classes by name; add -l to show full definitions including label expressions and options.

Options

Option	Description
-C CREATION_LABELS	Placement for newly created chunks (optional; defaults to keep labels)
-K KEEP_LABELS	Permanent chunk placement (mandatory)
-A ARCH_LABELS	Archive placement (requires -d)
-d ARCH_DELAY	Days since last modification before archiving
-a 0|1	0 = available to all users, 1 = admin only
-m D|L|S	Creation mode: Default, Loose, or Strict
-f	Force changes on a predefined Storage Class
-l	Long listing: show full definitions, not only class names

Listing Storage Classes

The list command without -l shows class names only:

mfsscadmin list

1
2
...
9
sclass1
sclass2

With -l, it shows the full definition of each class:

mfsscadmin list -l

sclass1: 2; admin_only:NO; create_mode:STD; create_labels:[A],[A]; keep_labels:[A],[A]
sclass2: 2; admin_only:NO; create_mode:STD; create_labels:[B],[B]; keep_labels:[B],[B]

Predefined Storage Classes

Classes 1 through 9 are predefined on every fresh or upgraded MooseFS instance. Their definitions are * through 9*, making them exact equivalents of classic numeric goals. They can be modified only when -f is specified, and they cannot be deleted or renamed. On an upgraded system, it is recommended to create new named classes and migrate files to them with mfsxchgsclass rather than modifying the predefined ones.

mfssclass — Storage Class assignment tools

These tools read and change the Storage Class assignment on files and directories:

• mfsgetsclass [-r] OBJECT... — print the current Storage Class of each object.
• mfssetsclass [-r] SCLASS_NAME OBJECT... — assign a Storage Class to each object.
• mfscopysclass [-r] SOURCE_OBJECT OBJECT... — copy the Storage Class from one object to one or more others.
• mfsxchgsclass [-r] SRC DST OBJECT... — replace Storage Class SRC with DST, but only on objects whose current class is SRC. Objects with a different class are left unchanged.
• mfslistsclass [-l] [MOUNT_POINT] — list all currently defined Storage Classes; -l shows full definitions.

The -r flag enables recursive operation on directories — it descends into subdirectories and processes all contained files and directories.

Inheritance

When a new object is created inside a MooseFS directory, it inherits the Storage Class (along with trashtime and extra attributes) from its parent directory at the moment of creation. Changing a directory's Storage Class later does not affect objects that already exist inside it. To apply a new class to a directory and all its existing contents, use mfssetsclass -r.

Output format

Most mfstools accept format flags for numeric output: -n (exact numbers), -h (binary prefixes: Ki, Mi, Gi), -H (SI prefixes: k, M, G), -k/-m/-g (plain kibis, mebis, gibis). The same behavior can be set via the MFSHRFORMAT environment variable: 0 = exact, 1 or h = binary prefixes, 2 or H = SI prefixes, 3 or h+ = exact and binary, 4 or H+ = exact and SI. The default is exact numbers.

Usage examples

Two server rooms

Label Chunkservers in room A as A and Chunkservers in room B as B. Create a Storage Class that places one copy in each room, then apply it recursively:

mfsscadmin create A,B crosssite
mfssetsclass -r crosssite /mnt/mfs/data

MooseFS will ensure that exactly one copy of each chunk is kept on an A-labeled server and one on a B-labeled server. If the filesystem appears unbalanced in the monitor, it is in fact balanced to the extent the Storage Class constraints allow.

note

A slow link between sites will reduce write performance because chunk writes are synchronous. A high-speed interconnect is strongly recommended for multi-site deployments.

SSD and HDD tiers

Label SSD Chunkservers as S and HDD Chunkservers as H. Store active data in four copies on SSDs for performance; once the data ages, change its Storage Class to three copies on HDDs to reclaim SSD space:

# Store active data on SSDs
mfsscadmin create 4S 4ssdcopies
mfssetsclass -r 4ssdcopies /mnt/mfs/Reports2025

# Later, move cold data to HDDs
mfsscadmin create 3H 3hddcopies
mfsxchgsclass -r 4ssdcopies 3hddcopies /mnt/mfs/Reports2025

mfsxchgsclass changes only the Storage Class assignment. MooseFS then takes care of physically moving the chunks from SSD to HDD servers in the background, keeping the data accessible throughout the migration.

Two server rooms with SSD and HDD tiers

Combine room labels with drive-type labels to build fine-grained placement policies:

• Server room A, SSD Chunkservers: labels A and S
• Server room A, HDD Chunkservers: labels A and H
• Server room B, SSD Chunkservers: labels B and S
• Server room B, HDD Chunkservers: labels B and H

Store frequently accessed data on SSDs, one copy per room:

mfsscadmin create AS,BS frequent
mfssetsclass -r frequent /mnt/mfs/Frequent

Store infrequently accessed data on HDDs, one copy per room:

mfsscadmin create AH,BH rare
mfssetsclass -r rare /mnt/mfs/Rare

Store backup data with one SSD copy in room A and two copies (SSD or HDD) in room B using a combined label expression:

mfsscadmin create AS,2B[H+S] backup
mfssetsclass -r backup /mnt/mfs/Backup

The expression AS,2B[H+S] places one copy on servers that have both A and S, and two copies on servers that have B and either H or S.

note

A slow link between sites will reduce write performance because chunk writes are synchronous. A high-speed interconnect is strongly recommended for multi-site deployments.

Creation, keep, and archive lifecycle

Suppose clients are located near server room A and need fast initial writes. The goal is to create chunks quickly on local SSD servers, then replicate to both sites for durability, and finally migrate to HDD storage after 30 days of inactivity.

Create the directory and define a Storage Class with all three phases:

mkdir /mnt/mfs/ImportantFiles

mfsscadmin create -C 2AS -K AS,BS -A AH,BH -d 30 important
mfssetsclass important /mnt/mfs/ImportantFiles

• -C 2AS — new chunks are written to two SSD Chunkservers in room A (fast local write).
• -K AS,BS — chunks are kept on one SSD server per room (cross-site redundancy).
• -A AH,BH -d 30 — after 30 days without modification, chunks are moved to HDD servers in both rooms.

MooseFS monitors mtimes and transitions chunks through each phase automatically. The data remains accessible at all times.