Skip to main content

Networking and Protocols

This section describes the networking requirements, supported technologies, and communication protocols used within a MooseFS deployment. Proper network configuration is essential to ensure the high availability, performance, and scalability of MooseFS.

IP Networking​

MooseFS components (Master Servers, Chunkservers, Metaloggers, and Clients) communicate over standard IP networks using TCP/IP. Each MooseFS service listens on specific ports for incoming connections, which are described in details in the Network chapter of the Requirements section.

Administrators should ensure these ports are open and properly routed in firewalls and security groups where necessary. While MooseFS operates on default ports, they can be reconfigured in the service settings if needed.

It is recommended to assign static IP addresses (or persistent DHCP leases) to all MooseFS nodes to maintain stable and predictable network connections.

Infiniband / IP over IB (IPoIB)​

For deployments requiring ultra-low latency and high-throughput networking, MooseFS supports communication over Infiniband networks via IP over Infiniband (IPoIB).

Using IPoIB, MooseFS operates transparently over Infiniband hardware while still relying on standard TCP/IP protocols. This enables MooseFS to benefit from Infiniband's bandwidth and low latency without requiring protocol modifications.

Considerations for IPoIB:

  • MTU (Maximum Transmission Unit) settings might need to be adjusted (commonly up to 65520 bytes).
  • Jumbo frames and proper tuning of network stack parameters can significantly improve performance.
  • Consistent network interface naming and routing configurations are important when combining IPoIB with Ethernet.

Physical Media​

MooseFS is network-agnostic regarding the underlying physical media. It supports a wide range of network infrastructures, including:

  • Ethernet
  • Infiniband (via IPoIB)
  • Fiber optics and other high-speed links

For production deployments, at least 1 Gbps Ethernet is recommended, with 10 Gbps or higher preferred for clusters with high throughput requirements or large numbers of clients and chunkservers.

Network reliability and redundancy (e.g., bonded interfaces, redundant switches) are highly recommended to ensure high availability.

MooseFS Protocol​

MooseFS components communicate using a proprietary, highly optimized protocol built on top of TCP/IP.

Key characteristics of the MooseFS protocol:

  • Binary format: Efficient data serialization minimizes overhead and maximizes performance.
  • Session-based communication: Persistent TCP connections are used for control commands, metadata synchronization, chunk management, and client operations.
  • Heartbeat mechanism: Regular heartbeat signals maintain active connections and detect failures quickly.

While the MooseFS protocol is not publicly documented for third-party integration, all official MooseFS tools and libraries are fully compliant and maintained to support compatibility across versions.