Reliable Server Pooling (RSerPool)

Welcome to Thomas Dreibholz's
Reliable Server Pooling (RSerPool) Page


Quick Navigation

Latest News

  • August 19, 2010
    RSPLIB has just been added to the FreeBSD ports collection! To install RSPLIB on FreeBSD, just go to /usr/ports/net/rsplib and run "make && make install"!
  • August 09, 2010
    Interested in research on protocols and applications with multi-homing support? Have a look at the Call for Papers of the 1st International Workshop on Protocols and Applications with Multi-Homing Support (PAMS 2011)!
  • July 24, 2010
    Added BibTeX and updated RSerPool Internet Drafts in the Internet Drafts section to the latest versions
  • July 23, 2010
    The new stable version 2.7.5 of RSPLIB is now available in the Download section!     This version contains a bug fix for the scripting service, which is recommended for users of SimProcTC.
  • July 8, 2010
    The new stable version 2.7.4 of RSPLIB is now available in the Download section! This version contains some scripting service improvements, which are required for the upcoming version of SimProcTC.
  • June 30, 2010
    The new stable version 2.7.2 of RSPLIB is now available in the Download section!
  • June 19, 2010
    The new stable version 2.7.1 of RSPLIB is now available in the Download section!
  • May 24, 2010
    Added the SERA 2010 presentation slides to the  Research section.
  • April 10, 2010
    Added the SERA 2010 paper to the  Research section.
  • January 04, 2010
    Updated the RSerPool Internet Drafts in the Internet Drafts section to the latest versions.
  • December 29, 2009
    The userland SCTP packages sctplib and socketapi have been updated to the latest stable versions. Note: these packages are only necessary when using RSPLIB on systems providing no kernel SCTP support (e.g. PlanetLab nodes).
  • December 7, 2009
    The new stable version 2.7.0 of RSPLIB is now available in the Download section!     Furthermore, a new section "RSerPool/RSPLIB" in the Media" can be found here.
  • November 30, 2009
    I have been awarded for my work on RSerPool with the award "Wissenschaftspreis der Sparkasse Essen". Read the press release here! Photos of the awarding ceremony can be found here.
  • October 26, 2009
    The new development version 2.7.0~rcA5.2 of RSPLIB is available in the Download section. Also, sctplib and socketapi have been updated to the latest versions.
  • October 16, 2009
    The new development version 2.7.0~rcA4.9 of RSPLIB is available in the Download section.
  • September 15, 2009
    The new development version 2.7.0~rcA4 of RSPLIB is available in the Download section.
  • July 5, 2009
    Updated the drafts in the Internet Drafts section. Furthermore, the SCTP/RSerPool teaching material in the Teaching Material section has been updated.
  • News Archive

    Go to the News Archive

RSerPool/RSPLIB in the Media

November 30, 2009

I have been awarded for my work on RSerPool with the award "Wissenschaftspreis der Sparkasse Essen".

News Articles

Photos

September 26, 2008

RSerPool/RSPLIB presentation to software developers and press in Haikou, Hainan/China.

TV Report

News Articles

Photos

RSerPool Presentations and Events

Interested in a presentation of RSerPool? Do not hesitate to contact us! Write e-mail to Thomas Dreibholz.

  • Talk by Xing Zhou on RSerPool security at the 8th <st1:personname w:st="on">ACIS</st1:personname> International Conference on Software Engineering Research, M<st1:personname w:st="on">ana</st1:personname>gement and Applications (<st1:stockticker w:st="on">SERA</st1:stockticker>2010), Montreal/Canada on May 24, 2010.
  • Further presentation schedules coming soon!

What is Reliable Server Pooling?

Introduction

The development and standardization of an application-independent server pooling architecture has been set as the goal of the IETF RSerPool WG. As a result, the working group has created their concept Reliable Server Pooling abbreviated as RSerPool, which at the moment consists on one RFC, several Internet Drafts and our prototype RSPLIB as reference implementation.

Requirements to the Reliable Server Pooling Architecture

As key requirements to the Reliable Server Pooling architecture, the following points has been identified:

Lightweight

The RSerPool solution may not require a significant amount of resources (e.g. CPU power or memory). In particular, it should be possible to realize RSerPool-based systems also on low-power devices like mobile phones, PDAs and embedded devices.

Real-Time

Real-time services like telephone signalling have very strict limitations on the duration of failovers. In the case of component failures, it may be necessary that a "normal" system state is re-established within a time frame of only a few hundreds of milliseconds. In telephone signalling, such a feature is in particular crucial when dealing with emergency calls.

Scalability

Providing services like Distributed Computing, it is necessary to manage pools of many hundreds or even thousands of servers (e.g. animation rendering pools). The RSerPool architecture must be able to efficiently handle such pools. But amount and size of pools are limited to a company or organization. In particular, it is not a goal of RSerPool to handle the global Internet in one pool set.

Extendability

It must be possible to easily adapt the RSerPool architecture to future applications. In particular, this means to have the possibility to add new server selection procedures. That is new applications can define special rules on which server of the pool is the most appropriate to use for the processing of a request (e.g. the least-used server). The configuration effort of RSerPool components (e.g. adding or removing servers) should be as small as possible. In the ideal case, the configuration should happen automatically, i.e. it should e.g. only be necessary to turn on a new server and it will configure automatically.

The Reliable Server Pooling Architecture


The figure above shows the building blocks of the RSerPool architecture, which has been defined by the IETF RSerPool WG in [draft-ietf-rserpool-arch]. In the terminology of RSerPool a server is denoted as a Pool Element (PE). In its Pool, it is identified by its Pool Element Identifier (PE ID), a 32-bit number. The PE ID is randomly chosen upon a PE's registration to its pool. The set of all pools is denoted as the Handlespace. In older literature, it may be denoted as Namespace. This denomination has been dropped in order to avoid confusion with the Domain Name System (DNS). Each pool in a handlespace is identified by a unique Pool Handle (PH), which is represented by an arbitrary byte vector. Usually, this is an ASCII oder Unicode name of the pool, e.g. "DownloadPool" or "WebServerPool".

Each handlespace has a certain scope (e.g. an organization or company), denoted as Operation Scope. It is an explicit non-goal of RSerPool to manage the global Internet's pools within a single handlespace. Due to the limitation of operation scopes, it is possible to keep the handlespace "flat". That is, PHs do not have any hierarchy in contrast to the Domain Name System with its top-level and sub-domains. This constraint results in a significant simplification of the handlespace management.

Within an operation scope, the handlespace is managed by redundant Registrars. In literature, this component is also denoted as ENRP Server or Name Server. Since "registrar" is the most expressive term, this denotation is used in the whole document. PRs have to be redundant in order to avoid a PR to become a single point of failure (SPoF). Each PR of an operation scope is identified by its Registrar ID (PR ID), which is a 32-bit random number. It is not necessary to ensure uniqueness of PR IDs. A PR contains a complete copy of the operation scope's handlespace. PRs of an operation scope synchronize their view of the handlespace using the Endpoint HaNdlespace Redundancy Protocol (ENRP) defined in [draft-ietf-rserpool-enrp,draft-ietf-rserpool-common-param]. Older versions of this protocol use the term Endpoint Namespace Redundancy Protocol; this naming has been replaced to avoid confusion with DNS, but the abbreviation has been kept. Due to handlespace synchronization by ENRP, PRs of an operation scope are functionally equal. That is, if any of the PRs fails, each other PR is able to seamlessly replace it.

Using the Aggregate Server Access Protocol (ASAP), defined in [draft-ietf-rserpool-asap,draft-ietf-rserpool-common-param] a PE can add itself to a pool or remove it from by requesting a registration or deregistration from an arbitrary PR of the operation scope. In case of successful registration, the PR chosen for registration becomes the PE's Home-PR (PR-H). A PR-H not only informs the other PRs of the operation scope about the registration or deregistration of its PEs, it also monitors the availability of its PEs by ASAP Keep-Alive messages. A keep-alive message by a PR-H has to be acknowledged by the PE within a certain time interval. If the PE fails to answer within a certain timeout, it is assumed to be dead and immediately removed from the handlespace. Furthermore, a PE is expected to re-register regularly. At a re-registration, it is also possible for the PE to change its list of transport addresses or its policy information (to be explained later).

To use the service of a pool, a client - called Pool User (PU) in RSerPool terminology - first has to request the resolution of the pool's PH to a list of PE identities at an arbitrary PR of the operation scope. This selection procedure is denoted as Handle Resolution. For the case that the requested pool is existing, the PR will select a list of PE identities according to the pool's Pool Member Selection Policy, also simply denoted as Pool Policy.

Possible pool policies are e.g. a random selection (Random) or the least-loaded PE (Least Used). While in the first case it is not necessary to have any selection information (PEs are selected randomly), it is required to maintain up-to-date load information in the second case of selecting the least-loaded PE. Using an appropriate selection policy, it is e.g. possible to equally distribute the request load onto the pool's PEs.

After reception of a list of PE identities from a PR, a PU will write the PE information into its local cache. This cache is denoted as PU-side Cache. Out of its cache, the PU will select exactly one PE - again using the pool's selection policy - and establish a connection to it using the application's protocol, e.g. HTTP over SCTP or TCP in case of a web server. Using this connection, the service provided by the server is used. For the case that the establishment of the connection fails or the connection is aborted during service usage, a new PE can be selected by repeating the described selection procedure. If the information in the PU-side cache is not outdated, a PE identity may be directly selected from cache, skipping the effort of asking a PR for handle resolution. After re-establishing a connection with a new PE, the state of the application session has to be re-instantiated on the new PE. The procedure necessary for session resumption is denoted as Failover Procedure and is of course application-specific. For an FTP download for example, the failover procedure could mean to tell the new FTP server the file name and the last received data position. By that, the FTP server will be able to resume the download session. Since the failover procedure is highly application-dependent, it is not part of RSerPool itself, though RSerPool provides far reaching support for the implementation of arbitrary failover schemes by its Session Layer mechanisms.

To make it possible for RSerPool components to configure automatically, PRs can announce themselves via UDP over IP multicast. These announces can be received by PEs, PUs and other PRs, allowing them to learn the list of PRs currently available in the operation scope. The advantage of using IP multicast instead of broadcast is that this mechanism will also work over routers (e.g. LANs connected via a VPN) and the announces will - for the case of e.g. a switched Ethernet - only be heard and processed by stations actually interested in this information. For the case that IP multicast is not available, it is of course possible to statically configure PR addresses.

A Migration Path for Legacy Applications

RSerPool is a completely new protocol framework. To make it possible for existing specialized or proprietary server pooling solutions to iteratively migrate to a RSerPool-based solution, it is mandatory to provide a migration path. For clients without support for RSerPool, the RSerPool concept provides the possibility of a Proxy PU (PPU). A PPU handles requests of non-RSerPool clients and provides an intermediation instance between them and the RSerPool-based server pool. From a PE's perspective, PPUs behave like regular PUs. Similar to a PPU allowing the usage of a non-RSerPool client, it is possible to use a Proxy PE (PPE) to continue using a non-RSerPool server in a RSerPool environment.

The Protocol Stack


The figure above shows the protocol stack of PR, PE and PU. The ENRP protocol is only used for the handlespace synchronization between PRs, all communications between PE and PR (registration, re-registration, deregistration, monitoring) and PU and PR (handle resolution, failure reporting) is based on the ASAP protocol. The failover support, based on an optional Session Layer between PU and PE, is also using ASAP. In this case, the ASAP protocol data (Control Channel) is multiplexed with the application protocol's data (Data Channel) over the same connection. Using the Session Layer functionality of ASAP, a pool can be viewed as a single, highly available server from the PU's Application Layer perspective. Failure detection and handling is mainly handled automatically in the Session Layer, transparent for the Application Layer.

The transport protocol used for RSerPool is usually SCTP. The important properties of SCTP requiring its usage instead of TCP are the following:

  • Multi-homing and path monitoring by Heartbeat messages for improved availability and verification of transport addresses,

  • Address reconfiguration (Add-IP, see [draft-ietf-tsvwg-addip-sctp]) to enable mobility and interruption-free address changes (e.g. adding a new network interface for enhanced redundancy),

  • Message framing for simplified message handling (especially for the Session Layer),

  • Security against blind flooding attacks by 4-way handshake and verification tag, and

  • Protocol identification by Payload Protocol Identifier (PPID) for protocol multiplexing (required for the ASAP Session Layer functionality).

For the transport of PR announces by ASAP and ENRP via IP multicast, UDP is used as transport protocol. The usage of SCTP is mandatory for all ENRP communication between PRs and the ASAP communication between PEs and PRs. For the ASAP communication between PU and PR and the Session Layer communication between PE and PU it is recommended to use SCTP, but the usage of TCP together with an adaptation layer defined in [draft-ietf-rserpool-tcpmapping] is possible. This adaptation layer adds functionalities like Heartbeats, message framing and protocol identification on top of a TCP connection. But nevertheless, some important advantages of SCTP are missing - especially the high immunity against flooding attacks and the multi-homing property. The only meaningful reason to use TCP is when the PU implementation cannot be equipped with an SCTP stack, e.g. when using a proprietary embedded system providing only a TCP stack.

A Screenshot

Watch a RSerPool/RSPLIB demonstration video here (Xvid codec required)!

A screenshot of our fractal graphics demo application: two PEs provide a fractal graphics computation service, which is requested by two PUs.


RSerPool RFCs and Internet Drafts

If something is missing, see the Internet Draft archive at Watersprings.org. Please also contact me in case of missing or dead links.

Research Publications and Presentations

Teaching Material

LaTeX source versions of the following teaching material is available on request. Write e-mail to Thomas Dreibholz.

Our Implementation

RSPLIB, socketapi and sctplib are released under the GNU Public Licence (GPL).

Current Stable Version: 2.7.2

Please have a look at the Handbook and the ReadMe files before installation!

Important: This version has been updated to the latest drafts, which have changed some parameter IDs. That is: this version will be incompatible to former implementations of ASAP and ENRP (e.g. rsplib up to version 2.3.x)! Furthermore, the default multicast addresses have been changed to 239.0.0.50 (ASAP) and 239.0.0.51 (ENRP). Finally, kernel SCTP usage is now the default for configure.

If you do not want to use kernel SCTP, you also need our userland SCTP implementation sctplib/socketapi. In this case, you also need the following two files. However, in most cases, you want kernel SCTP and simply use --enable-kernel-sctp at configure. If you are unsure, use kernel SCTP. See the handbook for details!

Old stable versions

Please have a look at the ReadMe file before trying this version!

The files have been signed using my GnuPG key. You can find my public key here. You will also find a mirror for the stable version files here: http://www.sctp.de/rserpool.html.

Ubuntu binary package installation

I have created Ubuntu binary packages in a Launchpad PPA repository. To use the repository, add the following lines to /etc/apt/sources.list (Important: you may have to replace "maverick" by the name of your actual Ubuntu distribution, e.g. maverick, lucid, karmic, intrepid, gutsy, feisty, edgy): 

deb ppa.launchpad.net/dreibh/ubuntu maverick main
deb-src ppa.launchpad.net/dreibh/ubuntu maverick main

Furthermore, also add my GnuPG key to ensure that the packages are valid: 

gpg --keyserver keyserver.ubuntu.com --recv-keys CCEB82DF916C56E0 && gpg --export -a CCEB82DF916C56E0 | sudo apt-key add -

After sources configuration, you can install the RSPLIB packages. The following packages are available:

  • rsplib-registrar - RSerPool registrar
  • rsplib-tools - RSerPool test tools
  • librsplib-dev - Headers for RSerPool core API library rsplib
  • librsplib2 - RSerPool core API library rsplib
  • rsplib-services - Example RSerPool services

FreeBSD ports installation

To install RSPLIB under FreeBSD from the ports collection, just go to "/usr/ports/net/rsplib" and run "make && make install" (as root!).

Installation from sources

  • 1. If you are using kernel SCTP, skip this step. Else, get a compatible version of the sctplib from http://www.sctp.de/sctp.html
    • Install it:
      prompt> tar xzvf sctplib-XXXX.tar.gz
      prompt> cd sctplib-XXX
      prompt> ./configure
      prompt> make
      prompt> make install (as root!)
    • Now, the sctplib should be installed under /usr/local.
  • 2. If you are using kernel SCTP, skip this step. Else, get a compatible version of the socketapi from http://www.sctp.de/sctp.html
    • Install it:
      prompt> ./configure
      If configure fails due to incompatible version of the sctplib, go to step 1 and install a compatible version first. Then, try again.
      prompt> make
      prompt> make install (as root!)
  • 3. Install RSPLIB:
    • Install it:
      prompt> ./configure
      To use userland SCTP, use the parameter --disable-kernel-sctp!
      prompt> make
    • Read the ReadMe file in the main directory for how to use the examples.

Our Mailing Lists

RSerPool Introductions on Wikipedia in Different Languages

What about helping Wikipedia by adding an article in your language?

Other Sources

 

Sie befinden sich gerade hier:

Institut für Informatik und Wirtschaftsinformatik