Installing Environment Modules

It’s been a while since I’ve done a proper Linux post. I found Environmental Modules a while ago, and it is an utterly fantastic idea. From their blurb:

What are Environmental Modules?

The Environment Modules package provides for the dynamic modification of a user’s environment via modulefiles.

Each modulefile contains the information needed to configure the shell for an application. Once the Modules package is initialized, the environment can be modified on a per-module basis using the module command which interprets modulefiles. Typically modulefiles instruct the module command to alter or set shell environment variables such as PATH, MANPATH, etc. modulefiles may be shared by many users on a system and users may have their own collection to supplement or replace the shared modulefiles.

Modules can be loaded and unloaded dynamically and atomically, in an clean fashion. All popular shells are supported, including bash, ksh, zsh, sh, csh, tcsh, as well as some scripting languages such as perl.

Modules are useful in managing different versions of applications. Modules can also be bundled into metamodules that will load an entire suite of different applications.

Let’s install!

This tutorial assumes that you are logged in as root, on Debian.

Resolve dependencies

apt-get install tcl tcl8.4-dev

Get the package and untar it

tar xvvf modules-3.2.9c.tar.gz

Make the folders to store your packages and modulefiles

mkdir /packages && mkdir /modules

Go into the modules folder

cd modules-3.2.9

Setup the build

./configure --with-module-path=/modules/

Make and install it

make && make install

Edit the modulefiles path. Comment out all lines except /modules/

vim /usr/local/Modules/3.2.9/init/.modulespath

Copy the following file

cp etc/global/profile.modules /etc/profile.d/

The file should be

# system-wide profile.modules #
# Initialize modules for all sh-derivative shells #
trap "" 1 2 3


case "$0" in
    -bash|bash|*/bash) . $MODULES/init/bash ;;
       -ksh|ksh|*/ksh) . $MODULES/init/ksh ;;
          -sh|sh|*/sh) . $MODULES/init/sh ;;
                    *) . $MODULES/init/sh ;; # default for scripts

trap - 1 2 3

Installing applications from source

So the install is complete, but how do you actually use modules? Let’s assume you want to use a different version of gcc that isn’t in your repo and you want to install gcc version 4.6.2. Download gcc, untar it and cd into the directory.

Create a directory for gcc-4.6.2 to be installed into.

mkdir -p /packages/gcc/4.6.2

Configure gcc to install to this directory.

./configure --prefix=/packages/gcc/4.6.2

Install as normal.

make && make install

If we look inside /packages/gcc/4.6.2, we’ll see a number of folder like bin, lib64, etc. So now we need to create the associated modulefile.

Creating module files

Above we specified that module files be placed in /modules, so that’s where we’ll put gcc’s module files.

Create a gcc directory, if there isn’t one

mkdir /modules/gcc

Add the associated module file

vim /modules/gcc/4.6.2

What’s in that file, then?

proc ModulesHelp { } {
global dotversion

puts stderr "\tGCC 4.6.2 (gcc, g++, gfortran)"

module-whatis "GCC 4.6.2 (gcc, g++, gfortran)"
conflict gcc
prepend-path PATH /packages/gcc/4.6.2/bin
prepend-path LD_LIBRARY_PATH /packages/gcc/4.6.2/lib64
prepend-path LIBRARY_PATH /packages/gcc/4.6.2/lib64
prepend-path MANPATH /packages/gcc/4.6.2/man
setenv CC gcc
setenv CXX g++
setenv FC gfortran
setenv F77 gfortran
setenv F90 gfortran

Modules allows you to set default versions of packages. So, say you have 4 versions of gcc, and you’d like the 4.6.2 as the default version, you can set it in a version file.

vim /modules/gcc/.version
set ModulesVersion "4.6.2"

How do I use modules?

Well, it’s about bloody time that we finally get to use the damn modules we’ve setup, otherwise you’d drive to my house and beat the piss out of me.

List the modules on your system with module avail.

[nick@zoidberg ~]$ module avail

---------------------------------- /modules/ -----------------------------------
gcc/4.6.2(default) haskell/ghc/7.0.4

The (default) means that I can just load gcc without specifying the version numbers.

Load a module on your system with module load

Before we do this, I’ll assure you it works.

[nick@zoidberg ~]$ gcc --version
gcc (Debian 4.4.5-8) 4.4.5

Let’s load gcc version 4.6.2

[nick@zoidberg ~]$ module load gcc/4.6.2
[nick@zoidberg ~]$ gcc --version
gcc (GCC) 4.6.2

We can also load this version of gcc without specifying the version number, as 4.6.2 is the default.

[nick@zoidberg ~]$ module load gcc
[nick@zoidberg ~]$ gcc --version
gcc (GCC) 4.6.2

See what modules are loaded

The modules loaded will always contain version numbers, if you’re install them into the same folder structure as myself.

[nick@zoidberg ~]$ module list
Currently Loaded Modulefiles:
  1) /gcc/4.6.2

Unloading modules

The syntax for unloading modules is the same as loading them.

[nick@zoidberg ~]$ module unload gcc
[nick@zoidberg ~]$ gcc --version
gcc (Debian 4.4.5-8) 4.4.5