Modules

From Peyton Hall Documentation
Jump to navigation Jump to search

Modules are an easy way to configure software which relies on environment variables to function, as well as maintain multiple versions of such software. The module system utilizes an executable program and aliases which call it, plus script files called 'modulefiles' which tell it what to do, and outputs shell-readable commands which are interpreted to perform the various actions needed to configure paths and environment variables for each program.

History

When disk space was costly, instead of installing all the same programs on every desktop (which can also make maintaining such software difficult) a network-shared directory called '/usr/peyton' was created which housed software that could be run on many machines. Unfortunately, /usr/peyton was used as a dumping ground for many things; New browser version available? Dump it in /usr/peyton/bin. Some software needed by a few people for their research? Dump it in /usr/peyton, and link it into /usr/peyton/bin. Over the years, /usr/peyton has become a cesspool of unused, underused and unusable programs (some of them won't run on any of the current machines, either because they're built for architectures that no longer exist or linked against libraries so old that they're no longer installed). Some programs are actively maintained, because they're segregated enough from the rest that maintenance is possible (such as IDL, Mathematica and Firefox). Others however are hopelessly intertwined in the directories and can't easily be separated from the rest to update them. With this in mind, a solution was sought that would allow for centrally-maintained software to be installed on the network, while allowing for ease of updates and selection of specific program versions if needed. Initially, a directory called /usr/peyton/utils was created and a group of interested users were given access to write to it so they could maintain a repository of useful programs. Unfortunately it too started to become cumbersome and suffer similar problems, one of which was how to use the software contained therein (one had to edit environment variables to include the right path at the right location or else the wrong version would be loaded). A better solution had to be created, and using the "Modules" package seemed like the best fit.


What It Does

The Modules package is simply a small executable program which reads in "modulefiles", which are specially formatted TCL scripts. One writes a module file for a specific program, and tells module to read it; module will then modify your environment variables accordingly so that whatever is needed for setting up the program is done. This could mean adding a specific directory to your PATH or adding new environment variables, for example. Modules can have dependencies, and most importantly for our purposes they can have multiple versions; for example, you can have a module to setup IDL version 8.1, but still have modules for version 7.1 and 6.4. This allows for multiple versions of a program to be installed, and users can choose the one they want to be able to use or use the latest by default, all without having to modify their shell startup files (something which has been problematic for certain programs, such as the Intel compilers). We have modules setup for most of the software programs one might want to run, and others are being added as programs are installed. You can also create your own modulefiles and put them in a path so that module can find them (see below).


How To Use

Invoking from the command line

  • module help will show you the available commands
  • module load <modulefile> will load the named module; module unload <modulefile> will unload it
    • If you want to use a specific version, append a slash and the version number. For example, module load idl/7.1 to load IDL 7.1
    • To switch module versions, instead of unloading and reloading you can use module switch idl/8.1 to unload any previous IDL module and load 8.1
  • module avail will show the modules available
  • module list lists the modules currently in use
  • module help <modulefile> will display any specific help for that module
  • module show <modulefile> will show you what commands would be run in the shell for that module


Invoking via shell startup scripts

If there's certain modules you always use, you can add the appropriate calls to your shell startup scripts; for example, adding to your ~/.bashrc module load idl intel would have the paths for IDL and the Intel compilers setup for you when you login. Once you've added these somewhere, you can later use the module init commands (such as module initadd) to edit those lines automatically; this is no different than editing the file(s) directly and adding, removing, switching or clearing the specified modules but exists for simplicity. As the manpage notes, using one of these commands will add/remove the module from every module command found in the script(s).

If you login to our machines from the console (that is, sitting right in front of the machine), we highly recommend using the following syntax for loading modules. This will prevent problems with some environmental variables getting clobbered by X itself. Instead of loading the modules every time your shell executes (which it does when you start an X session), it will instead only load when you have an interactive shell (which is typically the only time you want modules loaded).

If you use bash, use the following template (modified for the modules you want loaded, of course):

if [ "$PS1" ]; then
   module load firefox thunderbird
fi

If you use [t]csh, then the following template should work:

if ( $?prompt ) then
   module load firefox thunderbird
endif

You will need to log out of X and back in for these changes to take full effect.


Creating modulefiles

You can create your own modulefiles and tell the module system to look for them. One way to do this is by editing your ~/.modulerc file to contain the following:

#%Module
module use --append <your modulefile directory>

This will tell module to add the appropriate path to its list of modulefile directories when it first starts.

Another way is to use the "use.own" module; module load use.own will load it, and create a directory called ~/privatemodules/ if it doesn't already exist. Any modules in that directory will then be added to the search paths for the module commands. Note that it also does the same as the above 'module use' command; simply creating the directory is not sufficient for module to find its contents.

For more information on the content of modulefiles and how to create them, see the manpages for module and modulefile as well as the contents of existing modules for examples.


FAQ

Using a specific version by default

If you don't want to load the latest version of something available, you can specify its version on the module load command (ie, module load idl/7.1). If you want that default to be sticky and always be the one used, you can tell module that. Create the file ~/.modulerc as such:

#%Module
module-version idl/7.1 default

Now when you tell module to load idl, it will load that version by default.