Modules

From Peyton Hall Documentation

Revision as of 17:23, 31 October 2011

Modules are an easy way to maintain multiple software installations which rely on environment variables to function.

This article is actively undergoing a major edit for the day to write new documentation. As a courtesy, please do not edit this page while this message is displayed. The person who added this notice will be listed in its edit history or has placed their signature above. If this page has not been edited recently (several hours!), please remove this template (or replace it with {{underconstruction}}). This message is intended to help reduce edit conflicts; please remove it between editing sessions to allow others to improve this page.

History

When disk space was costly, instead of installing all the same programs on every desktop (which can also make maintaining such software difficult) a network-shared directory called '/usr/peyton' was created which housed software that could be run on many machines. Unfortunately, /usr/peyton was used as a dumping ground for many things; New browser version available? Dump it in /usr/peyton/bin. Some software needed by a few people for their research? Dump it in /usr/peyton, and link it into /usr/peyton/bin. Over the years, /usr/peyton has become a cesspool of unused, underused and unusable programs (some of them won't run on any of the current machines, either because they're built for architectures that no longer exist or linked against libraries so old that they're no longer installed). Some programs are actively maintained, because they're segregated enough from the rest that maintenance is possible (such as IDL, Mathematica and Firefox). Others however are hopelessly intertwined in the directories and can't easily be separated from the rest to update them. With this in mind, a solution was sought that would allow for centrally-maintained software to be installed on the network, while allowing for ease of updates and selection of specific program versions if needed. Initially, a directory called /usr/peyton/utils was created and a group of interested users were given access to write to it so they could maintain a repository of useful programs. Unfortunately it too started to become cumbersome and suffer similar problems, one of which was how to use the software contained therein (one had to edit environment variables to include the right path at the right location or else the wrong version would be loaded). A better solution had to be created, and using the "Modules" package seemed like the best fit.

What It Does

The Modules package is simply a small executable program which reads in "modulefiles", which are specially formatted TCL scripts. One writes a module file for a specific program, and tells module to read it; module will then modify your environment variables accordingly so that whatever is needed for setting up the program is done. This could mean adding a specific directory to your PATH or adding new environment variables, for example. Modules can have dependencies, and most importantly for our purposes they can have multiple versions; for example, you can have a module to setup IDL version 8.1, but still have modules for version 7.1 and 6.4. This allows for multiple versions of a program to be installed, and users can choose the one they want to be able to use or use the latest by default, all without having to modify their shell startup files (something which has been problematic for certain programs, such as the Intel compilers). We have modules setup for most of the software programs one might want to run, and others are being added as programs are installed. You can also create your own modulefiles and put them in a path so that module can find them (see below).

How To Use

Invoking from the command line

• module help will show you the available commands
• module load <modulefile> will load the named module; module unload <modulefile> will unload it
  • If you want to use a specific version, append a slash and the version number. For example, module load idl/7.1 to load IDL 7.1
  • To switch module versions, instead of unloading and reloading you can use module switch idl/8.1 to unload any previous IDL module and load 8.1
• module avail will show the modules available
• module list lists the modules currently in use
• module help <modulefile> will display any specific help for that module
• module show <modulefile> will show you what commands would be run in the shell for that module

Invoking via shell startup scripts

If there's certain modules you always use, you can add the appropriate calls to your shell startup scripts; for example, adding to your ~/.bashrc module load idl intel would have the paths for IDL and the Intel compilers setup for you when you login. Once you've added these somewhere, you can later use the module init commands (such as module initadd) to edit those lines automatically; this is no different than editing the file(s) directly and adding, removing, switching or clearing the specified modules but exists for simplicity. As the manpage notes, using one of these commands will add/remove the module from every module command found in the script(s).

Creating modulefiles

You can create your own modulefiles and tell the module system to look for them. One way to do this is by editing your ~/.modulerc file to contain the following:

#%Module
module use --append <your modulefile directory>

This will tell module to add the appropriate path to its list of modulefile directories when it first starts.

Another way is to use the "use.own" module; module load use.own will load it, and create a directory called ~/privatemodules/ if it doesn't already exist. Any modules in that directory will then be added to the search paths for the module commands.