Setting MODULEPATH

[m_mineter@ed_ac_uk]

In porting to a linux cluster I have got confused over how to manage
1) MODULEPATH
2) my python environment for this - i need to set a conda environemnt for the python I want to use and am puzzled where to put this and how - as "source activate mypython" deosnt fit neatly it seems to me into the module setup.

I set these in ,bashrc, but assume that is not relevant.

Also, in my config_machines.xml, I put the full path to module files, but got errors from some of the cheyenne tests, like:
"module command {} failed with message:\n{}".format(cmd, errout))
File "/exports/csce/eddie/geos/groups/cesd/CESM/my_cesm_sandbox/cime/scripts/Tools/../../scripts/lib/CIME/utils.py", line 130, in expect
raise exc_type(msg)
SystemExit: ERROR: module command /exports/applications/apps/SL7/environment-modules-3.2.10/Modules/3.2.10/bin/modulecmd python load /exports/applications/modulefiles/SL7/Compilers/intel/2017u4 /exports/csce/eddie/geos/groups/cesd/modules/intel_2017_u4/parallel_netcdff/4.5.3 failed with message:
2017u4(56):ERROR:107: 'MODULEPATH' not set

I donl;t see any example xml files setting MODULEPATH.
Where should I set it please?!

Thanks
Mike

 

[jedwards]

Setting the MODULEPATH variable is done by the module init script. The path to the module init script is set in config_machines.xml and looks like:
```<init_path lang="python">/glade/u/apps/th/opt/lmod/8.1.7/lmod/lmod/init/env_modules_python.py</init_path>
```
You should set the path to the init script for sh,csh,python, perl .

The python environment that you set in your shell should be the one that cesm will use. Make sure that your batch system is exporting the environment to compute nodes.

 

[m_mineter@ed_ac_uk]

Setting the MODULEPATH variable is done by the module init script. The path to the module init script is set in config_machines.xml and looks like:
```<init_path lang="python">/glade/u/apps/th/opt/lmod/8.1.7/lmod/lmod/init/env_modules_python.py</init_path>
```
You should set the path to the init script for sh,csh,python, perl .

The python environment that you set in your shell should be the one that cesm will use. Make sure that your batch system is exporting the environment to compute nodes.

Thanks for the prompt response.

When you say "Make sure that your batch system is exporting the environment to compute nodes" is that the bash environment, or the PYTHONPATH and PATH or what please? How do you routinely do this please.... I hope UNIVA lets me do likewise,.


"

 

[jedwards]

I am not familiar with the details of your queueing system - if you do a man on the submit command you should see a flag to export the environment.

 

[m_mineter@ed_ac_uk]

I am not familiar with the details of your queueing system - if you do a man on the submit command you should see a flag to export the environment.

For the whole environment, yes -V does exist in univa

The MODULEPATH remains an issue for me because my module files are in nonstandard directories so the system's default MODULEPATH needs to be extended,.
as in equivalent of export MODULPATH="mydir1":"mydir2":$MODULEPATH

Can I make that change in the machines.xml?
<environment_variables>

I guess not.... but how should I do that please

 

[jedwards]

You shouldn't have to do that, when you module add the path to your directory the MODULEPATH variable should be updated to reflect that.

 

[m_mineter@ed_ac_uk]

I asked some questions above that weren't too well-formed, and in case someone bumps into this unfamiliar with modules as used in CESM:

1.module use... extends MODULEPATH and so in config_machines lines such as
<command name="use">/exports/csce/eddie/geos/groups/CESM/sw/modules</command>
<command name="load">cesm_p_netcdf/1.12.1</command>
first set the path to the module file i want, then load it.

2. To find out what to put in the cmd_path in config_machines for perl, python etc, see the path in $MODULESHOME or $BASH_FUNC (when logged in and with modules enabled).

No response asked for from Jim and colleagues