Derecho Lmod Error: Unknown module "craype" blocking case.setup in release-cesm2.1.5 and cesm2.2.2 (F2000climo)

[usmanummuqulthum]

I am attempting to spin up a standalone F2000climo case on NCAR's Derecho supercomputer using a clean checkout of release-cesm2.1.5 and cesm2.2.2 , but the invocation of case.setup is completely blocked during the machine-specific environment configuration phase.

Issue Summary​

CIME attempts to dynamically load the software stack via Lmod wrappers but fails because it is explicitly requesting the module "craype", which is currently unrecognized or deprecated in my active environment. Because the module framework throws a fatal error, the environment handshake fails, aborting case.setup and preventing the compilation process (case.build) from proceeding.

Case Details​

• Model Version: release-cesm2.1.5
• Machine: Derecho (crhtc45.hpc.ucar.edu)
• Compset: F2000climo (Longname: 2000_CAM60_CLM50%SP_CICE%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV)
• Grid Resolution: f09_g17

 

[fischer]

Hi Usman,

I did a clean checkout of release-2.1.5 and ran
./create_newcase --compset F2000climo --res f09_g17 --case /glade/derecho/scratch/fischer/F2000_2.1.5_BB --run-unsupported
./case.setup
./case.build

I was able to build the model without any issues. Can you show me the create_newcase command you used, and attach the software_environment.txt in your case directory. Or point me to your case directory.

Thanks
Chris

 

[usmanummuqulthum]

Hi Chris,

Thanks for looking into this! Regarding the previous build failure, I realized that I mistakenly tried to run that setup on Casper instead of Derecho. Once I switched over to Derecho, that specific issue was resolved and the build went through successfully.

However, I am now running into a different issue on a new case where I changed the compset to a BSSP126CMIP. To help troubleshoot, I have attached my case run logs and other diagnostic documents. You can also find my active case directory at: /glade/u/home/usman/GGW_cases/SSP_SCENERIOS/cesm2.1.5.BSSP126cmip6.f09_g17.BSSP126_Daily_Run_215
Latest Batch Run Log in that directory: run.cesm2.1.5.BSSP126cmip6.f09_g17.BSSP126_Daily_Run_215.o6601236

Scratch Run Directory (for cesm.log and component logs):/glade/derecho/scratch/usman/cesm2.1.5.BSSP126cmip6.f09_g17.BSSP126_Daily_Run_215/run/

Please let me know if you need me to send over any additional logs or configuration files.

Best,

Usman

 

[fischer]

Hi Usman,

It looks like the number of tasks being used for WW3 is to high. But I see that you set the number of tasks for WW3 to 32. I'm not sure exactly how the ocean and ww3 interact, but I think ww3 might be seeing the number of tasks for the ocn. Also, you didn't change all of the ROOTPE, so this run is using 18 nodes instead of the 10 nodes you expected. What you should do is use the default processors layout for this run. Remove any changes to NTASKS, NTHRDS, and ROOTPE from your script. This will give you a run that uses 9 nodes.

NTASKS: ['CPL:1024', 'ATM:1024', 'LND:512', 'ICE:512', 'OCN:128', 'ROF:512', 'GLC:1024', 'WAV:256', 'ESP:1']
ROOTPE: ['CPL:0', 'ATM:0', 'LND:0', 'ICE:512', 'OCN:1024', 'ROF:0', 'GLC:0', 'WAV:0', 'ESP:0']

Chris