ERROR porting CIME in config files

[liviasancho@gmail_com]

Dear all,

I am trying to port CIME into an HPC called LoboC, and I am experiencing some problems.
CESM version is cesm2.1.3
Cloned from: git clone -b release-cesm2.1.3 ESCOMP/CESM cesm on September 15th, 2020.
Config files are attached as well as the modules avail in LoboC.

-> I tried to run a test and the following error returns ($CASEROOT is /home/users/sancho/cesm/cases):
sancho@service1:/scratch/21081a/sancho/cesm/cime/scripts> ./create_test /home/users/sancho/cesm/cases/SMS.f19_g17.X.loboc
ERROR: Expected 4th item of '/home/users/sancho/cesm/cases/SMS.f19_g17.X.loboc' ('loboc') to be in form machine_compiler

-> I also tried 2 more ways with the same output
./create_test /home/users/sancho/cesm/cases/SMS.f19_g17.X.loboc --machine=loboc
and
./create_test /home/users/sancho/cesm/cases/SMS.f19_g17.X.loboc --machine=loboc --compiler=intel

-> I also tried to run ECT
sancho@service1:/scratch/21081a/sancho/cesm/cime/tools/statistical_ensemble_test> python ensemble.py --case /home/users/sancho/cesm/cases/uf_ensemble/uf.camcase.cesm.tag.000 --ect cam --uf --mach loboc --project pd00001 --compset F2000climo --res f19_f19_mg17

-> It begins to configure and creates the case directory with all files:
sancho@service1:~/cesm/cases/uf.camcase.cesm.tag.000> ls
archive_metadata case.setup env_batch.xml env_run.xml.orig README.case
Buildconf CaseStatus env_build.xml env_workflow.xml SourceMods
case.build case.submit env_case.xml LockedFiles Tools
case.cmpgen_namelists check_case env_mach_pes.xml pelayout xmlchange
CaseDocs check_input_data env_mach_specific.xml preview_namelists xmlquery
case.qstatus env_archive.xml env_run.xml preview_run

-> But it ends with the following error:
Using charge_account from config_machines.xml: nacad
WARNING: No cesm Model version found.
Batch_system_type is pbs
job is case.run USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None WALLTIME_FORMAT %H:%M:%S
job is case.st_archive USER_REQUESTED_WALLTIME None USER_REQUESTED_QUEUE None WALLTIME_FORMAT %H:%M:%S
Creating Case directory /home/users/sancho/cesm/cases/uf.camcase.cesm.tag.000
STATUS: case = /home/users/sancho/cesm/cases/uf.camcase.cesm.tag.000
STATUS: Adjusting env_run.xml....
setting case file to env_run.xml
setting case file to env_run.xml
setting case file to env_run.xml
setting case file to env_run.xml
setting case file to env_run.xml
STATUS: running setup for single case...
ERROR: Not safe to leave unresolved items in env var value: '${CIME_OUTPUT_ROOT}/cime/tmp'
STATUS: Adjusting user_nl_* files....
Warning: no user_nl_cam found
STATUS: Updating namelists....
ERROR: Not safe to leave unresolved items in env var value: '${CIME_OUTPUT_ROOT}/cime/tmp'
STATUS: no-build = False
STATUS: no-submit = False
STATUS: building case ...
Building case in directory /home/users/sancho/cesm/cases/uf.camcase.cesm.tag.000
sharedlib_only is False
model_only is False
ERROR: ERROR: must invoke case.setup script before calling build script
Error building...

-> I tried to run case.setup on the case directory but the same error returns:
sancho@service1:~/cesm/cases/uf.camcase.cesm.tag.000> ./case.setup
ERROR: Not safe to leave unresolved items in env var value: '$CIME_OUTPUT_ROOT/cime/tmp'

I don't know if it is a misconfiguration in the config files or something I have to deal with LoboC administrators.

I sincerely thank you for your help.

Best,
Lívia Sancho

 

[jedwards]

Oi livia,
In the create test step the format expects the compiler name as well as the machine name so it should be
SMS.f19_g17.X.loboc_intel
also for tests you should not specify the full path, just the testname. If the machine regex in config_machines.xml is working correctly you should not need the machine name in the test
./create_test SMS.f19_g17.X
should work.

In config_machines.xml the variable TMPDIR is probably not needed at all, but if it is needed try setting a resolved path instead of using CIME_OUTPUT_ROOT.

 

[liviasancho@gmail_com]

Hey Jim! Thanks for the reply!

-> For the create_test, I tried
sancho@service1:/scratch/21081a/sancho/cesm/cime/scripts> ./create_test SMS.f19g17.X
Could not find machine match for 'pbsserver' or 'service1'
ERROR: Could not initialize machine object from /scratch/21081a/sancho/cesm/cime/config/cesm/machines/config_machines.xml or /home/users/sancho/.cime/config_machines.xml

-> But then I tried
sancho@service1:/scratch/21081a/sancho/cesm/cime/scripts> ./create_test SMS.f19_g17.X.loboc_intel

-> And it worked fine, I think!! Here is the last part of the output
Finished RUN for test SMS.f19_g17.X.loboc_intel in 156.627667 seconds (PEND). [COMPLETED 1 of 1]
Due to presence of batch system, create_test will exit before tests are complete.
To force create_test to wait for full completion, use --wait
At test-scheduler close, state is:
PEND SMS.f19_g17.X.loboc_intel RUN
Case dir: /scratch/21081a/sancho/cesm/SMS.f19_g17.X.loboc_intel.20201027_172322_i1opmp
test-scheduler took 274.89343214 seconds

-> For the ECT, I removed the TMPDIR from config_machines.xml, reran it, and it advanced a lot but ended with the following error:
sesp built in 4.528527 seconds
swav built in 4.564900 seconds
docn built in 5.951699 seconds
Component atm build complete with 14 warnings
cam built in 136.774492 seconds
Component glc build complete with 3 warnings
cism built in 147.352615 seconds
ifort: error #10106: Fatal error in /sw/apps/intel18/compilers_and_libraries_2018.3.222/linux/bin/intel64/fortcom, terminated by kill signal

cice built in 328.017223 seconds
mosart built in 328.018649 seconds
ERROR: BUILD FAIL: cice.buildlib failed, cat /scratch/21081a/sancho/cesm/uf.camcase.cesm.tag.000/bld/ice.bldlog.201027-180103
Error building...

-> I attached the ice.bldlog.201027-180103 log file and it seems to be a problem with gmake. I've already contacted the machine administrators but if you have any suggestions I would much appreciate it!

Best,
Lívia Sancho

 

[jedwards]

There is no error here - the job was killed, by a user or perhaps by another process.

 

[ykp990521]

Hey Jim! Thanks for the reply!

-> For the create_test, I tried
sancho@service1:/scratch/21081a/sancho/cesm/cime/scripts> ./create_test SMS.f19g17.X
Could not find machine match for 'pbsserver' or 'service1'
ERROR: Could not initialize machine object from /scratch/21081a/sancho/cesm/cime/config/cesm/machines/config_machines.xml or /home/users/sancho/.cime/config_machines.xml

-> But then I tried
sancho@service1:/scratch/21081a/sancho/cesm/cime/scripts> ./create_test SMS.f19_g17.X.loboc_intel

-> And it worked fine, I think!! Here is the last part of the output
Finished RUN for test SMS.f19_g17.X.loboc_intel in 156.627667 seconds (PEND). [COMPLETED 1 of 1]
Due to presence of batch system, create_test will exit before tests are complete.
To force create_test to wait for full completion, use --wait
At test-scheduler close, state is:
PEND SMS.f19_g17.X.loboc_intel RUN
Case dir: /scratch/21081a/sancho/cesm/SMS.f19_g17.X.loboc_intel.20201027_172322_i1opmp
test-scheduler took 274.89343214 seconds

-> For the ECT, I removed the TMPDIR from config_machines.xml, reran it, and it advanced a lot but ended with the following error:
sesp built in 4.528527 seconds
swav built in 4.564900 seconds
docn built in 5.951699 seconds
Component atm build complete with 14 warnings
cam built in 136.774492 seconds
Component glc build complete with 3 warnings
cism built in 147.352615 seconds
ifort: error #10106: Fatal error in /sw/apps/intel18/compilers_and_libraries_2018.3.222/linux/bin/intel64/fortcom, terminated by kill signal

cice built in 328.017223 seconds
mosart built in 328.018649 seconds
ERROR: BUILD FAIL: cice.buildlib failed, cat /scratch/21081a/sancho/cesm/uf.camcase.cesm.tag.000/bld/ice.bldlog.201027-180103
Error building...

-> I attached the ice.bldlog.201027-180103 log file and it seems to be a problem with gmake. I've already contacted the machine administrators but if you have any suggestions I would much appreciate it!

Best,
Lívia Sancho

hello, have you solved that problem? I ran into the same question as you.