Inputdada can’t be read correctly in CESM2.1.4: NetCDF: Variable not found/Invalid dimension ID or name/Attribute not found

[wilma wang]

Hi everyone, I am still working to solve the issue as shown in the last thread(CESM2.1.4 case_run error: without any hist output nc files).
I find there may be something wrong about the inputdata used in F2000climo because the cesm.log (also attached) says a lot of message as following:

/home/bozk/Models/inputdata/atm/waccm/lb/LBC_2000climo_CMIP6_0p5degLat_c180227.
 nc      196608
 NetCDF: Variable not found
 NetCDF: Variable not found
 NetCDF: Variable not found
 NetCDF: Variable not found
 NetCDF: Variable not found
 WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
 WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
 WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
 WARNING: Rearr optional argument is a pio2 feature, ignored in pio1

Shoud i update the version of NetCDF on my machine (now NetCDF-4.4.1 and netCDF-F is 4.4.4) and retry it?
Look forward to any useful information.

Best regards,

Wilma W.

 

[wilma wang]

I also find a similar post in the forum as following:

Hi Jesse,

Thank you for for your useful response. I also had similar doubt regarding the libraries (openmpi) it is using within current setup at HPC. Because model seems running well but it is not writing the final restrat pointer and files. I have explained the issue to HPC folks and they are looking into it.

Meanwhile, regarding your other questions.

1.) what happens if you use the CESM2.1.5 code base instead?

- I tried the same set but at the resolution f19_g16 (basically 2 degree), It gives an issue while initilizing the model. I posted the issue on forum and I am waiting for someone to reply, Here it is CLM: NaN Value while running slab ocean (ETEST) compset

Meanwhile, I got some hints to try it on newer branch. So i tried the same setup on cesm2.2.2.

I tried the same experiment setup, but at a f19_g16 resolution on cesm2.2.2 before changing to f09_g16. It asks for the 4 nodes at machine and this runs fine (Successful) without any issue. I am attaching the log file here (cesm222_f19_Ecmp.tar).

2.) what happens if you run an out-of-the-box F-case, like F2000climo?

- I also tried a F compset case at f09_g16 resolution :

./create_newcase --case /scratch/rs9552/cesm2.2.2_T2/F_CAM62k_CLM5BGCCrop_T1 --res f09_g16 --compset 2000_CAM60_CLM50%BGC-CROP_CICE%PRES_DOCN%DOM_MOSART_CISM2%NOEVOLVE_SWAV --machine greene --run-unsupported



It also gave the same issue, I am attaching log from this too.

Stack trace terminated abnormally.
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
Stack trace terminated abnormally.
forrtl: error (78): process killed (SIGTERM)
Image PC Routine Line Source
Stack trace terminated abnormally.
srun: Job step aborted: Waiting up to 32 seconds for job step to finish.

Please check.

HPC folks are also looking into this. Any hint would be a great help if you notice something which we are not able to catch.

Thank You.

-Ram

Any comments are welcome!

 

[dbailey]

Hi everyone, I am still working to solve the issue as shown in the last thread(CESM2.1.4 case_run error: without any hist output nc files).
I find there may be something wrong about the inputdata used in F2000climo because the cesm.log (also attached) says a lot of message as following:

/home/bozk/Models/inputdata/atm/waccm/lb/LBC_2000climo_CMIP6_0p5degLat_c180227.
 nc      196608
 NetCDF: Variable not found
 NetCDF: Variable not found
 NetCDF: Variable not found
 NetCDF: Variable not found
 NetCDF: Variable not found
 WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
 WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
 WARNING: Rearr optional argument is a pio2 feature, ignored in pio1
 WARNING: Rearr optional argument is a pio2 feature, ignored in pio1

Shoud i update the version of NetCDF on my machine (now NetCDF-4.4.1 and netCDF-F is 4.4.4) and retry it?
Look forward to any useful information.

Best regards,

Wilma W.

This is telling you that you have PIO version 1 installed on your machine. You will need PIO version 2. Moving this to porting.

 

[jedwards]

PIO version 1 is fine for cesm2.1 but you should update to the latest 2.1.5. The NetCDF: Variable not found are normal and
expected messages as the model searches for a configuration. These are not associated with problems in inputdata. I don't
see any error in the cesm.log file you have provided, but it is trying to write not read a file - are you sure that you have adequate disk space?