case.submit error with single point simulation

[xgao304]

I am following the tutorial (https://www.cgd.ucar.edu/events/2019/ctsm/files/practical4-wieder.pdf) to generate the single point case run. But I got some errors after case.submit. Here are the steps I took:

1.
create_newcase --case 2000CLM50Sp --compset 2000_DATM%GSWP3v1_CLM50%SP_SICE_SOCN_SROF_SGLC_SWAV --res f09_g17 --machine svante --compiler intel --run-unsupported
2. use tools "singlept" to generate the point-based domain, surface, and atmospheric forcing data
3. change the settings in env_mach_pes.xml
4. change the settings in env_run.xml
5. Specify the "fsurdat" in "user_nl_clm
6. edit the datm.streams files to the generated pointed-based atmospheric forcing in step 2.
7. case.setup
8. case.build
9. case.submit

I am attaching relevant settings (caseStatus) and cesm log files.

Any help is appreciated.

Xiang

 

[oleson]

ERROR: (shr_dmodel_readstrm) ERROR in data sizes
(shr_dmodel_readstrm) ERROR in data sizes 1 1 248 259200

This corresponds to nx, ny, nz, and gsize.
The error means that nx*ny is not equal to 259200.
nz=248 is the time dimension.
gsize=259200 likely indicates nx=720 * ny=360, so this seems to be the global GSWP3 grid or some 1/2 degree atmospheric forcing grid.
Maybe you are using the global domain file in combination with the extracted single-point forcing?
Look at your datm.streams.* files and your datm file in your CaseDocs directory.
Or post them and I'll see if I can spot anything.

 

[xgao304]

I have attached one example of user_datm.streams.txt.**. All precip, solar, TPQW stream files have been updated. But
presaero.clim_2000 and topo.observed have not been changed.

Another piece of information: I have used domain file,

domain.lnd.fv0.9x1.25_gx1v7_287.8_42.5.151020.nc

Then when I do "case.submit", it says

missing domain file "

domain.lnd.360x720_gswp3.0v1_287.8_42.5.c170606.nc

I copied it to the designated directory. Then the error is shown in my post.

Thanks,

Xiang

 

[xgao304]

One more question: Once you edit user_datm_streams.*, do you have to copy these "user_datm*" files to CaseDocs/datm_streams*?
In other words, do the contents of user_datm_streams.* and CaseDocs/datm_streams* have to be the same, with the only difference in
file names?

 

[xgao304]

For my previous question, it does need to run "preview_namelists" to update the stream files in the CaseDocs. I am testing it now.
Thanks.

 

[sandeepc]

I am running a similar case for single-point simulation and I got a different error.

ERROR: (shr_stream_getCalendar) ERROR: nf90_inq_varid

I have attached the relevant files.

Could you please resolve it?

 

[xgao304]

I still got the similar datasize error (see the new cesm log file). I made sure that all the user_datm.streams files have been updated and
same as the ones in CaseDocs (with different file names). What should I do? Thanks. Xiang

 

[xgao304]

I changed the domain file for forcing, now I got a different error for the function

ERROR: (shr_tInterp_getCosz) ERROR: lon lat cosz sizes disagree

I also attached the new cesm log file.

 

[oleson]

I could spend more time guessing but I'm going to need more information from you to help troubleshoot this.
Please attached all of your logs and the namelists (lnd_in, datm_in, datm.streams.*, etc) from your CaseDocs or run directories, from your most recent runs that show these errors. As well as output from ncdump -h on your surface dataset, your land and datm domain files and at least one of each of the three types of monthly datm forcing files (i.e., Solar, Precip, TPQWL). Thanks.

 

[xgao304]

Here are all the relevant files. Thanks.

 

[xgao304]

three log files

 

[xgao304]

*_in files

 

[oleson]

Thanks for all of that information. Unfortunately, I've reviewed those files and I don't see anything pbviously wrong.
The error is here (lines 386-388 in cime/src/share/streams/shr_tInterp_mod.F90):

lsize = size(lonr)
if (lsize < 1 .or. size(latr) /= lsize .or. size(cosz) /= lsize) then
call shr_sys_abort(subname//' ERROR: lon lat cosz sizes disagree')
endif

So one of those conditions in the if statement must be triggering. You could try adding code to print out the values of lsize, size(latr) and size(cosz).
Also, check to make sure the solar forcing file does indeed have dimensions of lat=1, lon=1. In particular, this file which is read in first:

/net/fs05/d1/xgao/cesm2.1.3/cases/test_sim/single_point/datmdata/clmforc.GSWP3.c2011.0.5x0.5.Solr.287.8_42.5.1901-01.nc

The only other thing I can think of is to run through the tutorial myself, but we know that it worked for those students who were in the tutorial.

 

[xgao304]

Thanks. The solar file looks fine. I will try to add the print statement. My question is : even if we know which condition is triggered,
what else could we do to resolve the issue? Thanks.

 

[oleson]

One other thought, did you execute these commands?

./xmlchange --file env_mach_pes.xml --id COST_PES --val 36
./xmlchange --file env_mach_pes.xml --id TOTALPES --val 1
./xmlchange --file env_mach_pes.xml --id NTASKS --val 1
./xmlchange --file env_mach_pes.xml --id NTASKS_PER_INST --val 1
./xmlchange --file env_mach_pes.xml --id ROOTPE --val 0
./xmlchange MPILIB=mpi-serial

Your drv_in seems to indicate you are 32 tasks when you should be using 1...

Can you attach your env_mach_pes.xml file?

 

[xgao304]

I did, but I adjusted based on our own system.

 

[xgao304]

When I did a global case successfully using 4 computer nodes and each has 32 cores:
I have the following, so I adjust it for one node for single-point case. Not sure if my adjustment is correct though.

 

[xgao304]

our MPILIB=openmpi in the configuration file, should I change it to "mpi-serial"?

 

[oleson]

For a single-point simulation, you can only make use of one core on one node.
Yes, I think I would change it to mpi-serial.
Configurations for machines other than our own machine are a bit out of my area of expertise, but I would try that.
I have to leave now but I'll check in tomorrow for any updates.

 

[xgao304]

I changed "TOTALPES", "NTASKS", and "NTASKS_PER_INST" to value "1", but keep MPILIB = openmpi.
It works now. Thanks very much for your help.