Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Not exact same results for two same running

J

Jack_ecology

Jack
New Member
Hi, I am quite new in running CESM2. I tried a hybrid startup running using B1850 (g19_g17) from a previous condition (e.g., 0001-03-01). I run 5 days and then it outputs a restart directory (0001-03-06-00000/) and I back up this directory as (0001-03-06-00000_bak). However, when I rerun this simulation (does not change anything) and got a new restart directory (0001-03-06-00000/). But when I compare the same files (.nc) between two directories (0001-03-06-00000/ and 0001-03-06-00000_bak/) using 'diff'. It shows these two binary files differ.

I would like to know if there are some wrong with my simulation? Or there is possible to have very small differences in output by two exact simulations.
 
erik

erik

Erik Kluzek
CSEG and Liaisons
Staff member
Using diff isn't adequate for making sure two NetCDF files have identical results. There are fields in the file that have to do with the date and time stamp that would be different even though the fields are identical. A better way to compare them is to use a tool to open up the NetCDF files and compare them directly (Perl, Python, NCL, ferret there are lots of tools to do this.

Two other simple tools are the NetCDF Operators (NCO) ncdiff command which will compare them directly. And under the tools directory of cime there is a tool called "cprnc" that will compare two NetCDF files. So those are two nice options as well.
 
J

Jack_ecology

Jack
New Member
erik said:
Using diff isn't adequate for making sure two NetCDF files have identical results. There are fields in the file that have to do with the date and time stamp that would be different even though the fields are identical. A better way to compare them is to use a tool to open up the NetCDF files and compare them directly (Perl, Python, NCL, ferret there are lots of tools to do this.

Two other simple tools are the NetCDF Operators (NCO) ncdiff command which will compare them directly. And under the tools directory of cime there is a tool called "cprnc" that will compare two NetCDF files. So those are two nice options as well.
Click to expand...

Dear Erik,

Thank you for the suggestions. I just tried ncdiff method, and this is done through one file subtract the other file and obtain a difference file. I find the obtained difference file does include some values not equal to zero. Does that mean these two restart files are not the exact same? I know this question seems to be weird, but I do hope to confirm what I have done is correct. Thank you so much!
 
erik

erik

Erik Kluzek
CSEG and Liaisons
Staff member
It depends on what fields are different. And actually we don't usually compare restart files, but compare history files to make sure the model output is the same. Although the restart files should be the same other than fields that have to do with the date/time and/or specifics of the case.

There is some documentation on using cprnc to compare history files.


Which specific files are you comparing? Something simpler to start with is that in the cpl.log files it writes out some global sums for the fields that are transferred in the coupler. You can compare those fields to see if the coupler is getting the same values. The next step is to compare the coupler history files, and then the specific component history files.

In our testing of the model we do a LOT of testing that compares history files, and we do ensure it comes back the way we expect. Note, that depending on your compset, if you change the number of processors the answers CAN be different. It doesn't sound like you did that, but you should be ready for that.
 
J

Jack_ecology

Jack
New Member
erik said:
It depends on what fields are different. And actually we don't usually compare restart files, but compare history files to make sure the model output is the same. Although the restart files should be the same other than fields that have to do with the date/time and/or specifics of the case.

There is some documentation on using cprnc to compare history files.


Which specific files are you comparing? Something simpler to start with is that in the cpl.log files it writes out some global sums for the fields that are transferred in the coupler. You can compare those fields to see if the coupler is getting the same values. The next step is to compare the coupler history files, and then the specific component history files.

In our testing of the model we do a LOT of testing that compares history files, and we do ensure it comes back the way we expect. Note, that depending on your compset, if you change the number of processors the answers CAN be different. It doesn't sound like you did that, but you should be ready for that.
Click to expand...

Thank you so much Erik, I compared restart files, but I will follow your suggestions to look at the historical and coupler files.
 
You must log in or register to reply here.
Top