Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Regional grid run computing time issue

adrienD

Adrien Damseaux
Member
Hello everyone,

I recently had to reinstall CTSM on a new version of the supercomputer I use (Levante from DKRZ). I took the opportunity to reinstall a more up-to-date version of CTSM (from dev023 to dev086).

So far, with your help, I have managed to build and run a default global grid with a much improved computation time compared to my previous setup.

However, when I try to reuse a regional grid that I used in my previous setup, the computation time is quite slow. I only managed to run 31 days with 594 nodes/hr. I was able to run a full year or more with this calculation in my previous setup.

Due to the nature of my problem, I cannot work on logs issues and I wonder if you have any idea what I could do.

I have attached my case file and the logs from a run it produces.

Thanks!
 

Attachments

  • 57_DOM02_028.log.zip
    31.6 KB · Views: 3
  • 57_DOM02_028.zip
    190.6 KB · Views: 1

jedwards

CSEG and Liaisons
Staff member
I see that you are setting PNETCDF_PATH, but the model is using netcdf not pnetcdf.
This is probably because you are only using 1 node for your run and so only have 1
iotask. I don't think that this performance is entirely due to io performance, but try changing
PIO_REARRANGER=1.
 

adrienD

Adrien Damseaux
Member
I see that you are setting PNETCDF_PATH, but the model is using netcdf not pnetcdf.
This is probably because you are only using 1 node for your run and so only have 1
iotask. I don't think that this performance is entirely due to io performance, but try changing
PIO_REARRANGER=1.
Thanks @jedwards. Sorry for my late answer, the forum was down for a few days and I didn't have time to see your answer.

I have tried to run without PNETCDF LND PIO_REARRANGER=1, and it's now working like before! Maybe better, need to do more tests.

Thank you so much
 
Top