Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Material for beginner to learn CESM

rajkmsaini

rajkmsaini

Dr. Raj Saini
Member
Hello CESM users/Team,

I'm a beginner in CESM but I am expert in CFD analysis. Some of my professional reason I want to learn CESM from scratch. I want your recommendations so I can learn quickly.

--- please suggest good material (like programming guide/details user guide)
---good well explained tutorial ( available tutorial are not well explained mostly)
--- the source of code


Now, I have some questions which are the following:

---- Can we use CESM on our 8 core /32GB ram/ 4.2Ghz local machine? If yes, please share some source links.
---- Can I use on AWS (cloud computing)? if yes, Please share the source of this so I can get quick help.

Thank you in advance.

Best regards,
Dr. Raj
IIT Bombay/ IIT Delhi (INDIA)
 
rajkmsaini

rajkmsaini

Dr. Raj Saini
Member
Still not get the answer.

Now, I have some questions which are the following:

---- Can we use CESM on our 8 core /32GB ram/ 4.2Ghz local machine? If yes, please share some source links.
---- Can I use on AWS (cloud computing)? if yes, Please share the source of this so I can get quick help.

Thank you in advance.
 
sacks

sacks

Bill Sacks
CSEG and Liaisons
Staff member
Regarding running on 8 cores, you may want to see: https://bb.cgd.ucar.edu/cesm/threads/viability-of-running-cesm-on-40-cores.4997 . Short answer: with only 8 cores and 32 GB ram, you are unlikely to be able to run any scientifically publishable configurations of the full, coupled model. There are various simpler model configurations that could be feasible: CESM Models | Simpler Models

I will ping someone else for information on running on the cloud. This is possible, but I'm not sure how much documentation is available yet.
 
D

dobbins

Brian Dobbins
CSEG and Liaisons
Staff member
Hi Dr. Raj,

I'll echo Bill's comments regarding the limitations of a 32GB machine - I've run a B1850 (fully coupled) configuration at f19_g17 resolution on 16 cores, and it takes ~67GB of RAM, so you'll likely be restricted to very low resolution (non-verified) or simpler model modes on 32GB of RAM. The number of processors is less important than the memory in terms of capability, but important in terms of model performance, obviously.

In terms of cloud computing, we do in fact have a 'beta' version of CESM2 on AWS - currently the images are only available in the US-West-1 and US-East-1 regions, but once we move from beta to public release, we expect we'll make more regions available. That said, pricing for compute is typically best in these regions, especially with spot prices, but if you have a specific request for a closer region, let me know. There's still work to be done on load-balancing across cloud systems, so for the moment most configurations still require manual configuration of a reasonable number of tasks, but we've run the above B1850 f19_g17 on 1-4 nodes, and we've done 4 node runs for the higher atmosphere resolution f09_g17 runs. I expect this will be publicly available in a few weeks, but if you'd like to get started sooner, send me an email and an AWS account number and I'll add you to the beta testing image.

Finally, if you're happy running simpler / very low resolution models, and want to stick to your local system, we are also working on having a 'containerized' (via Docker) version of CESM, eliminating the work involved in porting it to a new system (except, again, for load-balancing). This, too, is still in its early stages, but I'm happy to provide you a download link, and we'll soon have a space on Dockerhub too.

In short, your 32GB system will be limited in what it can run, but if you want to run there you can download and port the code, or I can give you access to the container. If you want to use the cloud, I can give you access to the auto-scaling pre-built image, just send me an email. Both of these are works in progress, but I expect we'll have public releases within the coming few months, too.

Cheers,
- Brian
 
rajkmsaini

rajkmsaini

Dr. Raj Saini
Member
sacks said:
Regarding running on 8 cores, you may want to see: https://bb.cgd.ucar.edu/cesm/threads/viability-of-running-cesm-on-40-cores.4997 . Short answer: with only 8 cores and 32 GB ram, you are unlikely to be able to run any scientifically publishable configurations of the full, coupled model. There are various simpler model configurations that could be feasible: CESM Models | Simpler Models

I will ping someone else for information on running on the cloud. This is possible, but I'm not sure how much documentation is available yet.
Click to expand...


Thanks for your suggestions.
 
rajkmsaini

rajkmsaini

Dr. Raj Saini
Member
dobbins said:
Hi Dr. Raj,

I'll echo Bill's comments regarding the limitations of a 32GB machine - I've run a B1850 (fully coupled) configuration at f19_g17 resolution on 16 cores, and it takes ~67GB of RAM, so you'll likely be restricted to very low resolution (non-verified) or simpler model modes on 32GB of RAM. The number of processors is less important than the memory in terms of capability, but important in terms of model performance, obviously.

In terms of cloud computing, we do in fact have a 'beta' version of CESM2 on AWS - currently the images are only available in the US-West-1 and US-East-1 regions, but once we move from beta to public release, we expect we'll make more regions available. That said, pricing for compute is typically best in these regions, especially with spot prices, but if you have a specific request for a closer region, let me know. There's still work to be done on load-balancing across cloud systems, so for the moment most configurations still require manual configuration of a reasonable number of tasks, but we've run the above B1850 f19_g17 on 1-4 nodes, and we've done 4 node runs for the higher atmosphere resolution f09_g17 runs. I expect this will be publicly available in a few weeks, but if you'd like to get started sooner, send me an email and an AWS account number and I'll add you to the beta testing image.

Finally, if you're happy running simpler / very low resolution models, and want to stick to your local system, we are also working on having a 'containerized' (via Docker) version of CESM, eliminating the work involved in porting it to a new system (except, again, for load-balancing). This, too, is still in its early stages, but I'm happy to provide you a download link, and we'll soon have a space on Dockerhub too.

In short, your 32GB system will be limited in what it can run, but if you want to run there you can download and port the code, or I can give you access to the container. If you want to use the cloud, I can give you access to the auto-scaling pre-built image, just send me an email. Both of these are works in progress, but I expect we'll have public releases within the coming few months, too.

Cheers,
- Brian
Click to expand...


Thanks for your suggestions.
 
You must log in or register to reply here.
Top