Scheduled Downtime
On Tuesday 24 October 2023 @ 5pm MT the forums will be in read only mode in preparation for the downtime. On Wednesday 25 October 2023 @ 5am MT, this website will be down for maintenance and expected to return online later in the morning.
Normal Operations
The forums are back online with normal operations. If you notice any issues or errors related to the forums, please reach out to help@ucar.edu

Comparing runs across different computers/laptops

simondriscoll

simondriscoll

Simon Driscoll
Member
Hi all, apologies if this is a stupid question. A while ago running simulations for another project, I encountered the issue that certain simulations may not be comparable if run on different machines. Is this the case for Icepack or is it not? e.g. can I run it on one mac and directly compare the output in a scientifically rigorous way to another test on another mac (slightly older and newer macs, say), and assume that these are equivalent to two different tests run on the same machine? Is there nuance to this, and if so how important is it? Thank you.
 
E

eclare@lanl_gov

Member
Hi Simon,
You shouldn't expect exactly the same (bit for bit) answers on different machines unless they are using exactly the same hardware (chips etc) and also exactly the same versions of software (compilers), and even then they might not be the same. They should be equivalent in a "climate" sense but the answers could be quite a bit different, e.g. thickness differences of several cm, just from the variability inherent in the nonlinear system. If you want to compare solutions that are different but you suspect should be climatically equivalent, you can use the Quality Control (QC) testing infrastructure available in the CICE repository. Instructions are in the docs.
Best,
e
 
simondriscoll

simondriscoll

Simon Driscoll
Member
Hi Elizabeth,

many thanks for your reply. This is very useful. By hardware you mean inherent to that computer (e.g. Apple M1 or Intel chips), i.e. two macs must have the same chips, and by software or compilers do you mean the choice in calling icepack itself?

e.g.
a) icepack.setup -c mycase -m macos -e conda
b) icepack.setup -c mycase -m banting -e intel

(as provided in: Icepack/configuration/scripts/machines amongst many others)

"--env, -e ENVIRONMENT1,ENVIRONMENT2,ENVIRONMENT3
specifies the compilation environment associated with the machine. This should be consistent with the name defined in the Macros and env files in configurations/scripts/machines. Each machine can have multiple supported environments including support for different compilers, different compiler versions, different mpi libraries, or other system settigs. "


I.e. for runs to be comparable across two laptops I would need two (e.g.) macs with both M1 chips or the same intel chips, and a) and b) examples must be using the same compilation environment, e.g.

a) = b) = icepack.setup -c mycase -m macos -e conda

Then I would be able to directly compare and otherwise not?

Thank you once again for your super quick reply! (and happy holidays to you also!)
 
E

eclare@lanl_gov

Member
Even different versions of the same compiler can produce different answers. The setup has to be absolutely identical to expect the same answers. (It's possible to get the same answers with different setups but highly unlikely.) In your example above, the conda environment would have to be identical -- same versions of compilers, etc. Newer conda setups might be different from older ones. Likewise for other software configuration choices like MPI and the flags used for any given compiler (e.g. running with -g always changes the answers compared with running without it, due largely to differences in the initializations).
 
You must log in or register to reply here.
Top