Initialize the CISO tracers within MARBL correctly with new initial conditions file

[shiannder]

Hi,

I am trying to initialize a CESM V2.1.3 compset G1850ECOIAF run with CISO tracers in MARBL using a modified initial conditions nc file that combines ecosys_jan_IC_gx3v7_20180308.nc with 14 CISO tracers from a 9963-9982 simulated yr run that is a 20-yr average output. The CISO tracers are diat13C, diat14C, diaz13C, diaz14C, spCa13CO3, spCa14CO3, zootot13C, zootot14C, sp13C, sp14C, DO14Ctot, DO13Ctot, DI13C, and DI14C. The goal is to not have to simulate carbon isotopes out to 10,000 yrs and just simulate with a CISO 10,000 yr spun-up run. Below are my 2 runs error results with the new initial condition file:

My jsl.091 model initialized, but gets DNS after a minute with the error below from cesm.log.21999.240622-110634:

• (Task 6, block 1) MARBL ERROR (marbl_diagnostics_mod:store_diagnostics_carbon_fluxes): abs(Jint_Ctot)= 0.957E+038 exceeds Jint_Ctot_thres= 0.317E-011
• (Task 15, block 1) MARBL ERROR (marbl_ciso_diagnostics_mod:store_diagnostics_ciso_interior): abs(CISO_Jint_13Ctot)= 0.283E-003 exceeds CISO_Jint_13Ctot_thres= 0.317E-011

My jsl.092 model raised the Jint_Ctot_thres and DISO_Jint_14Ctot_thres higher by modifying the code in marbl_settings_mod.F90

• Jint_Ctot_thres = 1.0e9_r8 * mpercm**2 * yps * Jint_Ctot_thres_molpm2pyr * 1e60
• CISO_Jint_14Ctot_thres = R14C_std * Jint_Ctot_thres * 1e10

However, the mode failed to initialized and gave Error: Real constant overflows its kind at (1)

Which I assume the overflow is from the 1e60.

To make the file I grabbed ecosys_jan_IC_gx3v7_20180308.nc and duplicated diatC, diazC, spCaCO3, zootot, spC, DOCtot, DIC and renamed to 13C and 14C equivalents. I copied the CISO variables from jsl.012.pop.h.9963_9983.nc into the new initial condition file called jsl012_ecosys_jan_IC_gx3v7_20180308_ciso.nc in /mnt/lustre/letscher/jsl1063/initcond/ciso9962y

I also removed time dimension and the z_t_150 depth dimension, which I converted to the z_t depth with 0 values for all the depths below depth 15-60. This dimension change is from my previous post that I did not get to till now, but refers to removing NaNs at KMT+1. Note, the error is same as my last error message from the previous post with jsl.041.

The updated model runs now have changes to DOMr_reminR_photo = 248 yr, modification to eps_dic_g equation, and iron flux modifications, which have all worked previously. I have also successfully changed the Jint_Ctot_thres within a few orders of magnitude.


An ideas on how I can fix the threshold MARBL error to initialize with this modified CISO initial conditions file?

 

[mlevy]

@shiannder -- I was on PTO the past couple of weeks, but will look into this... at first glance it sounds like the same "NaNs in the IC file" issue from your previous post, although I do see that you tried to use 0s instead of fill values below the sea floor. Maybe something weird in adjusting the vertical coordinate? I'll let you know what I find.

 

[mlevy]

Looking at the top level of DIC vs DI13C, it looks like there is a shift in the land mask in the file (DIC on the left):


So there are some columns that POP thinks are active ocean, and when it reads in DIC it gets a full depth field, but when it reads DI13C it gets a column of fill values. How did you do the following step?

I copied the CISO variables from jsl.012.pop.h.9963_9983.nc into the new initial condition file

 

[shiannder]

Hi Mike,

Thank you for checking my nc file. I guess I did not realize I had shifted the data since I have been plotting with NASA Panoply that gets the lat and lon automatically. However, the DI13C dimensions should just be a copy of the DIC dimensions. I will just plot the variables in python or R to check the dimensions from now on.

I used a AI generated matlab code based on the scripts below for copying the 14 ciso variables.

1. duplicate a variable and rename it. Example with duplicating diatC and renaming it diat13C. Need to change the variable names for each

• diatC-diat13C and diat14C
• diazC-diaz13C and diaz14C
• spCaCO3-spCa13CO3 and spCa14CO3
• zooC-zootot13C and zootot14C
• spC-sp13C and sp14C
• DOCtot-DO13Ctot and DO14Ctot
• DIC-DI13C and DI14C

% Open the netCDF file
filename = 'ecosys_jan_IC_gx3v7_20180308.nc';
ncid = netcdf.open(filename, 'NC_WRITE');
% Re-enter definition mode
netcdf.reDef(ncid);
% Get the ID of the variable 'diatC'
varid_diatC = netcdf.inqVarID(ncid, 'diatC');
% Get the attributes of 'diatC'
[varname, xtype, dimids, numatts] = netcdf.inqVar(ncid, varid_diatC);
% Create the new variable 'diat13C' with the same properties as 'diatC'
varid_diat13C = netcdf.defVar(ncid, 'diat13C', xtype, dimids);
% Copy the attributes from 'diatC' to 'diat13C'
for i = 0:numatts-1
attname = netcdf.inqAttName(ncid, varid_diatC, i);
attvalue = netcdf.getAtt(ncid, varid_diatC, attname);
netcdf.putAtt(ncid, varid_diat13C, attname, attvalue);
end
% End the definition mode
netcdf.endDef(ncid);
% Get the data from 'diatC'
data_diatC = netcdf.getVar(ncid, varid_diatC);
% Write the data to 'diat13C'
netcdf.putVar(ncid, varid_diat13C, data_diatC);
% Close the netCDF file
netcdf.close(ncid);

2. Transfer the variable diat13C from the file "jsl.012.pop.h.9963_9982.nc" to "ecosys_jan_IC_gx3v7_20180308.nc"

The goal is to overwrite the diat13C variable in the ecosys_jan_IC_gx3v7_20180308.nc file with the data from jsl.012.pop.h.9963_9982.nc, and to retain the original values where the new data is not available.

Need to change the variable name for each diat13C, diat14C, diaz13C, diaz14C, spCa13CO3, spCa14CO3, zootot13C, zootot14C, sp13C, sp14C, DO14Ctot, DO13Ctot, DI13C, DI14C

sourceFile = 'jsl.012.pop.h.9963_9982.nc';
destinationFile = 'ecosys_jan_IC_gx3v7_20180308.nc';
% Open the source netCDF file
srcId = netcdf.open(sourceFile, 'NC_NOWRITE');
% Get the ID of the variable 'diat13C' from the source file
varid_src = netcdf.inqVarID(srcId, 'diat13C');
% Read the data for 'diat13C' from the source file
data_diat13C_src = netcdf.getVar(srcId, varid_src);
% Close the source netCDF file
netcdf.close(srcId);
% Open the destination netCDF file
destId = netcdf.open(destinationFile, 'NC_WRITE');
% Get the ID of the variable 'diat13C' from the destination file
varid_dest = netcdf.inqVarID(destId, 'diat13C');
% Read the data for 'diat13C' from the destination file
data_diat13C_dest = netcdf.getVar(destId, varid_dest);
% Replace the non-NaN values from the source to the destination data
% Assuming NaN is used to represent missing data in the source file
mask = ~isnan(data_diat13C_src);
data_diat13C_dest(mask) = data_diat13C_src(mask);
% Write the updated data back to the 'diat13C' variable in the destination file
netcdf.putVar(destId, varid_dest, data_diat13C_dest);
% Close the destination netCDF file
netcdf.close(destId);

3. rename "ecosys_jan_IC_gx3v7_20180308.nc" to jsl012_ecosys_jan_IC_gx3v7_20180308_ciso.nc

 

[shiannder]

Hi,

I updated my new initial condition file (jsl012_ecosys_jan_IC_gx3v7_20180308_ciso.nc) to fix the shifted carbon isotope variable data.

14 CISO variables are called in from "user_nl_pop.ciso3" in /mnt/lustre/letscher/shared/SourceCode_gx3v7

init_ecosys_init_file = '/mnt/lustre/letscher/jsl1063/initcond/ciso9963_9982/jsl012_ecosys_jan_IC_gx3v7_20180308_ciso.nc'

The model jsl.094 ran through one 62-yr submission, but when it went to resubmit for another 62-yrs it gave DNS. The cesm.log.22273.240710-154235 file gave the error output below and attached. It says "variable not found" and "Error in getting varid for netCDF field PO4_CUR" several times.



I checked the nc and rpointer restart files in /mnt/lustre/letscher/jsl1063/archive/jsl.094/rest/0063-01-01-00000. The nc files have PO4_CUR and the rpointers have the right nc file called.


The ocn.log.22273.240710-154235 mentions PO4_CUR is being read from the new initial condition files too. Although it probably should be read from the "jsl.094.pop.r.0063-01-01-00000.nc" file

(passive_tracer_tools:rest_read_tracer_block) reading PO4_CUR from /mnt/lustre/letscher/jsl1063/initcond/ciso9963_9982/jsl012_ecosys_jan_IC_gx3v7_20180308_ciso.nc

I also ran new models (jsl.097) with modified ecosys_diagnostics.ciso.medium.all where all low_average was changed to medium_average for annual output, but it gave the same resubmitting error above.

Do you have any insights in how I could fix this to get resubmitting to work?

Thank you,

James

 

[mlevy]

This is a very common issue in POP -- if you set init_ecosys_init_file in user_nl_pop, that file is used instead of the restart files... but POP is expecting to read a restart file, so it looks for the ${TRACER}_CUR and ${TRACER}_OLD fields instead of just ${TRACER}. The best solution would have been to change the code 15 years ago to ignore init_ecosys_init_file if init_ecosys_option = 'ccsm_restart' (we do want users to be able to specify "use a restart file for physics, but initialize the MARBL tracers from an IC file" in situtations where the MARBL tracers are not present in the restart file), but that feature wasn't implemented prior to the decision to switch to MOM6 and halt POP development... so you should just comment out the init_ecosys_init_file line in the user_nl_pop file in your case directory.