Problem for nonsense WACCMX output after non-error build and run

[Tianliang]

What version of the code are you using?
CESM2.2.2 and ESMF 8.7.0

[yang2718@tylogin01 debug_module_original]$ echo $ESMFMKFILE
/work/home/yang2718/esmf/esmf-8.7.0/lib/libO/Linux.intel.64.intelmpi.default/esmf.mk

I create the case through:

./create_newcase --case ~/work/CESM/debug_module --compset FXHIST --res f09_f09_mg17 --mach tylogin02 --run-unsupported

Have you made any changes to files in the source tree?
For xml file changes:
in ~/.cime/, we have
config_batch.xml

<?xml version="1.0"?>
<config_batch version="2.0">
<batch_system type="slurm" MACH="tylogin02">
<batch_submit>sbatch</batch_submit>
<submit_args>
<arg flag="--time" name="$JOB_WALLCLOCK_TIME"/>
<arg flag="--partition" name="$JOB_QUEUE"/>
<arg flag="--account" name="$PROJECT"/>
</submit_args>
<queues>
<queue default="true" walltimemax="333:00:00" nodemin="1">tyhcnormal</queue>
</queues>
</batch_system>
</config_batch>

config_compilers

<?xml version="1.0"?>
<config_compilers version="2.0">
<compiler MACH="tylogin02" COMPILER="intel">
<CFLAGS>
<base>-qno-opt-dynamic-align -fp-model precise -std=gnu99</base>
<append compile_threaded="TRUE">-qopenmp</append>
<append DEBUG="FALSE">-O2 -debug minimal</append>
<append DEBUG="TRUE">-O0 -g</append>
</CFLAGS>

<FFLAGS>
<base>-qno-opt-dynamic-align -convert big_endian -assume byterecl -ftz -traceback -assume realloc_lhs -fp-model source</base>
<append compile_threaded="TRUE">-qopenmp</append>
<append DEBUG="FALSE">-O2 -debug minimal</append>
<append DEBUG="TRUE">-O0 -g -check uninit -check bounds -check pointers -fpe0 -check noarg_temp_created</append>
</FFLAGS>

<FIXEDFLAGS><base>-fixed -132</base></FIXEDFLAGS>
<FREEFLAGS><base>-free</base></FREEFLAGS>
<LDFLAGS><append compile_threaded="TRUE">-qopenmp</append></LDFLAGS>
<MPIFC>mpiifort</MPIFC>
<MPICC>mpiicc</MPICC>
<MPICXX>mpiicxx</MPICXX>
<SFC>ifort</SFC>
<SCC>icc</SCC>
<SCXX>icpc</SCXX>
<SLIBS>
<append MPILIB="intelmpi">-mkl=cluster</append>
<append MPILIB="mpi-serial">-mkl</append>
</SLIBS>

<SUPPORTS_CXX>TRUE</SUPPORTS_CXX>
</compiler>
</config_compilers>

and config_machines

<?xml version="1.0"?>
<config_machines version="2.0">
<machine MACH="tylogin02">
<DESC>Local PBS cluster with Intel 2021.3 + IntelMPI 2021.3</DESC>

<NODENAME_REGEX>ty.*</NODENAME_REGEX>
<OS>LINUX</OS>

<COMPILERS>intel</COMPILERS>
<MPILIBS>intelmpi, mpi-serial</MPILIBS>


<PROJECT>ac38xhl3b1</PROJECT>


<CIME_OUTPUT_ROOT>/work/home/yang2718/cesm/cases</CIME_OUTPUT_ROOT>
<DIN_LOC_ROOT>/work/home/yang2718/cesm/inputdata</DIN_LOC_ROOT>

<DIN_LOC_ROOT_CLMFORC>/work/home/yang2718/cesm/inputdata</DIN_LOC_ROOT_CLMFORC>
<DOUT_S_ROOT>$CIME_OUTPUT_ROOT/archive/$CASE</DOUT_S_ROOT>

<!-- gnu-compatible make -->
<GMAKE>gmake</GMAKE>
<GMAKE_J>8</GMAKE_J>


<BATCH_SYSTEM>slurm</BATCH_SYSTEM>
<ALLOCATE_SPARE_NODES>FALSE</ALLOCATE_SPARE_NODES>
<SUPPORTED_BY>yang.tianliang@nagoya-u.jp</SUPPORTED_BY>
<MAX_TASKS_PER_NODE>64</MAX_TASKS_PER_NODE>
<MAX_MPITASKS_PER_NODE>64</MAX_MPITASKS_PER_NODE>
<PROJECT_REQUIRED>FALSE</PROJECT_REQUIRED>


<mpirun mpilib="intelmpi">
<executable>mpirun</executable>
<arguments>
<arg name="num_tasks">-n {{ total_tasks }}</arg>

</arguments>
</mpirun>


<module_system type="module" allow_error="true">
<init_path lang="sh">/usr/share/Modules/init/sh</init_path>
<init_path lang="python">/usr/share/Modules/init/python.py</init_path>

<cmd_path lang="sh">module</cmd_path>
<cmd_path lang="python">module</cmd_path>


<modules>
<command name="purge"/>

<command name="load">python/3.8.10</command>

<command name="load">compiler/intel/2021.3.0</command>
<command name="load">mpi/intelmpi/2021.3.0</command>
<command name="load">mathlib/zlib/1.3-intel-2021</command>
<command name="load">mathlib/hdf5/1.12.2-intel-21</command>
<command name="load">mathlib/netcdf/c4.9.2_f4.6.1-intel21</command>
</modules>
</module_system>

<environment_variables>
<env name="NETCDF_C_PATH">/public/software/mathlib/netcdf/c4.9.2_f4.6.1_intel</env>
<env name="NETCDF_FORTRAN_PATH">/public/software/mathlib/netcdf/c4.9.2_f4.6.1_intel</env>

<env name="CC">/public/software/compiler/intel-compiler/2021.3.0/bin/intel64/icc</env>
<env name="CXX">/public/software/compiler/intel-compiler/2021.3.0/bin/intel64/icpc</env>
<env name="FC">/public/software/compiler/intel-compiler/2021.3.0/bin/intel64/ifort</env>

</environment_variables>
</machine>
</config_machines>

For input changes, we have user_nl_cam

ncdata = '/work/home/yang2718/FromJoe/f.e22.FXHIST.f09_f09_mg17.001.cam.i.2024-10-01-00000_c250315.nc'

&chem_inparm
ext_frc_type = 'CYCLICAL'
ext_frc_cycle_yr = 2000
srf_emis_type = 'CYCLICAL'
srf_emis_cycle_yr = 2000
/
&chem_surfvals_nl
flbc_type = 'CYCLICAL'
flbc_cycle_yr = 2000
/


solar_data_type = 'FIXED'
solar_data_ymd = 20021001


solar_irrad_data_file = '/work/home/yang2718/cesm/inputdata/atm/cam/solar/spectral_irradiance_Lean_1950-2014_daily_GOME-Mg_Leap_c150623.nc'
solar_parms_data_file = '/work/home/yang2718/cesm/inputdata/atm/waccm/phot/wa_avg_c20170519.nc'
epp_spe_filepath = 'NONE'
epp_all_filepath = 'NONE'


dtime = 300
empty_htapes = .true.
nhtfrq = -24, 1
mfilt = 1, 288
avgflag_pertape = 'A', 'I'
fincl1 = 'T','U','V','PS'
fincl2 = 'T','U','V','PS'

where we also tried the cimp6 and other default for the solar input.


Describe every step you took leading up to the problem:

./create_newcase --case ~/work/CESM/debug_module_original --compset FXHIST --res f09_f09_mg17 --mach tylogin02 --run-unsupported
cd ~/work/CESM/debug_module_original
./xmlchange NTASKS=64,NTHRDS=1
./xmlchange RUN_STARTDATE=2024-10-01
./xmlchange STOP_OPTION=nsteps
./xmlchange STOP_N=1

./case.build --skip-provenance-check

./case.setup

./case.submit

PS: The run option *nsteps* is just for debug and we have the same problem for

./xmlchange STOP_OPTION=ndays
./xmlchange STOP_N=1

Describe your problem or question:
I did not occur any problem during build or run but the output data is not reasonable at all. See pics and part of the list

For more information, I have also add eclipse module into SourceMods/src.cam (case name: debug_module_solar_bug and add ECLIPSE_MASK for the output file), see link

Index of /pub/joemci/WACCMX_ICFiles

where you can also find the initial file that I set in user_nl_cam. The eclipse factor shows 0/1.875 longtiutal flashing while the normal output should be all 1.