box_rearrange::compute_dest:: ERROR

[Mikasa]

Hello, I am running compset B1PCTcmip6 with CESM2.1.3. But an error was reported :

box_rearrange::compute_dest:: ERROR: no destination found for compdof=
-2147427135
box_rearrange::compute_dest:: INFO: gsize= 320 384 18249
box_rearrange::compute_dest:: INFO: nioproc 12 ioproc 12
ioindex -1
box_rearrange::compute_dest:: INFO io 1 start=
1 1 1 count=
320 384 1520

And my pio setting are:

CPL : pio_numiotasks = 12
CPL : pio_stride = 64
CPL : pio_root = 1
CPL : pio_iotype = 6

The pelayout is :

./pelayout
Comp NTASKS NTHRDS ROOTPE
CPL : 768/ 1; 0
ATM : 768/ 1; 0
LND : 320/ 1; 0
ICE : 448/ 1; 320
OCN : 640/ 1; 768
ROF : 320/ 1; 0
GLC : 32/ 1; 0
WAV : 32/ 1; 1408
ESP : 1/ 1; 0

I have roughly determined the cause of the problem, that is, I add my code to read a so large nc file (about 122GB) in cpl that exceeds the memory limit. If I read a much smaller file, it can run successfully.
For some reason, I'd better read this huge file at once. So I wonder how can I make this ?
I think my HPC hardware resources are completely sufficient to do that with 8GB memory per core.
Thanks for your help.

 

[jedwards]

The message
compdof=-2147427135
indicates an integer overflow. Can the input file be divided into several files?

 

[Mikasa]

The message
compdof=-2147427135
indicates an integer overflow. Can the input file be divided into several files?

If I divide the file into 5 files, can I read them all in sequence ? That seems to cost the same amout of memory.

 

[jedwards]

memory is not the problem - the offset into the file is exceeding the limits of a 32bit integer. By splitting the file
you should be able to stay within the limits.

 

[Mikasa]

memory is not the problem - the offset into the file is exceeding the limits of a 32bit integer. By splitting the file
you should be able to stay within the limits.

Thank you very much ! I will try it .