Intermittent Problem on Yellowstone with MPI_Wtime calls

A

aliceb

Member

There have been intermittent failures of CESM that can be traced back to a problem calling MPI_Wtime.   We do not use the MPI_Wtime function in the gptl timing library on yellowstone, however some of the CESM component models (notably pop) call MPI_Wtime directly.    Adding the module load command below replaces the MPI_Wtime function with a call to system clock.   If you have seen this problem on yellowstone, add the lines:module use -a /glade/apps/contrib/wtimemodule load wt

to your env_mach_specific file.
 
M

matthew_huber@unh_edu

New Member
Hi,  I think I may be having this problem on our cluster at Purdue (conte, which is functionally identical to yellowstone).  I was wondering if you could point me to the specific version of wtime that the module is loading so that we can build it locally? Thanks!-m
 
M

matthew_huber@unh_edu

New Member
Hi,  I think I may be having this problem on our cluster at Purdue (conte, which is functionally identical to yellowstone).  I was wondering if you could point me to the specific version of wtime that the module is loading so that we can build it locally? Thanks!-m
 
M

matthew_huber@unh_edu

New Member
Hi,  I think I may be having this problem on our cluster at Purdue (conte, which is functionally identical to yellowstone).  I was wondering if you could point me to the specific version of wtime that the module is loading so that we can build it locally? Thanks!-m
 
M

matthew_huber@unh_edu

New Member
Hi,  I think I may be having this problem on our cluster at Purdue (conte, which is functionally identical to yellowstone).  I was wondering if you could point me to the specific version of wtime that the module is loading so that we can build it locally? Thanks!-m
 
J

jedwards

CSEG and Liaisons
Staff member
Hi Matthew,Our solution is to use gettimeofday and just bypass the mpi implimentation.    #include

double MPI_Wtime(){

    struct timeval tv;
    struct timezone tz;

    double wt;

    gettimeofday(&tv, &tz);

    wt = (double)tv.tv_usec/1000000.;
    return (wt);

}

double mpi_wtime_(){ return MPI_Wtime(); } 
 
J

jedwards

CSEG and Liaisons
Staff member
Hi Matthew,Our solution is to use gettimeofday and just bypass the mpi implimentation.    #include

double MPI_Wtime(){

    struct timeval tv;
    struct timezone tz;

    double wt;

    gettimeofday(&tv, &tz);

    wt = (double)tv.tv_usec/1000000.;
    return (wt);

}

double mpi_wtime_(){ return MPI_Wtime(); } 
 
J

jedwards

CSEG and Liaisons
Staff member
Hi Matthew,Our solution is to use gettimeofday and just bypass the mpi implimentation.    #include

double MPI_Wtime(){

    struct timeval tv;
    struct timezone tz;

    double wt;

    gettimeofday(&tv, &tz);

    wt = (double)tv.tv_usec/1000000.;
    return (wt);

}

double mpi_wtime_(){ return MPI_Wtime(); } 
 
J

jedwards

CSEG and Liaisons
Staff member
Hi Matthew,Our solution is to use gettimeofday and just bypass the mpi implimentation.    #include

double MPI_Wtime(){

    struct timeval tv;
    struct timezone tz;

    double wt;

    gettimeofday(&tv, &tz);

    wt = (double)tv.tv_usec/1000000.;
    return (wt);

}

double mpi_wtime_(){ return MPI_Wtime(); } 
 
You must log in or register to reply here.
Back
Top