DO loop does not wait for all threads to complete

General OpenMP discussion

DO loop does not wait for all threads to complete

Postby ols6000 » Wed May 01, 2019 4:16 pm

I am having a problem with a multi-threaded Fortran DO loop. It behaves as if there is no barrier at the end of the loop, so the results are wrong.
Here is an outline of the code:
Code: Select all
!$OMP PARALLEL DEFAULT(SHARED)
!$OMP DO PRIVATE(iRun, ...)

DO iRun=1, MAX(nRuns, 1)
    CALL Work1(iRun, ...)
    CALL Work2(iRun, ...)
    CALL Work3(iRun, ...)
END DO ! iRun
!$OMP END DO
! optional !$OMP BARRIER
!$OMP END PARALLEL
CONTINUE

In this example, nRuns=3. I have verified with the debugger that all 3 threads enter Work1, but when execution reaches the final CONTINUE line, only the thread for iRun=1 (presumably OpenMP thread 0) has completed Work1.
The result is the same if the optional BARRIER line is used, although according to "Using OpenMP" there is a barrier automatically inserted at the end of the DO loop.

Can someone see what I am doing wrong?

64-bit Windows 7; Visual Studio 2017 (15.9.11); Intel Fortran XE 2019 (19.0.3.203)
ols6000
 
Posts: 3
Joined: Thu Jan 31, 2019 11:10 am

Re: DO loop does not wait for all threads to complete

Postby MarkB » Thu May 02, 2019 9:06 am

I think the debugger could be misleading you here: only the master thread (thread 0) will reach the CONTINUE statement, since it is outside of the parallel region (and the end of the parallel region acts like a barrier). What are the symptoms that suggest something is wrong?
MarkB
 
Posts: 779
Joined: Thu Jan 08, 2009 10:12 am
Location: EPCC, University of Edinburgh

Re: DO loop does not wait for all threads to complete

Postby ols6000 » Thu May 02, 2019 1:15 pm

@MarkB -

I have write statements in Work1 that show that thread 0 has completed, and threads 1 and 2 have not. I also initialized the array output by Work1 to a distinctive value within Work1, which shows me that at the time of the CONTINUE (outside the parallel region), thread 0 has the correct result, thread 1 has not reached the array initialization in Work1, while thread 2 has executed the array initialization, but not the calculations. The progress shown by the array agrees with the order of execution of Work1, which is thread 0, then 2, then 1.

So I am pretty sure this isn't a debugger artifact. At the CONTINUE (outside the parallel region) the debugger does show thread 0 in the calling pgm, as expected, and threads 1 and 2 in libiomp5md.dll (Intel's implementation of OpenMP).

Here is the stack (identical for threads 1 and 2) at the CONTINUE:
Code: Select all
   [External Code]   
>   libiomp5md.dll!__kmp_yield() Line 564   C++
   libiomp5md.dll!__kmp_hyper_barrier_release(barrier_type bt, kmp_info * this_thr, int gtid, int tid, int propagate_icvs, void * itt_sync_obj) Line 405   C++
   libiomp5md.dll!__kmp_fork_barrier(int gtid, int tid) Line 2334   C++
   libiomp5md.dll!__kmp_launch_thread(kmp_info * this_thr) Line 6088   C++
   libiomp5md.dll!__kmp_launch_worker(void * arg) Line 1032   C++
   [External Code]   
ols6000
 
Posts: 3
Joined: Thu Jan 31, 2019 11:10 am

Re: DO loop does not wait for all threads to complete

Postby MarkB » Wed May 08, 2019 6:47 am

Does it work if you code it as:

Code: Select all
!$OMP PARALLEL DEFAULT(SHARED) PRIVATE(iRun, ...)

    iRun=omp_get_thread_num()+1
    CALL Work1(iRun, ...)
    CALL Work2(iRun, ...)
    CALL Work3(iRun, ...)

!$OMP END PARALLEL
CONTINUE
MarkB
 
Posts: 779
Joined: Thu Jan 08, 2009 10:12 am
Location: EPCC, University of Edinburgh

Re: DO loop does not wait for all threads to complete

Postby ols6000 » Wed May 08, 2019 9:54 am

@MarkB -
Tnx for the suggestion. I had already verified that the thread # + 1 = iRun, using the debugger.

I did find the problem, though; I was looking at the wrong part of the output array with the debugger to determine whether all threads had completed. OpenMP is working as expected, i e, there is a barrier at the end of the DO loop. Phew!
ols6000
 
Posts: 3
Joined: Thu Jan 31, 2019 11:10 am


Return to Using OpenMP

Who is online

Users browsing this forum: No registered users and 5 guests

cron