Parallel DO hangs 2nd time thru

General OpenMP discussion
Forum rules
The OpenMP Forums are now closed to new posts. Please visit Stack Overflow if you are in need of help: https://stackoverflow.com/questions/tagged/openmp
Locked
ols6000
Posts: 4
Joined: Thu Jan 31, 2019 11:10 am

Parallel DO hangs 2nd time thru

Post by ols6000 »

I'm using OpenMP5 with Intel Fortran, running under 64-bit Windows 7, and using Visual studio 2019 16.6.1.
When I run my executable, it hangs at the beginning of a parallel region, but only the second time. I cannot understand why this should be.
The main program is C++. It calls a Fortran subroutine Compute directly (in the main thread), then starts a worker thread to call a second Fortran subroutine, Wide. When Wide has finished, the worker thread terminates.
Both Fortran subroutines contain a single parallel DO
All this works as expected the first time I run it. But the second time, it hangs at the start of the parallel DO in Compute.
The code looks like this:
Main C++ thread: Compute(...)
Fortran Compute:

Code: Select all

!$OMP PARALLEL DEFAULT(NONE) &
!$OMP SHARED(...)
!$OMP DO &
!$OMP   PRIVATE(iWavelengthF,... )

   DO iWavelengthF=1, nWavelengthsAlloc
...
   END DO ! wavelength
!$OMP END DO
!$OMP END PARALLEL
Fortran Wide:

Code: Select all

!$OMP PARALLEL DEFAULT(NONE) &
!$OMP   SHARED(...)
!$OMP DO &
!$OMP   PRIVATE(iWavelengthF, ... )

        DO iWavelengthF=1, nWavelengths
...
        END DO ! iWavelengthF
!$OMP END DO
!$OMP END PARALLEL
Here are the threads, as shown by the debugger; the initial column is the Windows thread ID. As is evident, OpenMP is not using the set of 7 parallel worker threads it created for the Compute parallel DO, but instead has created a second team of 7. It looks like the second time thru Compute, the first team of worker threads has been activated, but OpenMP is waiting for an event which never happens. This is a 4-core hyper-threaded Intel processor, so there are 8 logical CPUs.

Compute prior to parallel DO
2496 0 Main Thread Main Thread CppMain!Compute(, , , , , , , , , , , , , , , )

Compute after parallel DO
2496 0 Main Thread Main Thread CppMain!Compute(, , , , , , , , , , , , , , , )
9544 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_hyper_barrier_release
8052 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
5432 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
8236 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
9712 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
4112 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
6996 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_hyper_barrier_release

Wide prior to loop
2496 0 Main Thread Main Thread gdi32.dll!NtGdiGetRandomRgn
9544 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
8052 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
5432 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
8236 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9712 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
4112 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
6996 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64


Wide after parallel region
2496 0 Main Thread Main Thread CppMain!AfxInternalPumpMessage
9544 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
8052 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
5432 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
8236 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9712 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
4112 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
6996 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
2328 0 Worker Thread CppMain!thread_start<unsigned int (__cdecl*)(void *),1> CppMain!Wide(, , , )
9836 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_hyper_barrier_release
6584 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
9220 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
5216 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
1904 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
9656 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield
9872 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_yield

Main after Wide thread terminated
2496 0 Main Thread Main Thread CppMain!AfxInternalPumpMessage
9544 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
8052 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
5432 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
8236 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9712 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
4112 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
6996 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9836 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
6584 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9220 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
5216 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
1904 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9656 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9872 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64

Behavior OK to here, although it's a mystery why OpenMP started a second team of threads.

Compute before parallel DO:
2496 0 Main Thread Main Thread CppMain!Compute(, , , , , , , , , , , , , , , )
9544 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
8052 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
5432 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
8236 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9712 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
4112 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
6996 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9836 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
6584 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9220 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
5216 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
1904 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9656 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9872 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64

Compute hanging at start of parallel DO:
2496 0 Main Thread Main Thread libiomp5md.dll!__kmp_suspend_64
9544 0 Worker Thread libiomp5md.dll!__kmp_launch_worker CppMain!L_Compute.V_176__par_region0_2.1()
8052 0 Worker Thread libiomp5md.dll!__kmp_launch_worker CppMain!L_Compute.V_176__par_region0_2.1()
5432 0 Worker Thread libiomp5md.dll!__kmp_launch_worker CppMain!L_Compute.V_176__par_region0_2.1()
8236 0 Worker Thread libiomp5md.dll!__kmp_launch_worker CppMain!L_Compute.V_176__par_region0_2.1()
9712 0 Worker Thread libiomp5md.dll!__kmp_launch_worker CppMain!L_Compute.V_176__par_region0_2.1()
4112 0 Worker Thread libiomp5md.dll!__kmp_launch_worker CppMain!L_Compute.V_176__par_region0_2.1()
6996 0 Worker Thread libiomp5md.dll!__kmp_launch_worker CppMain!L_Compute.V_176__par_region0_2.1()
9836 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
6584 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9220 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
5216 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
1904 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9656 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64
9872 0 Worker Thread libiomp5md.dll!__kmp_launch_worker libiomp5md.dll!__kmp_suspend_64

MarkB
Posts: 808
Joined: Thu Jan 08, 2009 10:12 am
Location: EPCC, University of Edinburgh

Re: Parallel DO hangs 2nd time thru

Post by MarkB »

I would not be surprised if the (version of the) Intel compiler does not support mixing C++ threads and OpenMP in the same application in this way.

Locked