OMP program runs with 1 thread but fails on 2 threads

General OpenMP discussion

OMP program runs with 1 thread but fails on 2 threads

Postby nlcg » Sun Jul 21, 2019 2:50 pm

This has been a very frustrating problem because what I'm attempting to do is so simple.

The program is a sequence of four real vector * complex vector multiplicatins.
The ph(*,4) real matrix is a global variable in module "global_arrays".
I see no race condition problems.

Can anyone see any openMP problem with the following code? It runs fine with one thread.
The OMP environment variables are displayed after the problematic code.

SUBROUTINE PhiXnegOMP(p,ap)
!
! Called by PhiSolveOMP
!
! Real matrix, Complex vector
! This program is a compact matrix*vector multiply for FD system matrix
! ndn & nde=number of cells in the north & east directions
! ndb=number of cells in z directions
!
USE global_arrays
IMPLICIT NONE
INTEGER(4):: LMN,i,k,mythred,omp_get_thread_num
COMPLEX(8):: p(*),ap(*)
!
LMN=ndn*nde*ndb
! diagonal term
!
!$OMP PARALLEL DO
DO i=1,LMN
ap(i)=-ph(i,1)*p(i)
ENDDO
!$OMP END PARALLEL DO
! +1 & -1 from diagonal
!$OMP PARALLEL DO
DO i=1,LMN-1
ap(i)=ap(i)-ph(i,2)*p(i+1)
ap(i+1)=ap(i+1)-ph(i,2)*p(i)
ENDDO
!$OMP END PARALLEL DO
! +ndn & -ndn from diagonal
k=ndn
!$OMP PARALLEL DO
DO i=1,LMN-k
ap(i)=ap(i)-ph(i,3)*p(i+k)
ap(i+k)=ap(i+k)-ph(i,3)*p(i)
ENDDO
!$OMP END PARALLEL DO
! +ndn*nde & -ndn*nde from diagonal
! OPEN(unit=4,file='inthred.txt',status='unknown')
k=ndn*nde
!$OMP PARALLEL DO PRIVATE(mythred)
DO i=1,LMN-k
ap(i)=ap(i)-ph(i,4)*p(i+k)
ap(i+k)=ap(i+k)-ph(i,4)*p(i)
! mythred=omp_get_thread_num()
! write(4,*)i,n,mythred,p(i),p(n),ph(i,4)
ENDDO
!$OMP END PARALLEL DO
! close(4)
RETURN
END

*********************************************************
Here's what OMP environment variables look like

OPENMP DISPLAY ENVIRONMENT BEGIN
_OPENMP = '201511'
OMP_DYNAMIC = 'FALSE'
OMP_NESTED = 'FALSE'
OMP_NUM_THREADS = '2'
OMP_SCHEDULE = 'DYNAMIC'
OMP_PROC_BIND = 'FALSE'
OMP_PLACES = ''
OMP_STACKSIZE = '16777216'
OMP_WAIT_POLICY = 'PASSIVE'
OMP_THREAD_LIMIT = '4294967295'
OMP_MAX_ACTIVE_LEVELS = '2147483647'
OMP_CANCELLATION = 'FALSE'
OMP_DEFAULT_DEVICE = '0'
OMP_MAX_TASK_PRIORITY = '0'
GOMP_CPU_AFFINITY = ''
GOMP_STACKSIZE = '16777216'
GOMP_SPINCOUNT = '300000'
OPENMP DISPLAY ENVIRONMENT END
nlcg
 
Posts: 4
Joined: Mon Jun 17, 2019 8:45 am

Re: OMP program runs with 1 thread but fails on 2 threads

Postby nlcg » Sat Jul 27, 2019 12:10 pm

Ok, can anyone see any problem with following simple loop. I see no race condition. It fails on many threads but runs fine on one thread. Thanks, Bill

!$OMP PARALLEL DO
DO i=1,LMN-1
ap(i)=ap(i)-ph(i,2)*p(i+1)
ap(i+1)=ap(i+1)-ph(i,2)*p(i)
ENDDO
!$OMP END PARALLEL DO
nlcg
 
Posts: 4
Joined: Mon Jun 17, 2019 8:45 am

Re: OMP program runs with 1 thread but fails on 2 threads

Postby MarkB » Tue Jul 30, 2019 3:46 am

Yes, there is a race: e.g. on 2 threads the first iteration on the second thread writing to ap(i) is modifying the same memory location as the last iteration on the first thread accessing ap(i+1).
MarkB
 
Posts: 782
Joined: Thu Jan 08, 2009 10:12 am
Location: EPCC, University of Edinburgh

Re: OMP program runs with 1 thread but fails on 2 threads

Postby nlcg » Tue Jul 30, 2019 8:33 am

Thanks Mark, I see that now. I've reprogrammed and now it runs fine. Thanks again and cheers.
nlcg
 
Posts: 4
Joined: Mon Jun 17, 2019 8:45 am

Re: OMP program runs with 1 thread but fails on 2 threads

Postby nlcg » Tue Jul 30, 2019 8:37 am

Can there also be a problem with two threads accessing p() at the same time. In other words, is two threads reading a shared variable at the same time allowed?
nlcg
 
Posts: 4
Joined: Mon Jun 17, 2019 8:45 am

Re: OMP program runs with 1 thread but fails on 2 threads

Postby MarkB » Tue Jul 30, 2019 10:27 am

nlcg wrote:Thanks Mark, I see that now. I've reprogrammed and now it runs fine. Thanks again and cheers.


You are very welcome!

nlcg wrote:Can there also be a problem with two threads accessing p() at the same time. In other words, is two threads reading a shared variable at the same time allowed?


Multiple threads reading the same location is perfectly fine.
MarkB
 
Posts: 782
Joined: Thu Jan 08, 2009 10:12 am
Location: EPCC, University of Edinburgh

Re: OMP program runs with 1 thread but fails on 2 threads

Postby johncampbell » Tue Oct 15, 2019 5:24 pm

Your choice of !$OMP for simple loops may not be effective as the computation involved may not be sufficient to justify the !$OMP overhead.
To overcome the race condition, you would be better off merging the two lines, giving one thread to each ap(i):
ap(i) = ap(i) - ph(i-1,2)*p(i-1) - ph(i,2)*p(i+1)

You then have to handle i==1 and i==LMN.
You can use IF tests, as:
!$OMP PARALLEL DO
DO i=1,LMN
if (i/=LMN) ap(i) = ap(i) - ph(i,2)*p(i+1)
if (i/=1) ap(i) = ap(i) - ph(i-1,2)*p(i-1)
END DO
!$OMP END PARALLEL DO

or preferably extend the dimensions of ph(0:LMN) and p(0:LMN+1), with p(0) = p(LMN+1) = 0, which eliminates IF tests inside the loop.
(if LMN is large, the IF tests are very inefficient and they also inhibit optimisation)
!$OMP PARALLEL DO
DO i=1,LMN
ap(i) = ap(i) - ph(i-1,2)*p(i-1) - ph(i,2)*p(i+1)
END DO
!$OMP END PARALLEL DO

The practicalities of using !$OMP need to be better understood, as to gain advantage from !$OMP, you need to have significant computation load on each DO i iteration to overcome the overhead of initiating a !$OMP region. (OMP the outer loop; vectorise the inner loop)
In this case LMN would need to be very large, although, for the i7 processors I am familiar with, this would then lead to problems with memory <> cache delays, especially if AVX instructions are selected. SCHEDULE(STATIC) would be preferable to SCHEDULE(DYNAMIC) to help with this.
johncampbell
 
Posts: 14
Joined: Tue Aug 27, 2013 6:48 pm


Return to Using OpenMP

Who is online

Users browsing this forum: Google [Bot] and 1 guest

cron