Questions about OPENMP to transfer serial to parallel

[alex]

thanks Fernando, what you said is right! I just have the questions to ask you in two cases:
1.if I came across memory issues that the number of data read is so larger (ex. 20,000,000), and I want to use it again in the next time, so the array(ex. array(20000000, 20) or twenty array(20000000) )will exist in the memory till the next time dealing. that will bring the memory consuming , how to deal with this problem.
2.in the upper cases, in the case that I split the program into several parts(I/O reading, computing(parallel), writing) which abides by your suggestions, but if there exists I/O in the subroutine1 which also exists in the parallel computing block, is there any problems and how to solve this problem?

I do not understand 1. would you explain/give other details?
what I meant is that if I accept your suggestions, I must split the I/O(read & write) and the calculations. so I must read data from files to specific arrays, the dim of one array in my computation is upto 20000000 and I have the total 20 arrays like that. so large memory consuming is here, how to deal with this problem if I read data into these arrays?

About 2.: if you have I/O in several threads, the minimum I suggest is protecting the I/O with the critical synchronization construct (other issues would depend on the application). However, remember that threads will make interleaved I/O, which, in particular, usually produces different output data (more specifically: the same output in different relative order). Thus, it would be necessary to take into account the interleaved output when reading/using the generated data.
thanks, but I don't understand what you means. can you explain to me again?

HTH,

Fernando.

[MarkB]

If memory is a problem, then you can block your I/O into appropriate sizes, for example:

do b = 1, blocks

do i = 1, numlinesperblock !numline is the total lines of the somefile
read(10, format1, iostat=stat) x(i),y(i),z(i)
if(stat/=0) exit
enddo

!$omp parallel do
do i = 1, numlinesperblock
call subroutine1(x(i),y(i),z(i),val(i))
enddo

do i = 1, numlinesperblock
write(20, format2) val(i)
enddo


enddo

where blocks might be 200 and numlinesperblock is 100,000 for example.


If there is output inside subroutine1, then you could enclose the I/O statements in critical regions as Fernanado suggests, e.g.

!$omp critical
write(...), ....
!$omp end critical

provided that you do not mind that the output may appear in a different order from the sequential program.

If there is input inside subroutine1, I would recommend moving it outside the parallel region if at all possible.

[ftinetti]

Alex,

I think Markb's post has everything you need.

As far as I understand from your initial post, there is not I/O involved in subroutine1, right?

On "interleaved output data": remember that each thread reaches the write statement almost asynchronously.

Fernando.

[alex]

If memory is a problem, then you can block your I/O into appropriate sizes, for example:

do b = 1, blocks

do i = 1, numlinesperblock !numline is the total lines of the somefile
read(10, format1, iostat=stat) x(i),y(i),z(i)
if(stat/=0) exit
enddo

!$omp parallel do
do i = 1, numlinesperblock
call subroutine1(x(i),y(i),z(i),val(i))
enddo

do i = 1, numlinesperblock
write(20, format2) val(i)
enddo


enddo

where blocks might be 200 and numlinesperblock is 100,000 for example.


If there is output inside subroutine1, then you could enclose the I/O statements in critical regions as Fernanado suggests, e.g.

!$omp critical
write(...), ....
!$omp end critical

provided that you do not mind that the output may appear in a different order from the sequential program.

If there is input inside subroutine1, I would recommend moving it outside the parallel region if at all possible.

THANKS FOR MarkB very much!!!
I take your advice and have a test ,it's compiled, but the result is out of order and not right.
For the our of order I can understand , it's because I use !$OMP CRITICAL.
but for the not right result I analysed the scripts I wrote and I tthought the variables used in the parallel regions are not right.
cause I followed you and Fernando' s advice and did the parallel .
so can you give me some advice how to set the attribute of variables used in parallel regions.
in the tutorials about OPENMP, some criterions is talked about that but I am confused with those.

[alex]

Alex,

I think Markb's post has everything you need.

As far as I understand from your initial post, there is not I/O involved in subroutine1, right?

On "interleaved output data": remember that each thread reaches the write statement almost asynchronously.

Fernando.

thanks Fernando,
there is no I/O involved in subroutine1.

and as you said, MarB's post is very useful to me and so is yours.

and I have another questions about variables attribute set in the parallel regions which I put in upper block.

thanks again.

[MarkB]

You may be unintentionally sharing some variables between threads: you need to examine you code for cases where more than one thread could access the same shared variables at the same time. Remember that in Fortran, variables declared inside subroutines with the SAVE attribute are shared, as are variables declared in modules or COMMON blocks.

[alex]

You may be unintentionally sharing some variables between threads: you need to examine you code for cases where more than one thread could access the same shared variables at the same time. Remember that in Fortran, variables declared inside subroutines with the SAVE attribute are shared, as are variables declared in modules or COMMON blocks.

thanks MarkB,

I modified my scripts in Fortran 90, compiled it and run it , but there was a problem happened when using parallel method:
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
libpthread.so.0 00007F7863F699CB Unknown Unknown Unknown
libiomp5.so 00007F786449E2A8 Unknown Unknown Unknown

I googled this problem, some said the 'ulimit -s unlimited' in '.bashrc ' file is ok, but after I added it in my .bashrc file ,it won't be help.

can you give me some helpful advice?

PS：I do my parallel method using this declare ahead of my parallel block:

!$OMP PARALLEL DO &
!$OMP DEFAULT(SHARED) &
!$OMP PRIVATE(COSRAD,FGCSRAD,FBCSRAD,LATD,LONGD,UTCT,P,PCS,K,II,JJ,KK,MJT1,MJT2,MJT,NEWSTEC,NEWVTEC,VTECRMS,OLDSTEC,WT,YG,FACT1,FACT2,snindex)
wherein, varibles in !$OMP PRIVATE is the temporary intermediate variables in do loop block.

[MarkB]

If it is a stack size problem (which can be the case if you have large private arrays), then you may also need to set the slave thread stack size using the OMP_STACKSIZE environment variable. ulimit only affects the stack size of the master thread.

[alex]

You may be unintentionally sharing some variables between threads: you need to examine you code for cases where more than one thread could access the same shared variables at the same time. Remember that in Fortran, variables declared inside subroutines with the SAVE attribute are shared, as are variables declared in modules or COMMON blocks.

thanks MarkB,

I modified my scripts in Fortran 90, compiled it and run it , but there was a problem happened when using parallel method:
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
libpthread.so.0 00007F7863F699CB Unknown Unknown Unknown
libiomp5.so 00007F786449E2A8 Unknown Unknown Unknown

I googled this problem, some said the 'ulimit -s unlimited' in '.bashrc ' file is ok, but after I added it in my .bashrc file ,it won't be help.

can you give me some helpful advice?

PS：I do my parallel method using this declare ahead of my parallel block:

!$OMP PARALLEL DO &
!$OMP DEFAULT(SHARED) &
!$OMP PRIVATE(COSRAD,FGCSRAD,FBCSRAD,LATD,LONGD,UTCT,P,PCS,K,II,JJ,KK,MJT1,MJT2,MJT,NEWSTEC,NEWVTEC,VTECRMS,OLDSTEC,WT,YG,FACT1,FACT2,snindex)
wherein, varibles in !$OMP PRIVATE is the temporary intermediate variables in do loop block.

Hello MarkB,
why I add export OMP_STACKSIZE=1.5G into my .bashrc, I also get the same error report:
forrtl: severe (174): SIGSEGV, segmentation fault occurred
Image PC Routine Line Source
libpthread.so.0 00002AF07AEEC9CB Unknown Unknown Unknown
libiomp5.so 00002AF07ADB72A8 Unknown Unknown Unknown

hope your reply and advice
yours alex

[MarkB]

Time to start using a debugger, I think.
Do you get the error when running with OpenMP but on only one thread?