Memory Allocators

Discuss the OpenMP Examples document, updated for OpenMP 5.0
Forum rules
The OpenMP Forums are now closed to new posts. Please visit Stack Overflow if you are in need of help:
Posts: 19
Joined: Tue Aug 27, 2013 6:48 pm

Memory Allocators

Post by johncampbell »

I am looking for a way to manage large arrays and especially PRIVATE copies of large arrays in a Fortran OpenMP program.
I am not sure if the Memory Allocators provide a convenient way of achieving this result.

My memory use strategy is based on:
1) "small" arrays and variables should go on the stack and so do private copies go on the thread stack. This localises memory use to assist in cache use efficiency and minimises cache coherence problems between threads.
2) "large" arrays should go on the heap to minimise the problems of stack overflow, especially when scaling up problem size for general applications. Private copies of these large arrays should be placed on the heap. As there is a single heap, arrays for different threads should be located on separate memory pages to address cache coherence problems.
3) temporary copies of "large" arrays should be directed to the heap.
4) compiler directives, such as -fmax-stack-var-size=n should be able to work with not only local arrays, but also automatic arrays and PRIVATE arrays.

After reading the Ver 5.0 specs: 2.7 Memory Allocators and the openmp-examples: 8 Memory Model; I am not sure how this definition and example approach can assist me in managing the memory use for arrays and especially PRIVATE arrays.

The main problem I see with Memory Allocators is how vague their definition is. As with the Fortran Standard that refuses to acknowledge the "Byte", OpenMP appears to deny the existence of Stack and Heap. Some of the defined allocator names are certainly vague in their meaning, eg "high_bw", "low_lat" or "cgroup" could have an variety of interpretations. Stack and Heap have been available in all O/S I have used and I am not aware of any O/S that do not have this dynamic memory differentiation. These are the two main options for dynamic array location. With many O/S implementations, there is a single Heap that is extendable, while with many implementations of Stack, they have a finite (non-extendable), very small size in a 64-bit application, although separate for each thread. For robust applications the stack must be avoided for large arrays of varying size, which includes Private copies.

For my understanding, I would have preferred to have a qualifier applied to PRIVATE arrays, to address the considerations I have outlined above.
My understanding is that all PRIVATE arrays are placed on the thread stack.
A problem regarding private arrays in Fortran OpenMP is how ALLOCATABLE arrays are managed.
If the array is ALLOCATED before the !$OMP region, I assume the PRIVATE copies would go on the STACK, which is not preferable.
If the array is NOT ALLOCATED before the !$OMP region, I assume the PRIVATE copies would go on the HEAP, when they are ALLOCATED in the !$OMP region. Do they need to be declared as PRIVATE ? I most easily achieve this by calling a (recursive) routine in the !$OMP region to define and utilise these types of arrays.

I am looking for some comments or direction as to if Memory Allocators were devised for the purposes I have described above.

John Campbell