Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HIP updates #63

Merged
merged 6 commits into from
Jan 18, 2024
Merged

HIP updates #63

merged 6 commits into from
Jan 18, 2024

Conversation

MichaelSt98
Copy link
Contributor

  • adding arch files for LUMI CCE 16.0.1
  • adding col/s as metric for HIP variants

Copy link
Collaborator

@mlange05 mlange05 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look sane to me, but I'll let @reuterbal check the LUMI configuration before merging. GTG from me though

Copy link
Collaborator

@reuterbal reuterbal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Many thanks. HIP variants work perfectly as expected, and plain SCC still crashes the compiler, as expected.
But the OpenMP flags in the toolchain file aren't working for me as is. I've made a suggestion for a fix that I could build with successfully.

Comment on lines 20 to 28
set( ENABLE_OMP ON CACHE STRING "" )
set( OpenMP_C_FLAGS "-homp" CACHE STRING "" )
set( OpenMP_Fortran_FLAGS "-homp -hnoacc -hlist=aimd -maxrregcount 64" CACHE STRING "" )

####################################################################
# OpenACC FLAGS
####################################################################

set( ENABLE_ACC ON CACHE STRING "" )
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't -maxrregcount 64 an NVHPC compiler option? It didn't work for me with CCE.
Also, there's the usual OpenMP library lookup failure that we know from IFS. The following options work, though:

Suggested change
set( ENABLE_OMP ON CACHE STRING "" )
set( OpenMP_C_FLAGS "-homp" CACHE STRING "" )
set( OpenMP_Fortran_FLAGS "-homp -hnoacc -hlist=aimd -maxrregcount 64" CACHE STRING "" )
####################################################################
# OpenACC FLAGS
####################################################################
set( ENABLE_ACC ON CACHE STRING "" )
set( OpenMP_C_FLAGS "-fopenmp" )
set( OpenMP_CXX_FLAGS "-fopenmp" )
set( OpenMP_Fortran_FLAGS "-fopenmp" )
set( OpenMP_C_LIB_NAMES "craymp" )
set( OpenMP_CXX_LIB_NAMES "craymp" )
set( OpenMP_Fortran_LIB_NAMES "craymp" )
set( OpenMP_craymp_LIBRARY "/opt/cray/pe/cce/16.0.1/cce/x86_64/lib/libcraymp.so" )
####################################################################
# OpenACC FLAGS
####################################################################

@reuterbal
Copy link
Collaborator

Thanks! I can confirm that all variants build now.

Unfortunately, the env file still doesn't quite work for me:

  1. The import of cray-hdf5/1.12.2.3 is missing, which is required to build without Serialbox
  2. somehow, none of the OpenACC or OpenMP offload variants seem to be offloading to GPU.

Would you mind fixing 1 and briefly investigating 2?

Copy link
Collaborator

@reuterbal reuterbal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for figuring out the missing offload. Still some problems with the env file, at least when I've tried it, but a suggested solution that worked for me.

arch/eurohpc/lumi/cray-gpu/16.0.1/env.sh Outdated Show resolved Hide resolved
Co-authored-by: Balthasar Reuter <6384870+reuterbal@users.noreply.github.com>
@MichaelSt98
Copy link
Contributor Author

Just double-checked your proposed changes and committed them.
Thank you @reuterbal!

Copy link
Collaborator

@reuterbal reuterbal left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect, many thanks. Going in

@reuterbal reuterbal merged commit 54f7eea into develop Jan 18, 2024
16 checks passed
@reuterbal reuterbal deleted the nams_paper branch January 18, 2024 15:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants