Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Disable randomly failing Tpetra test (#10847), remove FORCE from all test disables (#10870) #10871

Merged
merged 3 commits into from
Aug 16, 2022
Merged

Disable randomly failing Tpetra test (#10847), remove FORCE from all test disables (#10870) #10871

merged 3 commits into from
Aug 16, 2022

Conversation

bartlettroscoe
Copy link
Member

@bartlettroscoe bartlettroscoe commented Aug 13, 2022 •
edited
Loading

CC: @trilinos/tpetra

Internal Issues

Description

This PR disables the randomly failing tests TpetraTSQR_SequentialTsqr_[non]contiguousCacheBlocks_MPI_1 that is taking out all kinds of PR iterations (see #10847). The PR also removes 'FORCE' from all test disables (see #10870).

How was this tested?

Using the scripts:

load-env-and-cmake-frag-file.sh:

export TRILINOS_DIR=/fgs/rabartl/Trilinos.base/Trilinos
FULL_BUILD_NAME=rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables

if [[ -e GenConfigSettings.cmake ]] ; then
  echo "Remvoing existing file GenConfigSettings.cmake ..."
  rm GenConfigSettings.cmake
fi

export WORKSPACE=$TRILINOS_DIR/..

source $TRILINOS_DIR/packages/framework/GenConfig/gen-config.sh \
--cmake-fragment GenConfigSettings.cmake \
$FULL_BUILD_NAME \
--force \
$TRILINOS_DIR

unset FULL_BUILD_NAME
unset WORKSPACE

# NOTE Above, it is an undocumented requirment that the env vars TRILINOS_DIR
# and WORKSPACE be set!

and:

do-configure:

if [[ -e CMakeCache.txt ]] ; then
  echo "Removing CMakeCache.txt ..."
  rm CMakeCache.txt
fi
if [[ -d CMakeFiles ]] ; then
  echo "Removing CMakeFiles ..."
  rm -r CMakeFiles
fi
cmake \
-G Ninja \
-C GenConfigSettings.cmake \
-D Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES=OFF \
-D Trilinos_ENABLE_TESTS=ON \
-D Trilinos_TRACE_ADD_TEST=ON \
"$@" \
$TRILINOS_DIR

I configured on 'ascicgpu17' with:

$ ssh ascicgpu17

$ cd /fgs/rabartl/Trilinos.base/BUILDS/PR/rhel7_sems-cuda-11.4.2/

$ . load-env-and-cmake-frag-file.sh

$ time ./do-configure -DTrilinos_ENABLE_Tpetra=ON &> configure.out

real    0m55.633s
user    0m22.293s
sys     0m19.367s

and the configure output showed:

$ grep TpetraTSQR_SequentialTsqr_ configure.out 
-- TpetraTSQR_SequentialTsqr_contiguousCacheBlocks_MPI_1: NOT added test because TpetraTSQR_SequentialTsqr_contiguousCacheBlocks_MPI_1_DISABLE='ON'!
-- TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1: NOT added test because TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1_DISABLE='ON'!

@bartlettroscoe
Remove FORCE from all test disables (#10870)
35dd8bf 
You *NEVER* want to force disable a test. If you force disable it, the
developer has no way to manually enable that test in order to fix it.
@bartlettroscoe
Disable test TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1…
6e31532 
… just on ascicgpu PR build (#10847)
@bartlettroscoe bartlettroscoe added pkg: Tpetra Framework tasks Framework tasks (used internally by Framework team) PA: Framework Issues that fall under the Trilinos Framework Product Area labels Aug 13, 2022
@bartlettroscoe bartlettroscoe self-assigned this Aug 13, 2022
@bartlettroscoe
Copy link
Member Author

FYI: I merged this branch into the topic branch for PR #10808 so we can see how this works in practice.

@bartlettroscoe bartlettroscoe mentioned this pull request Aug 13, 2022
Closed
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Test Inspection' - Auto Inspected - Inspection Is Not Necessary for this Pull Request.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 860
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-serial

  • Build Num: 861
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-debug

  • Build Num: 859
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_intel-17.0.1

  • Build Num: 860
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-17.0.1-mpich-3.2-serial_release-debug_static_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_clang-10.0.0

  • Build Num: 864
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-10.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-10.1.243

  • Build Num: 876
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME ats2_cuda-10.1.243-gnu-8.3.1-spmpi-rolling_release_static_Volta70_Power9_no-asan_no-complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL vortex
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 688
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 396
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Using Repos:

Repo: TRILINOS (trilinos/Trilinos)
  • Branch: 10847-ascicgpu-disable-tpetra-test
  • SHA: 6e31532
  • Mode: TEST_REPO

Pull Request Author: bartlettroscoe

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 860
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-serial

  • Build Num: 861
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-debug

  • Build Num: 859
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_intel-17.0.1

  • Build Num: 860
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-17.0.1-mpich-3.2-serial_release-debug_static_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_clang-10.0.0

  • Build Num: 864
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-10.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-10.1.243

  • Build Num: 876
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME ats2_cuda-10.1.243-gnu-8.3.1-spmpi-rolling_release_static_Volta70_Power9_no-asan_no-complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL vortex
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 688
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 396
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 6e31532
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc


CDash Test Results for PR# 10871.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
WARNING: NO REVIEWERS HAVE BEEN REQUESTED FOR THIS PULL REQUEST!

@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@bartlettroscoe
Copy link
Member Author

FYI: I merged this branch into the branch in PR #10808 so this will get tested the next time that PR build runs.

NOTE: Trilinos is not testing updates to these files in PR testing. That is not good!

@bartlettroscoe bartlettroscoe mentioned this pull request Aug 13, 2022
Merged
@bartlettroscoe
Copy link
Member Author

FYI: I merged this branch into the branch in PR #10808 so this will get tested the next time that PR build runs.

NOTE: Trilinos is not testing updates to these files in PR testing. That is not good!

NOTE: A PR test iteration for PR #10808 on 'ascicgpu050` has started and we see in the configure output here for the build PR-10808-test-rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables-407 showing:

- TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1: NOT added test because TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1_DISABLE='ON'!

So that test will not run in that CUDA-11 build.

But looking at the configure output [here](https://trilinos-cdash.sandia.gov/build/378206/configure for the CUDA build on 'vortex' in the build PR-10808-test-ats2_cuda-10.1.243-gnu-8.3.1-spmpi-rolling_release_static_Volta70_Power9_no-asan_no-complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables-887 we see:

-- TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1: Added test (BASIC, NUM_MPI_PROCS=1, PROCESSORS=1)!

So that is proof that test TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1 is only being disabled in that CUDA-11 builds where it is randomly failing but not the other builds.

@bartlettroscoe bartlettroscoe added the AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed label Aug 14, 2022
@bartlettroscoe
Copy link
Member Author

Hello @jhux2, @csiefer2, and @cgcgcg, can one of you please approve this PR so that it can merge? This will remove this test from taking out PR builds due to its random failing for the CUDA-11 build on the ascicgpu machines but keeps the test running in all other builds, including the CUDA build vortex (see above).

@bartlettroscoe
Copy link
Member Author

The impossible seems to have occurred. The test TpetraTSQR_SequentialTsqr_contiguousCacheBlocks_MPI_1 was reported to have run and filed in the build PR-10808-test-rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables-407 but that test is shown disabled in the configure output showing:

-- TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1: NOT added test because TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1_DISABLE='ON'!

I will try to reproduce this on the machine and see what happens.

@bartlettroscoe
Copy link
Member Author

I will try to reproduce this on the machine and see what happens.

For the Trilinos version from PR #10808

$ ssh ascicgpu17

$ cd /fgs/rabartl/Trilinos.base/Trilinos/

$ git log-short --name-status -1
3dac90f5fda "Merge remote-tracking branch 'github/10865-ats2-disable-shylu' into 10807-kokkos-kernels-cublas (#10865)"
Author: Roscoe A. Bartlett <rabartl@sandia.gov>
Date:   Fri Aug 12 20:06:05 2022 -0600 (25 hours ago)

using the files:

load-env-and-cmake-frag-file.sh

export TRILINOS_DIR=/fgs/rabartl/Trilinos.base/Trilinos
FULL_BUILD_NAME=rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables

if [[ -e GenConfigSettings.cmake ]] ; then
  echo "Remvoing existing file GenConfigSettings.cmake ..."
  rm GenConfigSettings.cmake
fi

export WORKSPACE=$TRILINOS_DIR/..

source $TRILINOS_DIR/packages/framework/GenConfig/gen-config.sh \
--cmake-fragment GenConfigSettings.cmake \
$FULL_BUILD_NAME \
--force \
$TRILINOS_DIR

unset FULL_BUILD_NAME
unset WORKSPACE

# NOTE Above, it is an undocumented requirment that the env vars TRILINOS_DIR
# and WORKSPACE be set!

do-configure

rm -rf CMake*
cmake \
-G Ninja \
-C GenConfigSettings.cmake \
-D Trilinos_ENABLE_ALL_FORWARD_DEP_PACKAGES=OFF \
-D Trilinos_ENABLE_TESTS=ON \
-D Trilinos_TRACE_ADD_TEST=ON \
"$@" \
$TRILINOS_DIR

I did the following configuration and build on 'ascicgp17' just now:

$ time ./do-configure -DTrilinos_ENABLE_Tpetra=ON  &> configure.out && time ninja -j32 &> make.out && time ctest -j32 &> ctest.out

real    1m1.765s
user    0m22.275s
sys     0m19.939s

real    9m5.558s
user    95m29.468s
sys     21m59.136s

real    2m39.617s
user    7m47.328s
sys     7m58.314s

$ grep -A 50 "failed out of" ctest.out 
100% tests passed, 0 tests failed out of 280

Subproject Time Summary:
Tpetra    = 4210.91 sec*proc (280 tests)

Total Test time (real) = 159.50 sec

In other words, there has to be something wrong with the PR tester to be posting test results including the test TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1 in the build PR-10808-test-rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables-407.

Looking at the Jenkins logs for this build at:

the file:

when downloaded does not even show that test:

$ grep TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1 junk/LastTest_20220814-0113.log
[empty]

What in the check is going on here?

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO REVIEWS HAVE BEEN PERFORMED ON THIS PULL REQUEST!

@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@bartlettroscoe
Copy link
Member Author

Wait, I missed that the failing test described above is TpetraTSQR_SequentialTsqr_contiguousCacheBlocks_MPI_1 which is NOT the same test as TpetraTSQR_SequentialTsqr_noncontiguousCacheBlocks_MPI_1 (contiguous vs noncontiguous).

I will update issue issue #10847 to see what other tests are showing this same error "an illegal instruction was encountered".

@bartlettroscoe
Copy link
Member Author

FYI: I put in the disable of the test TpetraTSQR_SequentialTsqr_contiguousCacheBlocks_MPI_1 as well in the commit 81a885d and I merged it to the topic branch for PR #10808 in merge commit 5c08a58.

Now we should see those errors clear up.

@bartlettroscoe
Copy link
Member Author

Hello @jhux2, @csiefer2, @cgcgcg, @ccober6, can one of you please approve this PR so that it can merge? These tests are taking out about 30% of the PR iterations where Tpetra tests are getting run (see updated Description on #10847).

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 883
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-serial

  • Build Num: 884
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-debug

  • Build Num: 882
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_intel-17.0.1

  • Build Num: 883
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-17.0.1-mpich-3.2-serial_release-debug_static_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_clang-10.0.0

  • Build Num: 887
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-10.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-10.1.243

  • Build Num: 899
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME ats2_cuda-10.1.243-gnu-8.3.1-spmpi-rolling_release_static_Volta70_Power9_no-asan_no-complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL vortex
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 710
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS pkg: Tpetra;Framework tasks;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 419
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc


CDash Test Results for PR# 10871.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - - This Pull Request Requires Inspection... The code must be inspected by a member of the Team before Testing/Merging
NO REVIEWS HAVE BEEN PERFORMED ON THIS PULL REQUEST!

@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

@bartlettroscoe bartlettroscoe added the AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration label Aug 15, 2022
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - User Requested Retest - Label AT: RETEST will be reset after testing.

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Testing Jenkins Projects:

Pull Request Auto Testing STARTING (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 886
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-serial

  • Build Num: 887
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-debug

  • Build Num: 885
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_intel-17.0.1

  • Build Num: 886
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-17.0.1-mpich-3.2-serial_release-debug_static_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_clang-10.0.0

  • Build Num: 890
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-10.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-10.1.243

  • Build Num: 902
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME ats2_cuda-10.1.243-gnu-8.3.1-spmpi-rolling_release_static_Volta70_Power9_no-asan_no-complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL vortex
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 713
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 422
  • Status: STARTED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Using Repos:

Repo: TRILINOS (trilinos/Trilinos)
  • Branch: 10847-ascicgpu-disable-tpetra-test
  • SHA: 81a885d
  • Mode: TEST_REPO

Pull Request Author: bartlettroscoe

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Jenkins Testing: all Jobs PASSED

Pull Request Auto Testing has PASSED (click to expand)

Build Information

Test Name: Trilinos_PR_gcc-8.3.0

  • Build Num: 886
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-8.3.0-openmpi-1.10.1-openmp_release-debug_static_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-serial

  • Build Num: 887
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_gcc-7.2.0-debug

  • Build Num: 885
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-openmpi-1.10.1-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_intel-17.0.1

  • Build Num: 886
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-intel-17.0.1-mpich-3.2-serial_release-debug_static_no-kokkos-arch_no-asan_no-complex_fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_clang-10.0.0

  • Build Num: 890
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-clang-10.0.0-openmpi-1.10.1-serial_release-debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_mpi_no-pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL trilinos-any
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-10.1.243

  • Build Num: 902
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME ats2_cuda-10.1.243-gnu-8.3.1-spmpi-rolling_release_static_Volta70_Power9_no-asan_no-complex_no-fpic_mpi_pt_no-rdc_uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL vortex
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_python3

  • Build Num: 713
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-gnu-7.2.0-anaconda3-serial_debug_shared_no-kokkos-arch_no-asan_no-complex_no-fpic_no-mpi_no-pt_no-rdc_no-uvm_deprecated-on_pr-framework
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL ascic
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc

Build Information

Test Name: Trilinos_PR_cuda-11.4.2-uvm-off

  • Build Num: 422
  • Status: PASSED

Jenkins Parameters

Parameter Name Value
FORCE_CLEAN true
GENCONFIG_BUILD_NAME rhel7_sems-cuda-11.4.2-sems-gnu-10.1.0-sems-openmpi-4.0.5_release_static_Volta70_no-asan_complex_no-fpic_mpi_pt_no-rdc_no-uvm_deprecated-on_no-package-enables
PR_LABELS pkg: Tpetra;Framework tasks;AT: RETEST;AT: AUTOMERGE;PA: Framework
PULLREQUESTNUM 10871
PULLREQUEST_CDASH_TRACK Pull Request
TEST_REPO_ALIAS TRILINOS
TRILINOS_NODE_LABEL GPU
TRILINOS_SOURCE_BRANCH 10847-ascicgpu-disable-tpetra-test
TRILINOS_SOURCE_REPO https://github.com/trilinos/Trilinos
TRILINOS_SOURCE_SHA 81a885d
TRILINOS_SRN_CONFIG true
TRILINOS_TARGET_BRANCH develop
TRILINOS_TARGET_REPO https://github.com/trilinos/Trilinos
TRILINOS_TARGET_SHA e061ffc


CDash Test Results for PR# 10871.

@trilinos-autotester trilinos-autotester removed the AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration label Aug 15, 2022
@trilinos-autotester
Copy link
Contributor

All Jobs Finished; status = PASSED, However Inspection must be performed before merge can occur...

jhux2
jhux2 approved these changes Aug 15, 2022
View reviewed changes
Copy link
Member

@jhux2 jhux2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

This was referenced Aug 15, 2022
Closed
Closed
@bartlettroscoe bartlettroscoe added AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) and removed AT: RETEST Causes the PR autotester to run a new round of PR tests on the next iteration AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed labels Aug 15, 2022
e10harvey
e10harvey requested changes Aug 15, 2022
View reviewed changes
Copy link
Contributor

@e10harvey e10harvey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use the FORCE option since this will be required during incremental rebuilds and is what cmake -DVAR=VAL does.

@bartlettroscoe
Copy link
Member Author

Please use the FORCE option since this will be required during incremental rebuilds and is what cmake -DVAR=VAL does.

Let's discuss this.

@bartlettroscoe
Copy link
Member Author

Please use the FORCE option since this will be required during incremental rebuilds and is what cmake -DVAR=VAL does.

Having the FORCE options on these cache vars breaks the workflow documented in:

in that if the developer passed in -D<FullTestName>_DISABLE=OFF, that will be ignored and the test will still be disabled.

@e10harvey, please document the use cases where you see a problem needing to be solved by using the FORCE option for these cache vars. I promise you, that if you blow away the cache before a reconfigure (in PR builds, in local builds, etc.), there is no issue here. (And you can never keep and existing CMakeCache.txt file in place and expect that rebuilds will be 100% correct.)

e10harvey
e10harvey approved these changes Aug 15, 2022
View reviewed changes
Copy link
Contributor

@e10harvey e10harvey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this just affects cuda UVM off builds, let's make a note of the FORCE option removal here and see if people start running into problems.

@bartlettroscoe
Copy link
Member Author

Since this just affects cuda UVM off builds, let's make a note of the FORCE option removal here and see if people start running into problems.

Thanks @e10harvey! I promise that as long as the PR tester and users run rm -rf CMake* before they configure with cmake ... to (re)build, then there is no need to set FORCE on these vars.

@bartlettroscoe bartlettroscoe added AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed and removed AT: WIP Causes the PR autotester to not test the PR. (Remove to allow testing to occur.) labels Aug 15, 2022
@trilinos-autotester
Copy link
Contributor

Status Flag 'Pre-Merge Inspection' - SUCCESS: The last commit to this Pull Request has been INSPECTED AND APPROVED by [ jhux2 e10harvey ]!

@trilinos-autotester
Copy link
Contributor

Status Flag 'Pull Request AutoTester' - Pull Request will be Automerged

@trilinos-autotester trilinos-autotester merged commit 67db3e9 into develop Aug 16, 2022
@trilinos-autotester
Copy link
Contributor

Merge on Pull Request# 10871: IS A SUCCESS - Pull Request successfully merged

@trilinos-autotester trilinos-autotester removed the AT: AUTOMERGE Causes the PR autotester to automatically merge the PR branch once approvals are completed label Aug 16, 2022
@jhux2 jhux2 deleted the 10847-ascicgpu-disable-tpetra-test branch August 17, 2022 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jhux2 jhux2 jhux2 approved these changes

@e10harvey e10harvey e10harvey approved these changes

@csiefer2 csiefer2 Awaiting requested review from csiefer2

@cgcgcg cgcgcg Awaiting requested review from cgcgcg

Assignees

@bartlettroscoe bartlettroscoe

Labels
Framework tasks Framework tasks (used internally by Framework team) PA: Framework Issues that fall under the Trilinos Framework Product Area pkg: Tpetra
Projects
Trilinos TriBITS Refactor
Status: Done
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@bartlettroscoe @trilinos-autotester @jhux2 @e10harvey