Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Address CodeQL issues #36

Closed
wants to merge 61 commits into from
Closed
Show file tree
Hide file tree
Changes from 59 commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
5098757
fixed HDF5_VOL_CONNECTOR settings for the tests
brtnfld Nov 21, 2024
f08cd7a
update the README
brtnfld Nov 21, 2024
11e05f4
lower test timeout
brtnfld Dec 9, 2024
02d0993
Create codeql.yml
brtnfld Dec 9, 2024
ac1dc7c
fixed syntax
brtnfld Dec 9, 2024
cc6a340
fixed syntax2
brtnfld Dec 9, 2024
c06c64b
fixed syntax3
brtnfld Dec 9, 2024
e26a4ed
fixed syntax4
brtnfld Dec 9, 2024
c071d5d
fixed syntax5
brtnfld Dec 9, 2024
20a90c6
fixed syntax6
brtnfld Dec 9, 2024
7284948
fixed syntax7
brtnfld Dec 9, 2024
39565ef
fixed syntax8
brtnfld Dec 9, 2024
9050634
fixed syntax9
brtnfld Dec 9, 2024
d6208c9
fixed syntax10
brtnfld Dec 9, 2024
7d86bdf
fixed syntax11
brtnfld Dec 9, 2024
adcfab1
fixed syntax12
brtnfld Dec 9, 2024
9ddc93d
fixed syntax12
brtnfld Dec 9, 2024
9a3c79f
fixed syntax14
brtnfld Dec 9, 2024
893fbd4
fixed syntax14
brtnfld Dec 9, 2024
9b27022
fixed syntax15
brtnfld Dec 9, 2024
b45e20f
fixed syntax16
brtnfld Dec 9, 2024
253533d
fixed syntax16
brtnfld Dec 9, 2024
26cb15a
fixed syntax17
brtnfld Dec 9, 2024
f7dd31b
fixed syntax18
brtnfld Dec 9, 2024
b59afe8
Merge remote-tracking branch 'upstream/develop' into develop
brtnfld Dec 9, 2024
eb7c21e
remove h5bench
brtnfld Dec 9, 2024
06d6369
fixed critical codeQL items
brtnfld Dec 10, 2024
9edb1a7
Committing clang-format changes
github-actions[bot] Dec 10, 2024
0869f1d
Create codeql.yml
brtnfld Dec 10, 2024
59a6a10
Merge branch 'develop' into fix1
brtnfld Dec 10, 2024
a69dac2
update codeql
brtnfld Dec 10, 2024
f8a45e4
fixed skipped directory
brtnfld Dec 10, 2024
be88f65
using sarif
brtnfld Dec 10, 2024
11011b1
using sarif
brtnfld Dec 10, 2024
38fa3c8
using sarif
brtnfld Dec 10, 2024
4a69603
using sarif
brtnfld Dec 10, 2024
623d663
using sarif
brtnfld Dec 10, 2024
0eb918e
using sarif
brtnfld Dec 10, 2024
3ae14a7
codeql testing
brtnfld Dec 11, 2024
4e0f48a
codeql fixes
brtnfld Dec 11, 2024
e48da78
Committing clang-format changes
github-actions[bot] Dec 11, 2024
5d61f13
sp
brtnfld Dec 11, 2024
6d82d36
debug
brtnfld Dec 11, 2024
1831ab9
debug
brtnfld Dec 11, 2024
cd1324b
debug
brtnfld Dec 11, 2024
3dce469
codeql updates
brtnfld Dec 11, 2024
f8f43e9
debug2
brtnfld Dec 11, 2024
c9fa2b5
debug3
brtnfld Dec 11, 2024
84716a2
debug4
brtnfld Dec 11, 2024
a3914ab
debug5
brtnfld Dec 11, 2024
3ab9bb9
debug8
brtnfld Dec 11, 2024
8c30ddc
debug9
brtnfld Dec 11, 2024
f8b43d4
Fix code scanning alert no. 276: Unbounded write
brtnfld Dec 11, 2024
664418a
debug10
brtnfld Dec 11, 2024
7e9b76e
Committing clang-format changes
github-actions[bot] Dec 11, 2024
ccb2c0b
debug11
brtnfld Dec 11, 2024
fef7ad6
debug12
brtnfld Dec 11, 2024
7c42f0f
Committing clang-format changes
github-actions[bot] Dec 11, 2024
b58b5a7
debug14
brtnfld Dec 11, 2024
70fa3c5
debug15
brtnfld Dec 11, 2024
13d3df0
debug16
brtnfld Dec 11, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/codeql-config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# Query filters to include or exclude specific queries
query-filters:
- exclude:
# See: https://codeql.github.com/codeql-query-help/cpp/cpp-short-global-name/
id: cpp/short-global-name
- exclude:
# See: https://codeql.github.com/codeql-query-help/cpp/cpp-commented-out-code/
id: cpp/commented-out-code
- exclude:
# See: https://codeql.github.com/codeql-query-help/cpp/cpp-poorly-documented-function/
id: cpp/poorly-documented-function
- exclude:
# See: https://codeql.github.com/codeql-query-help/cpp/cpp-trivial-switch/
id: cpp/trivial-switch
- exclude:
# See: https://codeql.github.com/codeql-query-help/cpp/cpp-irregular-enum-init/
id: cpp/irregular-enum-init
23 changes: 23 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
Expand Up @@ -198,3 +198,26 @@ jobs:
uses: github/codeql-action/analyze@v3
with:
category: "/language:c-cpp"
output: sarif-results
upload: failure-only

- name: filter-sarif
uses: advanced-security/filter-sarif@main
with:
patterns: |
-**/*
src/**/*
input: sarif-results/cpp.sarif
output: sarif-results/cpp.sarif

- name: Upload SARIF
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: sarif-results/cpp.sarif

- name: Upload loc as a Build Artifact
uses: actions/upload-artifact@v4
with:
name: sarif-results
path: sarif-results
retention-days: 1
4 changes: 2 additions & 2 deletions .github/workflows/linux.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ on:
jobs:
vol-cache:
runs-on: ubuntu-latest
timeout-minutes: 60
timeout-minutes: 20

steps:
- uses: actions/checkout@v4.1.1
Expand Down Expand Up @@ -60,7 +60,7 @@ jobs:
# Compile HDF5
mkdir -p hdf5/build
cd hdf5/build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HDF5_DIR -DHDF5_ENABLE_PARALLEL:BOOL=ON -DHDF5_ENABLE_THREADSAFE:BOOL=ON -DALLOW_UNSUPPORTED:BOOL=ON ..
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$HDF5_DIR -DHDF5_ENABLE_PARALLEL:BOOL=ON -DHDF5_ENABLE_THREADSAFE:BOOL=ON -DHDF5_ALLOW_UNSUPPORTED:BOOL=ON ..
make -j2 install
cd -
cd argobots
Expand Down
4 changes: 2 additions & 2 deletions src/H5LS.c
Original file line number Diff line number Diff line change
Expand Up @@ -144,7 +144,7 @@ herr_t readLSConf(char *fname, cache_storage_t *LS) {
linenum++;
if (line[0] == '#')
continue;
if (sscanf(line, "%[^:]:%s", ip, mac) != 2) {
if (sscanf(line, "%[^:]:%255s", ip, mac) != 2) {
if (RANK == io_node())
fprintf(stderr, "Syntax error, line %d\n", linenum);
continue;
Expand All @@ -153,7 +153,7 @@ herr_t readLSConf(char *fname, cache_storage_t *LS) {
if (strcmp(mac, "NULL") == 0)
LS->path = NULL;
else {
strcpy(LS->path, mac);
snprintf(LS->path, 255, "%s", mac);
}

else if (!strcmp(ip, "HDF5_CACHE_FUSION_THRESHOLD")) {
Expand Down
133 changes: 90 additions & 43 deletions src/H5VLcache_ext.c
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@
#include <sys/statvfs.h>
#include <unistd.h>
// debug
#define LOG_BUFFER_SIZE 1024
// VOL related header
#include "H5LS.h"
#include "H5VLcache_ext_private.h"
Expand Down Expand Up @@ -96,6 +97,7 @@
int RANK = 0;
int NPROC = 1;
hbool_t HDF5_CACHE_CLOSE_ASYNC = 0;
char log_buffer[LOG_BUFFER_SIZE];
// Functions from async VOL
int H5VL_async_set_delay_time(uint64_t time_us);
herr_t H5VL_async_set_request_dep(void *request, void *parent_request);
Expand All @@ -106,7 +108,9 @@

#define H5Pcopy(X) \
H5Pcopy(X); \
LOG_DEBUG(-1, "H5Pcopy called: %s:%d %s\n", __FILE__, __LINE__, __FUNCTION__);
snprintf(log_buffer, LOG_BUFFER_SIZE, "H5Pcopy called: %s:%d %s\n", \
__FILE__, __LINE__, __FUNCTION__); \
LOG_DEBUG(-1, "%s", log_buffer);

#define H5Scopy(X) \
H5Scopy(X); \
Expand Down Expand Up @@ -496,6 +500,8 @@
return t->remove_file_cache(obj, req);
else if (o->obj_type == H5I_DATASET)
return t->remove_dataset_cache(obj, req);
else
LOG_ERROR(-1, "Unknown object type for cache removal");
Fixed Show fixed Hide fixed
}

static herr_t create_cache(void *obj, void *arg, void **req) {
Expand All @@ -511,6 +517,8 @@
return t->create_file_cache(obj, arg, req);
else if (o->obj_type == H5I_DATASET)
return t->create_dataset_cache(obj, arg, req);
else
LOG_ERROR(-1, "Unknown object type for cache creation");
Fixed Show fixed Hide fixed
}
/*******************/
/* Local variables */
Expand Down Expand Up @@ -770,19 +778,20 @@
LOG_WARN(-1, "Close request is NULL.");
}
#ifndef NDEBUG
LOG_DEBUG(-1, "async task finished %d", task->type);
snprintf(log_buffer, LOG_BUFFER_SIZE, "async task finished %d", task->type);
LOG_DEBUG(-1, "%s", log_buffer);
double t1 = MPI_Wtime();
LOG_DEBUG(-1,
"Delay closed object: %d time: "
"%10.6f",
task->type, t1 - t0);
snprintf(log_buffer, LOG_BUFFER_SIZE, "Delay closed object: %d time: %10.6f",
task->type, t1 - t0);
LOG_DEBUG(-1, "%s", log_buffer);
#endif
if (o->read_cache || o->write_cache)
o->H5LS->cache_io_cls->remove_cache(task->obj, NULL);
H5VL_cache_ext_free_obj(o);
#ifndef NDEBUG
double t2 = MPI_Wtime();
LOG_DEBUG(-1, "Remove cache time: %10.6f", t2 - t1);
snprintf(log_buffer, LOG_BUFFER_SIZE, "Remove cache time: %10.6f", t2 - t1);
LOG_DEBUG(-1, "%s", log_buffer);
#endif
free(task->req);
return 0;
Expand Down Expand Up @@ -1422,11 +1431,22 @@

LOG_INFO(-1, " storage path: %s", p->H5LS->path);

LOG_INFO(-1, " storage size: %.4f GiB",
p->H5LS->mspace_total / 1024. / 1024. / 1024.);
int ret =
snprintf(log_buffer, LOG_BUFFER_SIZE, " storage size: %.4f GiB",

Check failure

Code scanning / CodeQL

Unbounded write Critical

This 'call to sprintf' with input from
string read by fgets
may overflow the destination.
p->H5LS->mspace_total / 1024. / 1024. / 1024.);
if (ret < 0 || ret >= LOG_BUFFER_SIZE) {
LOG_WARN(-1, "Log Error when formatting storage size message");
} else {
LOG_INFO(-1, "%s", log_buffer);
}

LOG_INFO(-1, " write buffer size: %.4f GiB",
p->H5LS->write_buffer_size / 1024. / 1024. / 1024.);
ret = snprintf(log_buffer, LOG_BUFFER_SIZE, " write buffer size: %.4f GiB",

Check failure

Code scanning / CodeQL

Unbounded write Critical

This 'call to sprintf' with input from
string read by fgets
may overflow the destination.
p->H5LS->write_buffer_size / 1024. / 1024. / 1024.);
if (ret < 0 || ret >= LOG_BUFFER_SIZE) {
LOG_WARN(-1, "Log Error when formatting write buffer size message");
} else {
LOG_INFO(-1, "%s", log_buffer);
}

LOG_INFO(-1, " storage type: %s", p->H5LS->type);

Expand Down Expand Up @@ -2539,11 +2559,12 @@
}
H5VL_request_status_t status;
#ifndef NDEBUG
LOG_DEBUG(-1,
"request wait(jobid: %d), current available space: "
"%.5f GiB ",
o->H5DWMM->io->current_request->id,
o->H5DWMM->cache->mspace_per_rank_left / 1024. / 1024. / 1024);
snprintf(log_buffer, LOG_BUFFER_SIZE,
"request wait(jobid: %d), current available space: "
"%.5f GiB ",
o->H5DWMM->io->current_request->id,
o->H5DWMM->cache->mspace_per_rank_left / 1024. / 1024. / 1024);
LOG_DEBUG(-1, "%s", log_buffer);
#endif
while ((o->H5DWMM->io->current_request != NULL &&
o->H5DWMM->io->current_request->req != NULL)) {
Expand Down Expand Up @@ -2627,8 +2648,7 @@
// nearby write requests.
t_com->id = r->id;
#ifndef NDEBUG

LOG_DEBUG(-1, "Merging %d tasks (%d - %d) ", ntasks, t_com->id,
LOG_DEBUG(-1, "Merging %d tasks (%d - %d)", ntasks, t_com->id,
t_com->id + ntasks - 1);

#endif
Expand Down Expand Up @@ -2657,8 +2677,8 @@
free(t_com);
double t1 = MPI_Wtime();
#ifndef NDEBUG
LOG_DEBUG(-1, "Merging time: %6.5f", t1 - t0);

snprintf(log_buffer, LOG_BUFFER_SIZE, "Merging time: %6.5f", t1 - t0);
LOG_DEBUG(-1, "%s", log_buffer);
#endif
return SUCCEED;
}
Expand Down Expand Up @@ -3066,8 +3086,10 @@
}
double t1 = MPI_Wtime();
#ifndef NDEBUG
LOG_DEBUG(-1, "H5VLreqeust_wait time (jobid: %d): %f",
o->H5DWMM->io->current_request->id, t1 - t0);
snprintf(log_buffer, LOG_BUFFER_SIZE,
"H5VLreqeust_wait time (jobid: %d): %g",
o->H5DWMM->io->current_request->id, t1 - t0);
LOG_DEBUG(-1, "%s", log_buffer);

LOG_DEBUG(-1, "Tasks %d(%ld merged) finished",
o->H5DWMM->io->current_request->id,
Expand Down Expand Up @@ -3218,11 +3240,11 @@
p->async_close_task_list->obj = NULL;
double t1 = MPI_Wtime();
#ifndef NDEBUG

LOG_DEBUG(-1,
"dataset close time: "
"%.6f seconds",
t1 - t0);
snprintf(log_buffer, LOG_BUFFER_SIZE,
"dataset close time: "
"%.6f seconds",
t1 - t0);
LOG_DEBUG(-1, "%s", log_buffer);

#endif
return ret_value;
Expand All @@ -3234,10 +3256,11 @@
double t1 = MPI_Wtime();
#ifndef NDEBUG

LOG_DEBUG(-1,
"dataset remove cache time (including wait time): "
"%.6f seconds",
t1 - t0);
snprintf(log_buffer, LOG_BUFFER_SIZE,
"dataset remove cache time (including wait time): "
"%.6f seconds",
t1 - t0);
LOG_DEBUG(-1, "%s", log_buffer);

#endif
}
Expand All @@ -3259,7 +3282,9 @@
H5VL_cache_ext_free_obj(o);
double tt1 = MPI_Wtime();
#ifndef NDEBUG
LOG_DEBUG(-1, "H5VL_cache_ext_dataset_close time: %.6f seconds", tt1 - tt0);
snprintf(log_buffer, LOG_BUFFER_SIZE,
"H5VL_cache_ext_dataset_close time: %.6f seconds", tt1 - tt0);
LOG_DEBUG(-1, "%s", log_buffer);

#endif
return ret_value;
Expand Down Expand Up @@ -5649,15 +5674,30 @@

if (dset->H5LS->path != NULL) {
strcpy(dset->H5DRMM->cache->path, p->H5DRMM->cache->path); // create
strcat(dset->H5DRMM->cache->path, "/");
strcat(dset->H5DRMM->cache->path, name);
strcat(dset->H5DRMM->cache->path, "/");
strcpy(dset->H5DRMM->mmap->fname, dset->H5DRMM->cache->path);
strcat(dset->H5DRMM->mmap->fname, "/dset-mmap-");
strncat(dset->H5DRMM->cache->path, "/",
sizeof(dset->H5DRMM->cache->path) -
strlen(dset->H5DRMM->cache->path) - 1);
strncat(dset->H5DRMM->cache->path, name,
sizeof(dset->H5DRMM->cache->path) -
strlen(dset->H5DRMM->cache->path) - 1);
strncat(dset->H5DRMM->cache->path, "/",
sizeof(dset->H5DRMM->cache->path) -
strlen(dset->H5DRMM->cache->path) - 1);
strncpy(dset->H5DRMM->mmap->fname, dset->H5DRMM->cache->path,
sizeof(dset->H5DRMM->mmap->fname) - 1);
dset->H5DRMM->mmap->fname[sizeof(dset->H5DRMM->mmap->fname) - 1] =
'\0'; // Ensure null-termination
strncat(dset->H5DRMM->mmap->fname, "/dset-mmap-",
sizeof(dset->H5DRMM->mmap->fname) -
strlen(dset->H5DRMM->mmap->fname) - 1);
char cc[255];
int2char(dset->H5DRMM->mpi->rank, cc);
strcat(dset->H5DRMM->mmap->fname, cc);
strcat(dset->H5DRMM->mmap->fname, ".dat");
strncat(dset->H5DRMM->mmap->fname, cc,
sizeof(dset->H5DRMM->mmap->fname) -
strlen(dset->H5DRMM->mmap->fname) - 1);
strncat(dset->H5DRMM->mmap->fname, ".dat",
sizeof(dset->H5DRMM->mmap->fname) -
strlen(dset->H5DRMM->mmap->fname) - 1);
#ifndef NDEBUG

LOG_DEBUG(-1, "Dataset read cache created: %s",
Expand Down Expand Up @@ -5735,9 +5775,15 @@
memcpy(group->H5DRMM->mpi, o->H5DRMM->mpi, sizeof(MPI_INFO));
if (group->H5LS->path != NULL) {
strcpy(group->H5DRMM->cache->path, o->H5DRMM->cache->path); // create
strcat(group->H5DRMM->cache->path, "/");
strcat(group->H5DRMM->cache->path, name);
strcat(group->H5DRMM->cache->path, "/");
size_t remaining_size = sizeof(group->H5DRMM->cache->path) -
strlen(group->H5DRMM->cache->path) - 1;
strncat(group->H5DRMM->cache->path, "/", remaining_size);
remaining_size = sizeof(group->H5DRMM->cache->path) -
strlen(group->H5DRMM->cache->path) - 1;
strncat(group->H5DRMM->cache->path, name, remaining_size);
remaining_size = sizeof(group->H5DRMM->cache->path) -
strlen(group->H5DRMM->cache->path) - 1;
strncat(group->H5DRMM->cache->path, "/", remaining_size);
#ifndef NDEBUG
LOG_DEBUG(-1, "group cache created: %s", group->H5DRMM->cache->path);
#endif
Expand Down Expand Up @@ -5790,7 +5836,8 @@
H5VL_cache_ext_dataset_wait(dset);
double t1 = MPI_Wtime();
#ifndef NDEBUG
LOG_DEBUG(-1, "dataset_wait time: %f", t1 - t0);
snprintf(log_buffer, LOG_BUFFER_SIZE, "dataset_wait time: %f", t1 - t0);
LOG_DEBUG(-1, "%s", log_buffer);
#endif
o->H5DWMM = NULL;
}
Expand Down
Loading