Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DAOS-16381 test: Run IOR with HDF5-VOL with multiple object classes #14964

Merged
merged 1 commit into from
Aug 21, 2024

Conversation

shimizukko
Copy link
Contributor

Currently, the logic in util/file_count_test_base.py runs IOR with HDF5-VOL with only one object class. The second (and beyond) object class will not run with HDF5-VOL (it runs as a normal IOR). Update line 120 to:

if api == 'HDF5-VOL':

so that if multiple object classes are defined in the test yaml, all of them will run with HDF5-VOL.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount

Before requesting gatekeeper:

  • Two review approvals and any prior change requests have been resolved.
  • Testing is complete and all tests passed or there is a reason documented in the PR why it should be force landed and forced-landing tag is set.
  • Features: (or Test-tag*) commit pragma was used or there is a reason documented that there are no appropriate tags for this PR.
  • Commit messages follows the guidelines outlined here.
  • Any tests skipped by the ticket being addressed have been run and passed in the PR.

Gatekeeper:

  • You are the appropriate gatekeeper to be landing the patch.
  • The PR has 2 reviews by people familiar with the code, including appropriate owners.
  • Githooks were used. If not, request that user install them and check copyright dates.
  • Checkpatch issues are resolved. Pay particular attention to ones that will show up on future PRs.
  • All builds have passed. Check non-required builds for any new compiler warnings.
  • Sufficient testing is done. Check feature pragmas and test tags and that tests skipped for the ticket are run and now pass with the changes.
  • If applicable, the PR has addressed any potential version compatibility issues.
  • Check the target branch. If it is master branch, should the PR go to a feature branch? If it is a release branch, does it have merge approval in the JIRA ticket.
  • Extra checks if forced landing is requested
    • Review comments are sufficiently resolved, particularly by prior reviewers that requested changes.
    • No new NLT or valgrind warnings. Check the classic view.
    • Quick-build or Quick-functional is not used.
  • Fix the commit message upon landing. Check the standard here. Edit it to create a single commit. If necessary, ask submitter for a new summary.

Currently, the logic in util/file_count_test_base.py runs IOR with HDF5-VOL
with only one object class. The second (and beyond) object class will not
run with HDF5-VOL (it runs as a normal IOR). Update line 120 to:

if api == 'HDF5-VOL':

so that if multiple object classes are defined in the test yaml, all of them
will run with HDF5-VOL.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount

Signed-off-by: Makito Kano <makito.kano@intel.com>
Copy link

Ticket title is 'Run IOR with HDF5-VOL with multiple object classes'
Status is 'Open'
https://daosio.atlassian.net/browse/DAOS-16381

@shimizukko
Copy link
Contributor Author

shimizukko commented Aug 20, 2024

  • io_sys_admin.py ended with WARN as expected.
  • Checked that both object classes (SX and EC_2P1GX) had the env vars (-genv HDF5_VOL_CONNECTOR=daos -genv HDF5_PLUGIN_PATH=/usr/lib64/mpich/lib) set to the mpirun command.

@shimizukko shimizukko marked this pull request as ready for review August 20, 2024 04:46
@shimizukko shimizukko requested review from a team as code owners August 20, 2024 04:46
@shimizukko shimizukko requested a review from a team August 20, 2024 20:24
@daltonbohning daltonbohning added the forced-landing The PR has known failures or has intentionally reduced testing, but should still be landed. label Aug 21, 2024
@daltonbohning daltonbohning merged commit 4361a3e into master Aug 21, 2024
48 checks passed
@daltonbohning daltonbohning deleted the makito/DAOS-16381 branch August 21, 2024 17:56
@daltonbohning
Copy link
Contributor

Thanks - please backport

shimizukko added a commit that referenced this pull request Aug 22, 2024
…14964)

Currently, the logic in util/file_count_test_base.py runs IOR with HDF5-VOL
with only one object class. The second (and beyond) object class will not
run with HDF5-VOL (it runs as a normal IOR). Update line 120 to:

if api == 'HDF5-VOL':

so that if multiple object classes are defined in the test yaml, all of them
will run with HDF5-VOL.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount
Signed-off-by: Makito Kano <makito.kano@intel.com>
@mchaarawi
Copy link
Contributor

I am not sure what this PR is doing. what do you mean different object classes for HDF5-VOL?
to change the object classes for the VOL, we need to set env variable "HDF5_DAOS_OBJ_CLASS"
where is that being set?

@daltonbohning
Copy link
Contributor

I am not sure what this PR is doing. what do you mean different object classes for HDF5-VOL? to change the object classes for the VOL, we need to set env variable "HDF5_DAOS_OBJ_CLASS" where is that being set?

This is basically what happens now

for api in ['HDF5-VOL', 'DFS', ...]:
    for oclass in [oclass1, oclass2]:
        if api == 'HDF5-VOL':
            self.ior_cmd.api.update('HDF5')
            # set plugin path, env, etc.

So we run in total

HDF5 + VOL, oclass1
HDF5 + VOL, oclass2
DFS, oclass1
DFS, oclass2

Previously the if api check was wrong so we were running

HDF5 + VOL, oclass1
HDF5, oclass2  # without VOL!
DFS, oclass1
DFS, oclass2

@mchaarawi
Copy link
Contributor

HDF5 + VOL, oclass1
HDF5 + VOL, oclass2

my question is about this.. i do not see where oclass1 and oclass2 are applied for HDF5+VOL

@daltonbohning
Copy link
Contributor

HDF5 + VOL, oclass1
HDF5 + VOL, oclass2

my question is about this.. i do not see where oclass1 and oclass2 are applied for HDF5+VOL

It's not obvious. A test config will have this for example

largefilecount:
api:
- MPIIO
- POSIX
- HDF5
- HDF5-VOL
- DFS
object_class:
ior_oclass: # Run once with SX and then with EC_16P2GX
- SX
- EC_2P1GX
mdtest_oclass: # Run once with S1 and then with EC_16P2G1
- S1
- EC_2P1G1

And the code loops over the apis


And then the oclasses
for oclass in ior_oclass:

Most of the implementations details are scattered across a few magic functions

@mchaarawi
Copy link
Contributor

HDF5 + VOL, oclass1
HDF5 + VOL, oclass2

my question is about this.. i do not see where oclass1 and oclass2 are applied for HDF5+VOL

It's not obvious. A test config will have this for example

largefilecount:
api:
- MPIIO
- POSIX
- HDF5
- HDF5-VOL
- DFS
object_class:
ior_oclass: # Run once with SX and then with EC_16P2GX
- SX
- EC_2P1GX
mdtest_oclass: # Run once with S1 and then with EC_16P2G1
- S1
- EC_2P1G1

And the code loops over the apis

And then the oclasses

for oclass in ior_oclass:

Most of the implementations details are scattered across a few magic functions

let me clarify my question / concern.
HDF5+VOL is not like HDF5 or DFS or MPIIO or POSIX, where you set the oclass on the POSIX container create for ior. HDF5+VOL creates its own container. the POSIX container that i believe you think is being used is just for storing a stub file for the namespace and changing oclass or props on that container has no effect on the VOL container.

If you do not set "HDF5_DAOS_OBJ_CLASS" or "HDF5_DAOS_FILE_PROP" (the first for object class, the latter is for a string of container properties), the VOL is just using its defaults, RF0, S1 for groups, SX for datasets.

So what im saying is that these 2 combinations you posted:
HDF5 + VOL, oclass1
HDF5 + VOL, oclass2

are not doing what this test thinks its doing and probably running the same configuration twice

@mchaarawi
Copy link
Contributor

ultimately, maybe this PR is OK, but the result is probably not what you are expecting to test (before or after this PR)

@daltonbohning
Copy link
Contributor

If you do not set "HDF5_DAOS_OBJ_CLASS" or "HDF5_DAOS_FILE_PROP" (the first for object class, the latter is for a string of container properties), the VOL is just using its defaults, RF0, S1 for groups, SX for datasets.

I see. We don't set those vars anywhere :(

daltonbohning pushed a commit that referenced this pull request Aug 27, 2024
…14964) (#14982)

Currently, the logic in util/file_count_test_base.py runs IOR with HDF5-VOL
with only one object class. The second (and beyond) object class will not
run with HDF5-VOL (it runs as a normal IOR). Update line 120 to:

if api == 'HDF5-VOL':

so that if multiple object classes are defined in the test yaml, all of them
will run with HDF5-VOL.

Signed-off-by: Makito Kano <makito.kano@intel.com>
daltonbohning pushed a commit that referenced this pull request Sep 5, 2024
#15004)

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from container -> properties field in the test yaml.

This fix is discussed in PR #14964.

Signed-off-by: Makito Kano <makito.kano@intel.com>
shimizukko added a commit that referenced this pull request Sep 8, 2024
#15004)

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from self.container.properties.value.

This fix is discussed in PR #14964.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount
Signed-off-by: Makito Kano <makito.kano@intel.com>
daltonbohning pushed a commit that referenced this pull request Sep 9, 2024
#15004) #15098

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from self.container.properties.value.

This fix is discussed in PR #14964.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount
Required-githooks: true

Signed-off-by: Makito Kano <makito.kano@intel.com>
daltonbohning pushed a commit that referenced this pull request Sep 9, 2024
#15004) #15098

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from self.container.properties.value.

This fix is discussed in PR #14964.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount
Required-githooks: true

Signed-off-by: Makito Kano <makito.kano@intel.com>
mchaarawi added a commit that referenced this pull request Sep 11, 2024
* DAOS-16484 test: Exclude local host in default interface selection (#15049)

When including the local host in the default interface selection a
difference in ib0 speeds will cause the logic to select eth0 and then
the tcp provider.

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>

* DAOS-15800 client: create cart context on specific interface (#14804)

Cart has added the ability to select network interface on context creation. The daos_agent also added a numa-fabric map that can be queried at init time. Update the DAOS client to query from the agent a map of numa to network interface on daos_init(), and on EQ creation, select the best interface for the network context based on the numa of the calling thread.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

* DAOS-16445 client: Add function to cycle OIDs non-sequentially (#14999)

We've noticed that with sequential order, object placement is poor.

We get 40% fill for 8GiB files with 25 ranks and 16 targets per rank
with EC_2P1G8. With this patch, we get a much better distribution.

This patch adds the following:

1. A function for cycling oid.hi incrementing by a large prime
2. For DFS, randomize the starting value
3. Modify DFS to cycle OIDs using the new function.

Signed-off-by: Jeff Olivier <jeffolivier@google.com>

* DAOS-16251 dtx: Fix dtx_req_send user-after-free (#15035)

In dtx_req_send, since the crt_req_send releases the req reference, din
may have been freed when dereferenced for the DL_CDEBUG call.

Signed-off-by: Li Wei <wei.g.li@intel.com>

* DAOS-16304 tools: Add daos health net-test command (#14980)

Wrap self_test to provide a simplified network test
to detect obvious client/server connectivity and
performance problems.

Signed-off-by: Michael MacDonald <mjmac@google.com>

* DAOS-16272 dfs: fix get_info returning incorrect oclass (#15048)

If user creates a container without --file-oclass, the get_info call was
returning the default oclass of a directory on daos fs get-attr. Fix
that to properly use the enum types for default scenario.

Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>

* DAOS-15863 container: fix a race for container cache (#15038)

* DAOS-15863 container: fix a race for container cache

while destroying a container, cont_child_destroy_one() releases
its own refcount before waiting, if another ULT releases its
refcount, which is the last one, wakes up the waiting ULT and frees
it ds_cont_child straightaway, because no one else has refcount.

When the waiting ULT is waken up, it will try to change the already
freed ds_cont_child.

This patch changes the LRU eviction logic and fixes this race.


Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
Co-authored-by: Jeff Olivier <jeffolivier@google.com>

* DAOS-16471 test: Reduce targets for ioctl_pool_handles.py (#15063)

The dfuse/ioctl_pool_handles.py test is overloading the VM so reduce the number of engine targets.

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>

* DAOS-16483 vos: handle empty DTX when vos_tx_end (#15053)

It is possible that the DTX modified nothing when stop currnet backend
transaction. Under such case, we may not generate persistent DTX entry.
Then need to bypass such case before checking on-disk DTX entry status.

The patch makes some clean and removed redundant metrics for committed
DTX entries.

Enhance vos_dtx_deregister_record() to handle GC case.

Signed-off-by: Fan Yong <fan.yong@intel.com>

* DAOS-16271 mercury: Add patch to avoid seg fault in key resolve. (#15067)

Signed-off-by: Joseph Moore <joseph.moore@intel.com>

* DAOS-16484 test: Support mixed speeds when selecting a default interface (#15050)

Allow selecting a default interface that is running at a different speed
on different hosts.  Primarily this is to support selecting the ib0
interface by default when the launch node has a slower ib0 interface
than the cluster hosts.

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>

* DAOS-16446 test: HDF5-VOL test - Set object class and container prope… (#15004)

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from container -> properties field in the test yaml.

This fix is discussed in PR #14964.

Signed-off-by: Makito Kano <makito.kano@intel.com>

* DAOS-16447 test: set D_IL_REPORT per test (#15012)

set D_IL_REPORT per test instead of setting defaults values in
utilities. This allows running without it set.

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

* DAOS-16450 test: auto run dfs tests when dfs is modified (#15017)

Automatically include dfs tests when dfs files are modified in PRs.

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

* DAOS-16510 cq: update pylint to 3.2.7 (#15072)

update pylint to 3.2.7

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

* DAOS-16509 test: replace IorTestBase.execute_cmd with run_remote (#15070)

replace usage of IorTestBase.execute_cmd with run_remote

Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>

* DAOS-16458 object: fix invalid DRAM access in obj_bulk_transfer (#15026)

For EC object update via CPD RPC, when calculate the bitmap to skip
some iods for current EC data shard, we may input NULL for "*skips"
parameter. It may cause the old logic in obj_get_iods_offs_by_oid()
to generate some undefined DRAM for "skips" bitmap. Such bitmap may
be over-written by others, as to subsequent obj_bulk_transfer() may
be misguided.

The patch also fixes a bug inside obj_bulk_transfer() that cast any
input RPC as UPDATE/FETCH by force.

Signed-off-by: Fan Yong <fan.yong@intel.com>

* DAOS-16486 object: return proper error on stale pool map (#15064)

Client with stale pool map may try to send RPC to a DOWN target, if the
target was brought DOWN due to faulty NVMe device, the ds_pool_child could
have been stopped on the NVMe faulty reaction, We'd ensure proper error
code is returned for such case.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>

* DAOS-16514 vos: fix coverity issue (#15083)

Fix coverity 2555843 explict null dereferenced.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>

* DAOS-16467 rebuild: add DAOS_POOL_RF ENV for massive failure case (#15037)

* DAOS-16467 rebuild: add DAOS_PW_RF ENV for massive failure case

Allow user to set DAOS_PW_RF as pw_rf (pool wise RF).
If SWIM detected engine failure is going to break pw_rf, don't change
pool map, also don't trigger rebuild.
With critical log message to ask administrator to bring back those
engines in top priority (just "system start --ranks=xxx", need not to
reintegrate those engines).

a few functions renamed to avoid confuse -
pool_map_find_nodes() -> pool_map_find_ranks()
pool_map_find_node_by_rank() -> pool_map_find_dom_by_rank()
pool_map_node_nr() -> pool_map_rank_nr()

Signed-off-by: Xuezhao Liu <xuezhao.liu@intel.com>

* DAOS-16508 csum: retry a few times on checksum mismatch on update (#15069)

Unlike fetch, we return DER_CSUM on update (turned into EIO by dfs) without any retry.
We should retry a few times in case it is a transient error.

The patch also prints more information about the actual checksum mismatch.

Signed-off-by: Johann Lombardi <johann.lombardi@gmail.com>

* DAOS-10877 vos: gang allocation for huge SV (#14790)

To avoid allocation failure on a fragmented system, huge SV allocation will
be split into multiple smaller allocations, each allocation size is capped
to 8MB (the DMA chunk size, that could avoid huge DMA buffer allocation).

The address of such scattered SV payload is represented by 'gang address'.

Removed io_allocbuf_failure() vos unit test, it's not applicable in gang
SV mode now.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>

* DAOS-16304 tools: Adjust default RPC size for net-test (#15091)

The previous default of 1MiB isn't helpful at large scales.
Use a default of 1KiB to get faster results and a better
balance between raw latency and bandwidth.

Also include calculated rpc throughput and bandwidth in
JSON output.

Signed-off-by: Michael MacDonald <mjmac@google.com>

* SRE-2408 ci: Increase timeout (to 15 minutes) for system restore (#14926)

Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@intel.com>

* DAOS-16251 object: Fix obj_ec_singv_split overflow (#15045)

It has been seen that obj_ec_singv_split may read beyond the end of
sgl->sg_iovs[0].iov_buf:

    iod_size=8569
    c_bytes=4288
    id_shard=0
    tgt_off=1
    iov_len=8569
    iov_buf_len=8569

The memmove read 4288 bytes from offset 4288, whereas the buffer only
had 8569 - 4288 = 4281 bytes from offset 4288. This patch fixes the
problem by adding the min(...) expression.

Signed-off-by: Li Wei <wei.g.li@intel.com>

---------

Signed-off-by: Phil Henderson <phillip.henderson@intel.com>
Signed-off-by: Mohamad Chaarawi <mohamad.chaarawi@intel.com>
Signed-off-by: Jeff Olivier <jeffolivier@google.com>
Signed-off-by: Li Wei <wei.g.li@intel.com>
Signed-off-by: Michael MacDonald <mjmac@google.com>
Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: Fan Yong <fan.yong@intel.com>
Signed-off-by: Joseph Moore <joseph.moore@intel.com>
Signed-off-by: Makito Kano <makito.kano@intel.com>
Signed-off-by: Dalton Bohning <dalton.bohning@intel.com>
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Signed-off-by: Xuezhao Liu <xuezhao.liu@intel.com>
Signed-off-by: Johann Lombardi <johann.lombardi@gmail.com>
Signed-off-by: Tomasz Gromadzki <tomasz.gromadzki@intel.com>
Co-authored-by: Phil Henderson <phillip.henderson@intel.com>
Co-authored-by: Jeff Olivier <jeffolivier@google.com>
Co-authored-by: Li Wei <wei.g.li@intel.com>
Co-authored-by: Michael MacDonald <mjmac@google.com>
Co-authored-by: Liang Zhen <liang.zhen@intel.com>
Co-authored-by: Nasf-Fan <fan.yong@intel.com>
Co-authored-by: Joseph Moore <26410038+jgmoore-or@users.noreply.github.com>
Co-authored-by: Makito Kano <makito.kano@intel.com>
Co-authored-by: Dalton Bohning <dalton.bohning@intel.com>
Co-authored-by: Niu Yawei <yawei.niu@intel.com>
Co-authored-by: Liu Xuezhao <xuezhao.liu@intel.com>
Co-authored-by: Johann Lombardi <johann.lombardi@gmail.com>
Co-authored-by: Tomasz Gromadzki <tomasz.gromadzki@intel.com>
daltonbohning pushed a commit that referenced this pull request Sep 24, 2024
#15004) #15098

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from self.container.properties.value.

This fix is discussed in PR #14964.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount
Required-githooks: true

Signed-off-by: Makito Kano <makito.kano@intel.com>
daltonbohning pushed a commit that referenced this pull request Oct 4, 2024
#15004) #15098

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from self.container.properties.value.

This fix is discussed in PR #14964.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount
Required-githooks: true

Signed-off-by: Makito Kano <makito.kano@intel.com>
daltonbohning pushed a commit that referenced this pull request Oct 10, 2024
#15004) #15098

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from self.container.properties.value.

This fix is discussed in PR #14964.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium-md-on-ssd: false
Test-tag: test_io_sys_admin test_largefilecount test_smallfilecount
Required-githooks: true

Signed-off-by: Makito Kano <makito.kano@intel.com>
daltonbohning pushed a commit that referenced this pull request Oct 16, 2024
#15004) (#15098)

In HDF5, DFS, MPIIO, or POSIX, object class and container properties are defined
during the container create. If it’s DFS, object class is also set to the IOR
parameter. However, in HDF5-VOL, object class and container properties are
defined with the following environment variables of mpirun.

HDF5_DAOS_OBJ_CLASS (Object class)
HDF5_DAOS_FILE_PROP (Container properties)

The infrastructure to set these variables are already there in run_ior_with_pool().
In file_count_test_base.py, pass in the env vars to run_ior_with_pool(env=env) as a
dictionary. Object class is the oclass variable. Container properties can be
obtained from self.container.properties.value.

This fix is discussed in PR #14964.

Signed-off-by: Makito Kano <makito.kano@intel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
forced-landing The PR has known failures or has intentionally reduced testing, but should still be landed.
Development

Successfully merging this pull request may close these issues.

5 participants