[pull] master from axboe:master #314
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![pull[bot]](https://avatars.githubusercontent.com/in/12910?s=60&v=4){kind=small}
If any error is observed for read requests, skip all end to end data protection checks. Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com> Link: https://lore.kernel.org/r/20231212144718.568406-1-ankit.kumar@samsung.com Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
On FreeBSD, write access to Rank 1 geom providers is disabled by default. However, it can be enabled with the `kern.geom.debugflags` sysctl as documented in geom(4). Point users to that manual page for smoother experience. Signed-off-by: Mateusz Piotrowski 0mp@FreeBSD.org
* 'patch-3' of https://github.com/0mp/fio: doc: Reference geom(4) for FreeBSD users
Found by Red Hat's OpenScanHub: fio-3.35/engines/http.c:253: leaked_storage: Variable r going out of scope leaks the storage it points to. Signed-off-by: Pavel Reichl <preichl@redhat.com>
* 'master' of https://github.com/preichl/fio: engines/http: Fix memory leak
The issues was found by Red Hat's OpenScanHub:
fio-3.35/engines/rdma.c:279:3: warning[deadcode.DeadStores]:
Value stored to 'ret' is never read
Signed-off-by: Pavel Reichl <preichl@redhat.com>
The issue was found by Red Hat's OpenScanHub:
fio-3.35/server.c:1884:3: warning[deadcode.DeadStores]:
Value stored to 'extended_buf_wp' is never read
Signed-off-by: Pavel Reichl <preichl@redhat.com>
Signed-off-by: Pavel Reichl <preichl@redhat.com>
* 'master' of https://github.com/preichl/fio: engines/http: Drop unused varible client/server: remove dead code engines/rdma: remove dead code
The engine sets -1 for some odd reason, where ->error fields are supposed to be a positive value. Set it to EIO. Signed-off-by: Jens Axboe <axboe@kernel.dk>
Since the commit fb0259f ("zbd: Ensure first I/O is write for random read/write to sequential zones"), fio issues write as the first I/O when zonemode=zbd and rw=randrw options are specified. However, fio issues the first write even when rwmixwrite=0 option is specified. Users do not expect such write and it confuses the users. To avoid the confusion, suppress the write by referring td->o.rwmix[DDIR_WRITE]. Fixes: fb0259f ("zbd: Ensure first I/O is write for random read/write to sequential zones") Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com> Reviewed-by: Damien Le Moal <dlemoal@kernel.org> Link: https://lore.kernel.org/r/20231220005846.1371456-2-shinichiro.kawasaki@wdc.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
The previous commit fixed the issue of the unexpected write with options zonemode=zbd, rw=randrw and rwmixwrite=0. Add a test to confirm the fix. Signed-off-by: Shin'ichiro Kawasaki <shinichiro.kawasaki@wdc.com> Reviewed-by: Damien Le Moal <dlemoal@kernel.org> Link: https://lore.kernel.org/r/20231220005846.1371456-3-shinichiro.kawasaki@wdc.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
Make sure we call the parent class' check_result() method to check the return code, stderr output, etc. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Make sure we call the parent class' check_result() method to check the return code, stderr output, etc. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Make sure we call the parent class' check_result() method to check the return code, stderr output, etc. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
check_result() erroneously returned True or False. Drop the return value to be consistent with the parent and related classes. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Use tabs consistently for the paragraphs describing the group_reporting option. Also, make sure that there is no space between ":option:" and "`group_reporting`". Signed-off-by: Mateusz Piotrowski <0mp@FreeBSD.org>
* 'group_reporting_indentation' of https://github.com/0mp/fio: doc: group_reporting: Fix indentation and syntax
Link: #1701 Signed-off-by: Jens Axboe <axboe@kernel.dk>
GitHub increased the number of CPUs in its GitHub-hosted runners from two to four for Linux and Windows. macOS remains at two CPUs. Stop hard-coding the number of CPUs when we build fio and detect the number at runtime. https://github.blog/2024-01-17-github-hosted-runners-double-the-power-for-open-source/ Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
If verify_only is set we don't need to open the file with the O_RDWR flagi for write workloads. So we should clear this flag. This will help when the file is on a read-only file system. Fixes: #1681 Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
pkg-config doesn't need to have a cross prefixed version. It can be
pointed at alternate sysroots with environment variables like
PKG_CONFIG_SYSROOT_DIR. The configure script was already inconsistent
with using `pkg-config` in some places and `${cross_prefix}pkg-config`
in others. Make check_min_lib_version() and the gtk checks consistent
with the rest by dropping the `${cross_prefix}` usage with pkg-config.
Signed-off-by: Chris Packham <chris.packham@alliedtelesis.co.nz>
Link: https://lore.kernel.org/r/20240123202636.179467-1-chris.packham@alliedtelesis.co.nz
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Switch from actions/checkout@v3 to v4 and from actions/upload-artifacts@v3 to v4. This resolves the below warnings from GitHub Actions: Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/checkout@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/. Node.js 16 actions are deprecated. Please update the following actions to use Node.js 20: actions/upload-artifact@v3. For more information see: https://github.blog/changelog/2023-09-22-github-actions-transitioning-from-node-16-to-node-20/ Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Add option log_window_value alias of log_max_value which reports average, max or both the values. Retain backward compatibility by allowing =0 and =1 values to specify avg and max values respectively. There is no change to existing log formats while reporting only average or max values. Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com> Link: https://lore.kernel.org/r/20240125110124.55137-2-ankit.kumar@samsung.com Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com> Link: https://lore.kernel.org/r/20240125110124.55137-3-ankit.kumar@samsung.com Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
This didn't make it into the kernel just yet, and if/when it does, it'll likely look a bit different than the prototype. Since it's dead code for now, just prune it and we can always reintroduce it when kernel support is there. Signed-off-by: Jens Axboe <axboe@kernel.dk>
Fix the two Null pointer dereferences issue reported by Coverity scan Null pointer dereferences (FORWARD_NULL) Dereferencing null pointer "l->td" Null pointer dereferences (REVERSE_INULL) Null-checking "p->td" suggests that it may be null, but it has already been dereferenced on all paths leading to the check. For aggregate read, write and trim bandwidth log, the setup_log function gets called with NULL pointer reference for thread data. Thus before dereferencing further we should check "l->td". Signed-off-by: Ankit Kumar <ankit.kumar@samsung.com>
Fix configure with --build-static option which enables static RDMA in Issue #1801 This pull request to probe static libibverbs/librdmacm library. It's neccessary to enable static RDMA engine build with netlink libnl-3 and libnl-route-3. Signed-off-by: Huang Shumin longway68@qq.com
We no longer need unzip, cmake, and wget because we no longer build librpma. Stop installing these three packages. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Some containers no longer have libunwind-14-dev in their repositories. Remove this package only if it is installed. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
We run our tests as root in containers and VMs. Those platforms do not have sudo. Use a small bash function to eliminate the need to install sudo at the very beginning of the dependency installation script. This function just checks for the existence of sudo and uses it when installed but does nothing when it is not installed. With this function we can just add sudo to the list of packages to install like all the other packages. We may still need sudo in one of our test scripts. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Since GitHub Actions' free runners now support nested virtualization, let's create a guest VM image and use it to run some tests that cannot be run directly via a GitHub Actions runner (because these runners do not allow insertion of kernel modules). This patch adds a workflow that runs nightly and: - creates a Debian guest VM image using libguestfs - builds QEMU - starts the VM - clones/builds fio on the VM - runs the test Separate instances of the job are spawned to run the following tests: - basic io_uring_cmd - NVMe protection information w/16-bit Guard PI - NVMe PI 4096+16 w/64-bit Guard PI - NVMe PI 4096+64 w/64-bit Guard PI - FDP - t/zbd/run-tests-against-nullb Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
__io_u_log_error expects a positive value of io_u->error parsing it with strerror() expecting it to be an errno. io_uring_cmd (cmd_type=nvme), for example, has returned errno value as a positive value and device-specific CQE status type and code as well. Commit 78831c6 ("io_uring: Fix the flip to negative of CQE status") has put the abs() to the cqe->res, and it caused confusions between the actual CQE stauts and the system error value (errno). Now we have Commit 2a13699 ("io_uring: Add .errdetails to parse CQ status"), meaning that io_uring_cmd ioengines will parse the actual value of io_u->error value as CQE status value, so we should know if the value is for CQE status or errno. This patch added a flag IO_U_F_DEVICE_ERROR to io_u to represent if io_u->error has device-specific error value, otherwise it's errno. Signed-off-by: Minwoo Im <minwoo.im.dev@gmail.com> Link: https://lore.kernel.org/r/20240926004134.26004-1-minwoo.im.dev@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
Added 'verify_mode' option to io_uring_cmd with --cmd_type=nvme to support data compare verification with Compare commands rather than Read commands. This patch newly added IO_U_F_VER_IN_DEV io_u flag to represent that verification should be done in device side, not the host side to skip the actual verification phase in verify_io_u(). Signed-off-by: Minwoo Im <minwoo.im.dev@gmail.com> Link: https://lore.kernel.org/r/20240926010802.27131-1-minwoo.im.dev@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
upload-artifacts v1 is now deprecated. Switch to a maintained version. We may also consider just getting rid of this workflow as it's not clear that it has ever found any issue or if anyone looks at the output. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
* 'master' of https://github.com/HuangShumin/fio: Fix configure with --build-static which enables static RDMA in #1801
It doesn't make any sense, for obvious reasons. Fixes: ebe67b6 ("io_uring: Add IO_U_F_DEVICE_ERROR to identify error types") Signed-off-by: Jens Axboe <axboe@kernel.dk>
'verify_mode' option has been added by Commit 6170d92 ("io_uring: Support Compare command for verification"). Added description for this option. Signed-off-by: Minwoo Im <minwoo.im.dev@gmail.com> Link: https://lore.kernel.org/r/20240926225411.13754-1-minwoo.im.dev@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
io_u->error can have either negative errno value or device-specific error status as a positive value. fio always tries to parse the errno with strerrno() and it expects the value to be a positive one. Commit ebe67b6 ("io_uring: Add IO_U_F_DEVICE_ERRROR to identify error types") tried to abs(io_u->error) to convert it first. And it caused the following build warning: engines/io_uring.c:553:16: error: taking the absolute value of unsigned type 'unsigned int' has no effect [-Werror,-Wabsolute-value] io_u->error = abs(io_u->error); ^ engines/io_uring.c:553:16: note: remove the call to 'abs' since unsigned values cannot be negative io_u->error = abs(io_u->error); ^~~ Commit 9eaa8e7 ("engines/io_uring: don't use abs() on an unsigned value") tried to remove the warning by removing abs() on io_u->error, but if so, negative errno (e.g., -EINVAL) can't be parsed properly like: fio: io_u error on file /dev/ng0n1: Unknown error -22: write offset=429916160, buflen=1048576 This patch fixed this fixed to convert to positive value properly by casting it first to int and then do abs() on it. Fixes: 9eaa8e7 ("engines/io_uring: don't use abs() on an unsigned value") Signed-off-by: Minwoo Im <minwoo.im.dev@gmail.com> Link: https://lore.kernel.org/r/20240927030857.17001-1-minwoo.im.dev@gmail.com Signed-off-by: Jens Axboe <axboe@kernel.dk>
This patch adds a 'workflow_dispatch' trigger to our three GitHub Actions workflows. This allows us to trigger each workflow by navigating to the repository's Actions tab, selecting the desired workflow, and clicking on th 'Run workflow' button. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Reduce confusion by giving the guest VM workflow a name different from our run-fio-tests.py workflow. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
It'll never change with the settings that t/io_uring uses to setup the ring - and even if it could, it's still cheaper to just read the tail once per reap. Signed-off-by: Jens Axboe <axboe@kernel.dk>
6.12 kernels and newer support async trim, which means the non-cmd path no longer needs to quiesce the queue and issue a sync trim for a workload that includes trim/discard requests. The engine will assume the support is there, and if it gets -EINVAL when trying an async trim, then it'll punt back to the using the sync trim again. Signed-off-by: Jens Axboe <axboe@kernel.dk>
Instead of faking a GitHub Actions runner environment, just create a source code tarball and transfer it to the guest VM. This is simpler and does not depend on the continued existence of the action used previously. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Mention the unique_filename option in the client/server section and also note that fio will produce aggregate summary output for all jobs when running jobs on multiple servers. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
The ?lat_percentile toggles and list of percentiles to report from the thread_stat payload have not been propagated to the overall summary report. Enable "All clients" latency percentile reporting by arbitrarily using the toggles and percentile list from the first thread_stat payload sent from the server. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
It does not make sense when group_reporting is used for the constituent jobs to have different values for options like unified_rw_reporting, ?lat_percentiles, etc. Mention that configurations with different values for these parameters are unsupported. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
io_uring needs the rings to be a power-of-2 in sizing, but it does not need to be messing with the queue depth driven to the device. Rather than round everything up, only round up the API parts. Reported-by: Riley Thomasson <riley@purestorage.com> Signed-off-by: Jens Axboe <axboe@kernel.dk>
For all of our custom local actions make sure we update the apt cache before trying to install packages. This will prevent errors when the runner is started with a stale apt cache. Example: https://github.com/vincentkfu/fio/actions/runs/11339008514/job/31533113877 Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Add an option for fio to set the DEAC bit for write zeroes commands. This instructs the device to also deallocate the specified logical blocks in the course of completing the write zeroes operation. This only has an effect when write_mode=zeroes is specified. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
For write zeroes commands, do not send a data bufffer address. If we do send an address, the driver will try to carry out DMA mapping and encounter driver mdts limitations which do not apply to write zeroes commands. This lets us send large write zeroes commands to deallocate the device with the deac=1. This is only done for the non-vectored IO case. For the vectored IO case, the driver requires data buffer addresses to be sent. Regardless it does not make sense to use vectored IO for write zeroes. Signed-off-by: Vincent Fu <vincent.fu@samsung.com>
Rather than register N buffers for depth of N, just allocate a single contig region and register that as a single buffer. That's more efficient on the kernel side, as only a single resource need to be managed. Should have no functional changes. Signed-off-by: Jens Axboe <axboe@kernel.dk>
Enable the -N NOP mode to use registered files and buffers, handy for testing the kernel side. Signed-off-by: Jens Axboe <axboe@kernel.dk>
Add them to io_uring.h and use them in t/io_uring. Signed-off-by: Jens Axboe <axboe@kernel.dk>
* io_uring-depth: engines/io_uring: don't mess with non power-of-2 queue depth
It should always be using the size passed in as to what amount of memory should be allocated, not the individual IO size. This didn't matter previously, as each buffer was allocated independently, but it matters now where a single region will cover all IO buffers. Signed-off-by: Jens Axboe <axboe@kernel.dk>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot]
Can you help keep this open source service alive? 💖 Please sponsor : )