Is this page helpful?

Runtime Settings#

Compatibility Mode KVIKIO_COMPAT_MODE#

When KvikIO is running in compatibility mode, it doesn’t load libcufile.so. Instead, reads and writes are done using POSIX. Notice, this is not the same as the compatibility mode in cuFile. It is possible that KvikIO performs I/O in the non-compatibility mode by using the cuFile library, but the cuFile library itself is configured to operate in its own compatibility mode. For more details, refer to cuFile compatibility mode and cuFile environment variables .

The environment variable KVIKIO_COMPAT_MODE has three options (case-insensitive):

  • ON (aliases: TRUE, YES, 1): Enable the compatibility mode.

  • OFF (aliases: FALSE, NO, 0): Disable the compatibility mode, and enforce cuFile I/O. GDS will be activated if the system requirements for cuFile are met and cuFile is properly configured. However, if the system is not suited for cuFile, I/O operations under the OFF option may error out.

  • AUTO: Try cuFile I/O first, and fall back to POSIX I/O if the system requirements for cuFile are not met.

Under AUTO, KvikIO falls back to the compatibility mode:

  • when libcufile.so cannot be found.

  • when running in Windows Subsystem for Linux (WSL).

  • when /run/udev isn’t readable, which typically happens when running inside a docker image not launched with --volume /run/udev:/run/udev:ro.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name compat_mode.

Thread Pool KVIKIO_NTHREADS#

KvikIO can use multiple threads for IO automatically. Set the environment variable KVIKIO_NTHREADS to the number of threads in the thread pool. If not set, the default value is 1.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name num_threads.

Task Size KVIKIO_TASK_SIZE#

KvikIO splits parallel IO operations into multiple tasks. Set the environment variable KVIKIO_TASK_SIZE to the maximum task size (in bytes). If not set, the default value is 4194304 (4 MiB).

When opportunistic Direct I/O read is enabled (KVIKIO_AUTO_DIRECT_IO_READ=1), this value should be a multiple of page size (typically 4 KiB) so that parallel tasks start at page-aligned file offsets, avoiding buffered I/O fallback.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name task_size.

GDS Threshold KVIKIO_GDS_THRESHOLD#

In order to improve performance of small IO, .pread() and .pwrite() implement a shortcut that circumvent the threadpool and use the POSIX backend directly. Set the environment variable KVIKIO_GDS_THRESHOLD to the minimum size (in bytes) to use GDS. If not set, the default value is 16384 (16 KiB).

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name gds_threshold.

Size of the Bounce Buffer KVIKIO_BOUNCE_BUFFER_SIZE#

KvikIO might have to use intermediate host buffers (one per thread) when copying between files and device memory. Set the environment variable KVIKIO_BOUNCE_BUFFER_SIZE to the size (in bytes) of these “bounce” buffers. If not set, the default value is 16777216 (16 MiB).

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name bounce_buffer_size.

HTTP Retries KVIKIO_HTTP_STATUS_CODES, KVIKIO_HTTP_MAX_ATTEMPTS#

The behavior when a remote I/O read returns an error can be controlled through the KVIKIO_HTTP_STATUS_CODES, KVIKIO_HTTP_MAX_ATTEMPTS, and KVIKIO_HTTP_TIMEOUT environment variables.

KvikIO will retry a request should any of the HTTP status code in KVIKIO_HTTP_STATUS_CODES is received. The default values are 429, 500, 502, 503, 504. This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name http_status_codes.

Transport failures that may succeed on a new attempt are also retried: a timeout, a failed DNS lookup or connection, a connection error while sending or receiving, and a response that ends early or is empty.

The maximum number of attempts to make before throwing an exception is controlled by KVIKIO_HTTP_MAX_ATTEMPTS. The default value is 3. This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name http_max_attempts.

The maximum duration of each HTTP request is controlled by KVIKIO_HTTP_TIMEOUT. The default value is 60, which is the duration in seconds to allow. This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name http_timeout.

Each retry emits a notice through the KvikIO logger at the WARN level, i.e. KVIKIO_LOG_LEVEL=WARN. Sustained retries indicate that the server is throttling the requests, which degrades read throughput.

HTTP Verbose KVIKIO_REMOTE_VERBOSE#

For debugging HTTP requests, you can enable verbose output that shows detailed information about HTTP communication including headers, request/response bodies, connection details, and SSL handshake information.

Set the environment variable KVIKIO_REMOTE_VERBOSE to true, on, yes, or 1 (case-insensitive) to enable verbose output. Otherwise, verbose output is disabled by default.

Warning

This may show sensitive contents from headers and data.

Remote I/O Backend KVIKIO_REMOTE_IO_BACKEND#

KvikIO supports two backends for remote (HTTP/S3/WebHDFS) reads, selected via the environment variable KVIKIO_REMOTE_IO_BACKEND. The accepted values (case-insensitive) are:

  • EASY_THREADPOOL: Libcurl easy API running in the KvikIO thread pool. Each sub-range of a kvikio.RemoteFile.pread() is dispatched to a worker thread that blocks in curl_easy_perform() until its transfer completes. Concurrency is bounded by the thread pool size: one busy thread per in-flight transfer.

  • MULTI_POLL (default): Libcurl multi API driven by N reactor threads, each of which blocks in curl_multi_poll(). A single reactor multiplexes many in-flight easy handles concurrently, so the number of simultaneous transfers is bounded by KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS rather than by the reactor count.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name remote_io_backend. kvikio.RemoteFile.pread() reads the setting on every call, so a change applies to subsequent reads while reads already in flight finish on the backend they started on.

Under the default MULTI_POLL backend, KVIKIO_NTHREADS no longer affects remote reads, and a read into device memory requires KVIKIO_TASK_SIZE to be no larger than KVIKIO_BOUNCE_BUFFER_SIZE (a larger task size raises ValueError). Set KVIKIO_REMOTE_IO_BACKEND=EASY_THREADPOOL to restore the previous behavior.

The MULTI_POLL backend honors two additional settings, described in the sections below: KVIKIO_REMOTE_IO_NUM_REACTORS and KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS. They have no effect under EASY_THREADPOOL. These settings are captured once when the reactor pool is first used (i.e. the first MULTI_POLL remote I/O), and become immutable for the rest of the process lifetime; attempting to change any of them afterwards raises an exception.

Remote I/O Reactor Count KVIKIO_REMOTE_IO_NUM_REACTORS#

Number of reactor threads used by the MULTI_POLL backend. The default value is 8. Each reactor owns one CURLM* handle and serializes its libcurl-multi calls. More reactors spread the in-callback memcpy cost across more cores. This setting has no effect under EASY_THREADPOOL.

The default of 8 is a moderate starting point. For a single process doing heavy remote reads using HTTPS, try one reactor per core first. Keep KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS a multiple (such as 8x, 16x, 32x) of the reactor count. A higher cap is not always faster, and it uses more pinned memory for device reads. KVIKIO_REMOTE_IO_NUM_REACTORS and KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS need to be tuned together.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name remote_io_num_reactors, as long as the MULTI_POLL reactor pool has not already started.

Remote I/O Reactor Dispatch KVIKIO_REMOTE_IO_REACTOR_DISPATCH#

Warning

Deprecated. This setting has no effect.

Reactor dispatch is no longer configurable. The reactors always take sub-ranges from one shared queue, which is what SHARED_QUEUE used to select.

Setting KVIKIO_REMOTE_IO_REACTOR_DISPATCH only logs a deprecation warning. The property name remote_io_reactor_dispatch remains accepted by kvikio.defaults.get() and kvikio.defaults.set() with a FutureWarning. Reading it always returns SHARED_QUEUE, and setting it is a no-op.

Remote I/O Concurrency Cap KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS#

Upper bound on the number of HTTP range requests the MULTI_POLL backend keeps in flight at once, summed across all reactor threads. The default value is 256. It must be a non-negative integer, and 0 means unlimited. The value is ignored when the active backend is not MULTI_POLL (EASY_THREADPOOL is already bounded by KVIKIO_NTHREADS).

The global budget is divided into an equal private share per reactor (KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS divided by KVIKIO_REMOTE_IO_NUM_REACTORS). Each reactor enforces its own cap with no cross-reactor synchronization. When the budget is not a multiple of the reactor count, the remainder is spread one extra slot each over the first reactors to keep the total exact. When the budget is smaller than the reactor count, every reactor still gets one slot, and the effective total is then the reactor count.

A reactor pulls a sub-range from the shared queue only when its share has room. Sub-ranges therefore go to whichever reactor frees up first, and the full budget stays in flight as long as enough work is queued.

Each in-flight read into device memory stages through a pinned bounce buffer of KVIKIO_BOUNCE_BUFFER_SIZE bytes, whatever the task size. Each reactor keeps up to twice its share of these buffers per CUDA context for reuse. Pinned memory per CUDA context is therefore bounded by 2 x KVIKIO_REMOTE_IO_MAX_CONCURRENT_REQUESTS x KVIKIO_BOUNCE_BUFFER_SIZE, which is 8 GiB with the defaults (4 GiB of it in flight at once). Lower either setting to reduce it. With 0 (unlimited) there is no bound.

For remote reads into device memory under MULTI_POLL, set KVIKIO_BOUNCE_BUFFER_SIZE equal to KVIKIO_TASK_SIZE, since any extra space is pinned but never used.

This setting can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name remote_io_max_concurrent_requests, as long as the MULTI_POLL reactor pool has not already started.

Shared DNS Caches KVIKIO_REMOTE_SHARE_DNS_CACHE, KVIKIO_REMOTE_MAX_THREADS_PER_DNS_CACHE#

Let the easy handles in EASY_THREADPOOL share DNS caches to reduce lookups.

Sharing is enabled by default. Set KVIKIO_REMOTE_SHARE_DNS_CACHE to false, off, no, or 0 (case-insensitive) to disable sharing.

KVIKIO_REMOTE_MAX_THREADS_PER_DNS_CACHE sets how many threads may share one DNS cache. The default value is 16. A thread is assigned a cache on first use and keeps it for its lifetime, and a new cache is created once the current one is full. The total number of caches is roughly the number of threads divided by this value.

Each cache holds one DNS result per host, which for S3 is a set of addresses drawn afresh on every resolution. Smaller KVIKIO_REMOTE_MAX_THREADS_PER_DNS_CACHE value leads to more caches, spreading connections over more addresses, at the cost of more lookups and less reuse. A thread returns its cache assignment when it exits, and resizing the thread pool with kvikio.defaults.set() reuses the existing caches rather than adding more.

Both variables are read only from the environment, and only when the caches are first used. Neither has any effect under MULTI_POLL.

Receive Buffer Size KVIKIO_REMOTE_IO_BUFFER_SIZE#

Size in bytes of libcurl’s receive buffer, one per transfer. When unset, libcurl’s own default of 16 KiB (CURL_MAX_WRITE_SIZE) is used. The value must be between 1 KiB and CURL_MAX_READ_SIZE (10 MiB in recent versions of curl).

This variable is read only from the environment, once per process.

Network Interface Binding KVIKIO_REMOTE_IO_INTERFACE#

Bind every connection to one network interface, for hosts with several NICs on one subnet. The value is passed to libcurl verbatim: <ip> binds the source address, if!<name> binds the device, and ifhost!<name>!<ip> binds both.

Unset by default. This variable is read only from the environment, once per process.

Non-temporal Copy KVIKIO_REMOTE_IO_NONTEMPORAL_COPY#

Copy received data into host memory with non-temporal stores, which skip fetching the destination cache lines. This includes the pinned bounce buffers of device reads.

It helps only when the destination is much larger than the last-level cache and is not read again soon. It requires x86-64 with AVX2, and falls back to memcpy elsewhere.

Enabled by default. Set to false, off, no, or 0 (case-insensitive) to disable. This variable is read only from the environment, once per process.

Non-temporal Copy Threshold KVIKIO_REMOTE_IO_NONTEMPORAL_COPY_THRESHOLD#

Size in bytes from which a range request uses non-temporal copy, when KVIKIO_REMOTE_IO_NONTEMPORAL_COPY is enabled. A range request covers at most KVIKIO_TASK_SIZE bytes of a read.

The default value is 1048576 (1 MiB). This variable is read only from the environment, once per process.

Discard Received Data KVIKIO_REMOTE_IO_DISCARD_DATA#

For benchmark purposes only. Drop received data instead of copying it into host memory, to benchmark the network path alone. Reads into device memory are not affected.

Set to true, on, yes, or 1 (case-insensitive) to enable. Disabled by default. This variable is read only from the environment, once per process. It is experimental and may change without a deprecation period.

Warning

The destination buffer is left untouched, with no error raised.

CA bundle file and CA directory CURL_CA_BUNDLE, SSL_CERT_FILE, SSL_CERT_DIR#

The Certificate Authority (CA) paths required for TLS/SSL verification in libcurl can be explicitly specified using the following environment variables in order of overriding priority:

  • CURL_CA_BUNDLE (also used in the curl program) or SSL_CERT_FILE (also used in OpenSSL): Specifies the CA certificate bundle file location.

  • SSL_CERT_DIR (also used in OpenSSL): Specifies the CA certificate directory.

When neither is specified, KvikIO searches several standard system locations for the CA file and directory, and if the search fails falls back to the libcurl compile-time defaults.

Opportunistic POSIX Direct I/O operations KVIKIO_AUTO_DIRECT_IO_READ, KVIKIO_AUTO_DIRECT_IO_WRITE#

Overview#

By default, POSIX I/O operations perform buffered I/O using the OS page cache. However, Direct I/O (bypassing the page cache) can significantly improve performance in certain scenarios, such as writes and cold page-cache reads.

Traditional Direct I/O has strict requirements: The buffer address must be page-aligned, the file offset must be page-aligned, and the transfer size must be a multiple of page size (typically 4096 bytes). kvikio.CuFile provides the feature of opportunistic Direct I/O, which removes these restrictions by automatically handling alignment. Specifically, KvikIO can split a POSIX I/O operation into unaligned and aligned segments and apply buffered I/O and direct I/O respectively.

Configuration#

Set the environment variable KVIKIO_AUTO_DIRECT_IO_READ / KVIKIO_AUTO_DIRECT_IO_WRITE to true, on, yes, or 1 (case-insensitive) to enable opportunistic Direct I/O.

export KVIKIO_AUTO_DIRECT_IO_READ=1
export KVIKIO_AUTO_DIRECT_IO_WRITE=1

Set them to false, off, no, or 0 to disable this feature and use buffered I/O.

export KVIKIO_AUTO_DIRECT_IO_READ=0
export KVIKIO_AUTO_DIRECT_IO_WRITE=0

If not set, the default setting is buffered I/O for POSIX read (KVIKIO_AUTO_DIRECT_IO_READ=0) and Direct I/O for POSIX write (KVIKIO_AUTO_DIRECT_IO_WRITE=1).

Programmatic Access#

These settings can be queried (kvikio.defaults.get()) and modified (kvikio.defaults.set()) at runtime using the property name auto_direct_io_read and auto_direct_io_write.

Example:

import kvikio.defaults

# Check current settings
print(kvikio.defaults.get("auto_direct_io_read"))
print(kvikio.defaults.get("auto_direct_io_write"))

# Enable Direct I/O for reads, and disable it for writes
kvikio.defaults.set({"auto_direct_io_read": True, "auto_direct_io_write": False})

Over-read Alignment KVIKIO_AUTO_DIRECT_IO_READ_OVERREAD#

When opportunistic Direct I/O is enabled for reads, unaligned prefix and suffix portions of each transfer fall back to buffered I/O. Setting KVIKIO_AUTO_DIRECT_IO_READ_OVERREAD=1 forces all disk reads to use Direct I/O by aligning offsets down and sizes up to page boundaries, discarding the extra bytes after the read. This only affects the device memory read path (disk to bounce buffer to GPU). Host memory reads are unaffected. Defaults to disabled.

export KVIKIO_AUTO_DIRECT_IO_READ_OVERREAD=1

Logging KVIKIO_LOG_LEVEL, KVIKIO_LOG_FILE#

By default, logging is disabled and no output is produced.

Set the environment variable KVIKIO_LOG_LEVEL to enable logging (case-insensitive). From most to least verbose:

  • TRACE

  • DEBUG

  • INFO

  • WARN

  • ERROR

  • CRITICAL

Each level includes all messages from less verbose levels.

If not set or set to any other value, logging is disabled.

By default, log output are written to the standard error stream. To write log output to a file, set the environment variable KVIKIO_LOG_FILE to a file path. The file is overwritten on each process start. If the file cannot be opened (e.g. the parent directory does not exist), KvikIO falls back to the standard error with a warning. KVIKIO_LOG_FILE has no effect when logging is disabled.