Repository navigation
Conversation
…cks up intel#33) (intel#38) * Add dtoctl wrapper to drive DTO without manual LD_PRELOAD setup Using the DTO library today requires the user to understand two orthogonal mechanisms before a single program can be offloaded to DSA. The library has to be injected into the target through LD_PRELOAD, and its behavior is steered entirely through a set of roughly twenty DTO_* environment variables that must be discovered by reading the manual and then exported by hand. This is easy to get wrong: the LD_PRELOAD path and the variable names are stringly typed, an exported variable silently leaks into every later command in the same shell, and there is no validation or discoverability at the point of use. Provide a small wrapper, dtoctl, that turns this into a single self-describing command line. It follows the same model as hmctl from the hmsdk project: parse options, translate them into the environment the library expects, and then exec the target program so no state lingers in the caller's shell. On top of that model dtoctl also takes care of the injection itself by prepending the resolved libdto.so to LD_PRELOAD, which is possible because DTO works by transparently interposing on the standard memory APIs. Every DTO_* variable is exposed as an option so the mapping stays obvious and greppable. Only options that are actually given are written to the child environment; anything left unset keeps whatever the caller already exported or the library's own default, and an explicit option wins over an inherited variable. The library path is resolved from -l/--library, then $DTO_LIBRARY, then a compiled-in default, and execvp() is used so a bare command name on PATH works as well as an explicit path. Build and install dtoctl with both Make and CMake. Set its compiled-in library path to match the installation directory, since CMake's default /usr/local/lib differs from the Makefile's /usr/lib64. Allow that path to be overridden with DTOCTL_LIBPATH. With this in place the previous incantation $ export LD_PRELOAD=/usr/lib64/libdto.so $ export DTO_WAIT_METHOD=busypoll $ export DTO_CPU_SIZE_FRACTION=0.33 $ ./prog collapses to $ dtoctl -w busypoll -c 0.33 ./prog Signed-off-by: Honggyu Kim <honggyu.kim@sk.com> * Add a dtoctl man page The dtoctl options mirror the DTO_* environment variables, but a user coming to the tool for the first time still needs a single authoritative place that explains what each option does, what values it accepts, and how the library path and LD_PRELOAD are resolved. Scattering that information across the source help text alone makes it hard to consult without running the tool. Add a manual page in the same troff-friendly markdown form the hmsdk project uses for hmctl, and render it as a section 1 man page for the user command. Besides the per-option reference it documents the library resolution order and the override semantics, and it gathers the common tuning goals - latency, power, and cache-pollution avoidance - as ready-to-run examples such as $ dtoctl -w umwait -c 0.0 --no-auto-adjust ./prog so users can map an intent directly onto a command instead of reverse engineering it from the individual variables. Keep the markdown as the source form and generate dtoctl.1 with pandoc during the Makefile and CMake builds. Both build systems install the generated page under man1, so it can be read with $ man 1 dtoctl Pandoc is optional: when it is unavailable, print a message recommending its installation and skip manual generation. The Makefile can still install an already generated page. Ignore its generated doc/dtoctl.1 and remove it on make clean; CMake keeps its generated page in the build directory. Signed-off-by: Honggyu Kim <honggyu.kim@sk.com> * Document dtoctl in the README The README is where users first learn how to configure and run DTO, and until now it only described the manual LD_PRELOAD and DTO_* export workflow. A reader following it has no reason to know that dtoctl exists or that it removes most of that ceremony, so the easier path stays hidden precisely where people look for guidance. Introduce dtoctl after the environment variable reference and the common usage models that accompany it, so that discussion stays in one piece and the new section can point back at those usage models. Contrast the old export sequence with the equivalent $ dtoctl -w busypoll -c 0.33 ./prog and spell out the resolution order for libdto.so along with the rule that only the options actually passed are applied. Keep the per-option reference in dtoctl(1) instead of repeating it here. The option list already exists in the --help output and in the manual, and a third copy in the README would only be one more place to forget when an option changes. Signed-off-by: Honggyu Kim <honggyu.kim@sk.com> * dto: honor DTO_WAIT_METHOD=yield The yield wait method is implemented in dsa_wait() and documented in the README, but dsa_init() never matched "yield" when parsing DTO_WAIT_METHOD, so DTO_WAIT_METHOD=yield silently ran with busypoll. * dtoctl: validate options, resolve the library up front, add --dry-run Make the wrapper catch the mistakes it exists to prevent instead of passing them on to libdto, which silently falls back to a default: - Validate every option value: wait method names, integer ranges for --numa-aware and --log-level, a cpu fraction in [0, 1), a positive umwait delay, and non-empty paths and WQ list. --min-bytes takes an optional K/M/G suffix and is passed on as plain bytes; libdto would read "64K" as 64 and "abc" as 0. - Resolve libdto.so to an absolute path and fail with exit status 125 if it cannot be read. ld.so only warns about a missing preload and runs the program without DTO, and a relative path stopped working as soon as the program changed directory. When built with LIBDTO_REL_PATH, prefer the library installed next to dtoctl so a relocated install finds its own copy. - Drop other libdto.so entries from an inherited LD_PRELOAD, so one process never loads two copies of DTO. - --stats-file implies --stats; libdto ignores DTO_STATS_FILE otherwise. - Add -i/--ignore-env to unset inherited DTO_* variables. The --no-* flags can only turn features off, so this is the way to undo an exported DTO_USESTDC_CALLS=1 or DTO_DSA_MEMCPY=0. - Add --dry-run to print the resolved library, LD_PRELOAD, each DTO_* variable with whether it came from an option or the environment, and the command, without running it. - Add --version, and exit with 126/127 when the program cannot be run or is not found, as env(1) and timeout(1) do. - Make the options local to main() and drop dead argument handling. * dtoctl: follow the install prefix, report the version, add tests CMake cached the default library path on the first configure, so a later CMAKE_INSTALL_PREFIX change was ignored, and "cmake --install --prefix" produced a dtoctl that preloaded /usr/local/lib/libdto.so. Compute the default on every configure, pass dtoctl the library's path relative to its own bindir so a relocated install finds its own copy, and keep DTOCTL_LIBPATH as an explicit override. Pass the project version to both builds for --version, and make the Makefile's install target build dtoctl first. Add tests/test_dtoctl.sh as the "dtoctl" CTest (label functional). It needs no DSA: it checks the option to variable mapping, validation, library resolution, LD_PRELOAD handling and exit statuses through --dry-run and the program's environment, and that libdto applies the settings through its own configuration trace. * docs: document dtoctl's validation, library resolution and --dry-run Update the dtoctl man page and README section for the new behavior: option validation, -i/--ignore-env, --dry-run, --version, size suffixes, --stats-file implying --stats, how the library is resolved, and the exit statuses. Add a note on sudo, which resets the environment and strips LD_PRELOAD when it sits between dtoctl and the program, and on settings being inherited by child processes. Fix stale defaults in the README's variable list: DTO_WAIT_METHOD defaults to busypoll and DTO_MIN_BYTES to 65536, tpause is a valid wait method, and DTO_LOG_FILE gets .<program>.<pid> appended. Describe the dtoctl test in TESTING.md. * gitignore: ignore the libdto_explicit build output --------- Signed-off-by: Honggyu Kim <honggyu.kim@sk.com> Co-authored-by: Honggyu Kim <honggyu.kim@sk.com>
test_perf.c uses fork(), _exit(), sysconf() and _SC_PAGESIZE without including <unistd.h>. With GCC 12 the transitive include is missing and the CMake build fails with "_SC_PAGESIZE undeclared".
…ntel#43) With DTO_WAIT_METHOD=tpause, init_dto() converts the wait from ns to TSC cycles using CPUID leaf 0x15 and divides by the denominator unconditionally. If the leaf is absent (__get_cpuid() returns 0 with the outputs uninitialized) or reports den=0, as on many VMs, the constructor dies with SIGFPE before main(). It also ran after dsa_init() had failed and the library was falling back to CPU, where the value is unused. Only do the conversion when DSA is in use and tpause is selected, and require the leaf to be present with non-zero den, num and frequency. Otherwise log an error and fall back to busypoll. Also fix waitpkg detection in dsa_init(), which queried CPUID leaf 0 instead of leaf 7. Leaf 0's ECX is the ASCII "ntel", which has bit 5 set, so waitpkg_support was always 1 and tpause/umwait on a CPU without WAITPKG would SIGILL instead of falling back.
…ntel#40) get_wq() picked a work queue with next_wq++ on a global atomic (and on a per-device atomic in NUMA-aware mode), so every DSA submit on every thread did a locked read-modify-write on one shared cache line. Each thread now keeps its own round-robin position. A thread takes its starting position once from a shared counter, so concurrent threads still spread across the queues, and from then on picking a queue touches only thread-local state. The position is 32 bits, so the rotation no longer skews each time the old atomic_uchar counter wrapped at 256 with a queue count that does not divide it. Measured on a Xeon 6787P with four DSA devices on one node, one shared WQ each: N pinned threads each doing back-to-back memcpy of one size through LD_PRELOAD, auto-tuning off, ns per call (median of 3): 4 KiB 16 threads 2295 -> 1432 32 threads 6629 -> 1549 16 KiB 32 threads 5920 -> 2991 64 KiB 32 threads 16970 -> 12124 Single-thread and device-bound cases are unchanged within noise, and the descriptors stay evenly spread: DSA perfmon counted the same number on each of the four devices in every NUMA mode, before and after. Claude-Session: https://claude.ai/code/session_01Mv7xKtQPkGoXQUvvuuhAEz Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
When another project pulls DTO in with add_subdirectory or FetchContent: - The perf test's build steps used CMAKE_SOURCE_DIR and CMAKE_BINARY_DIR, which are the parent's, so tests/build_baseline.sh and dto.c were not found and the parent's build failed -- with DTO's default options. - DTO_BUILD_TESTS defaulted to ON, adding DTO's tests, its dto-test apps and a perf baseline fetched from GitHub to the parent's build. - An unset CMAKE_BUILD_TYPE was forced to RelWithDebInfo in the cache, changing the parent's build type. Use the CMAKE_CURRENT_* directories, default DTO_BUILD_TESTS and the new DTO_BUILD_TEST_APPS (dto-test, dto-test-wodto) to ON only when DTO is the top-level project, and leave the build type alone otherwise. A top-level build is unchanged.
DTO_COLLECT_STATS kept one global table that every thread updated with
atomic adds, so the cost of collecting grew with the thread count: a
4 KiB memcpy on the CPU path cost 55 ns more with stats on at one
thread and 2.6 us more at 32 threads. Each thread now counts in its own
block, mapped when the thread first records a call and folded into a
retired total when the thread exits; output sums the live blocks and
the retired total. Stats now cost about 40 ns per call (the two clock
reads) at any thread count:
extra cost per call, CPU path 1 thr 4 thr 16 thr 32 thr
4 KiB, before +55 +357 +1319 +2612 ns
4 KiB, after +44 +46 +42 +37 ns
The histogram's 512 linear 4 KiB buckets, the last of which held
everything from 2 MiB up, become log-scaled: under 64 bytes, then four
buckets per power of two up to 4 GiB, then one for anything larger.
New settings:
DTO_STATS_INTERVAL_MS=N a sampler thread writes what happened in
each N ms; the records add up to the summary
DTO_STATS_FORMAT=json JSON Lines: interval and summary records with
calls/bytes/ns per op and path, failures,
histogram, and the configuration at exit
DTO_STATS_FILE=...%p... %p becomes the process ID
DTO_STATS_FILE is now appended to rather than truncated, so a forked
or exec'd child's records don't overwrite its parent's; each record
carries its pid. After fork the child starts with empty counts, and a
%p file is reopened under its own pid. The old dto_stats_log() wrote
each line twice when DTO_LOG_FILE was set and reused a consumed
va_list; it is gone, along with the TRACE log level that turning on
stats forced.
Adds tests/test_stats.py (ctest -R stats) with a workload of known
calls: totals and buckets, intervals summing to totals, 200
short-lived threads all counted and their blocks unmapped, fork,
%p, and text output.
The stats rework uses thread keys, a sampler thread and its condition variable. Before glibc 2.34 those functions are in libpthread, not libc, so preloading libdto into a program that doesn't link libpthread itself (sh, true, most single-threaded tools) failed at load time: true: symbol lookup error: libdto.so: undefined symbol: pthread_key_create Link libdto with pthread in CMake and the Makefile (an empty stub on newer glibc), and check in the stats test that stats work in a program without libpthread. Seen on RHEL 8 (glibc 2.28); Ubuntu 24.04 (2.39) is not affected.
- stats_fork_child() called pthread_setspecific(stats_key, NULL) even when stats were off and the key had never been created, i.e. on key 0, which belongs to whichever library created it first. Only touch the key when it exists. - sampler_lock was not held across fork(), so a child could inherit it locked by the parent's sampler thread and hang in its first stats_start_sampler(). Take it in the prepare handler with the other two and release it in parent and child. - stats_stop_sampler() reset sampler_started, so a thread whose first counted call landed after cleanup_dto() restarted the sampler during process exit, writing to closed descriptors. Set stats_shutdown in cleanup_dto() and check it before starting the sampler. - stats_flush() gave up on EINTR and EAGAIN and discarded the rest of the record. Retry on EINTR and poll() on EAGAIN. - The 64 KiB output buffer was smaller than a worst-case JSON summary, so a record could go out in two write()s and interleave with another process's in a shared O_APPEND file. Make it 256 KiB. - DTO_STATS_INTERVAL_MS accepted negative, non-numeric and absurd values silently; reject them with an error like DTO_STATS_FORMAT does.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
DTO_COLLECT_STATSkept one global table that every thread updated with atomic adds. Collecting stats got slower as threads were added, so it couldn't be left on to profile a multithreaded workload. This PR changes how stats are collected and adds the output needed to follow a run over time and process it with tools.Changes
DTO_STATS_INTERVAL_MS=N. A sampler thread writes what happened in each N ms. The interval records add up to the summary.DTO_STATS_FORMAT=json. Output is JSON Lines:intervalandsummaryrecords with calls/bytes/ns for each op (memset/memcpy/memmove/memcmp) and path (cpu/dsa/dsa_failed), plus failure reasons. The summary also has the histogram and the configuration at exit (wait method, the auto-tunedmin_bytesandcpu_size_fraction, NUMA, number of WQs). Schema is in the README.DTO_STATS_FILEexpands%pto the PID and appends instead of truncating, so a child process no longer overwrites its parent's output. Every record carries itspid.%pfile under its own PID. Locks are taken around fork, so a child can't inherit a lock held by the sampler or a printer. SIGINT output uses trylock.dto_stats_log(). WithDTO_LOG_FILEset it wrote each line twice, the second time from ava_listthat had already been consumed. Also removed: turning stats on no longer forces the TRACE log level.Overhead
Extra time per call with stats on, CPU path (
DTO_USESTDC_CALLS=1), threads pinned, median of 5 runs, gnr-qual1:What's left is the two
clock_gettimecalls. With stats off, nothing changed. On the DSA path (1 and 4 WQs), runs with stats on were 12–20% faster than main at 4 and 16 threads; 32 threads was too noisy to compare.Testing
New
ctest -R stats(tests/test_stats.py plus a workload of known calls,-fno-builtin). It needs no DSA hardware and checks:%pfilesOn gnr-qual1,
statspasses as root (dsa0 wq0.0) and as a user (CPU path), andfunctionalpasses 10/10.The second commit links libdto with pthread. The per-thread blocks use thread keys, and the sampler uses a thread and a condition variable; before glibc 2.34 these are in libpthread, not libc. Without the link, preloading DTO into a program that doesn't link libpthread itself (
sh,true) failed on RHEL 8 withundefined symbol: pthread_key_create. The stats test now checks this case.Merging with other open PRs
dto.cmerges cleanly with #34 and #40. #34 and #38 conflict only in the lines each PR adds to CMakeLists.txt, TESTING.md and README.md.A follow-up will add
dtoctl --stats-run. It runs a program CPU-only and then with DSA, optionally underperf, and reports DSA speedup per operation with an Amdahl's-law estimate, using the JSON output added here.