This page lets you do two things: reproduce XLOG’s performance measurements on your own machine, and tell which of XLOG’s published numbers you can trust as evidence. Not every number on this page is equal. Some are measured — they come from a real benchmark run, and this page names the harness or the procedure that produced them so you can rerun it. Others are targets — a number a benchmark is aimed at, which no run backs at all. When you quote XLOG’s performance, quote the measured numbers only. XLOG’s benchmarks run on Criterion.rs, a Rust harness that repeats each measurement many times and reports a confidence interval instead of a single fragile timing.

​
When to use this page

Reach for this page when you want to:
  • Check what XLOG’s real, backed-by-evidence speedups are before you cite them.
  • Rerun a benchmark yourself to confirm a number or to check for a regression on your own hardware.
  • Understand how the numbers were produced, so you can judge whether they apply to your workload.

​
Measured results

All three rows below carry a measured speedup. They differ in how much of the underlying run you can inspect. The neural-symbolic cache ablation publishes its per-run data in this repository at examples/neural/results/evidence/cache_ablation_20260218.json, so you can open that file and recompute the ratio yourself. The two WCOJ rows are reported from measured runs whose per-run data is not published, so the way to check those two is to measure them yourself. How you reproduce a row differs too. wcoj_triangle_bench ships, so the super-hub row reproduces with one cargo bench invocation. No bench target for the aggregate-fusion route exists in the repository, so that row reproduces as an A/B against the kill switch that turns fusion off. The last column gives the exact procedure for each row, and a speedup you measure that way on your own hardware is the one to trust for your own deployment. A worst-case-optimal join (WCOJ) computes multi-way patterns — such as triangles in a graph — directly, instead of joining two tables at a time and building a large intermediate result on the way to a small answer. That is why it can be many times faster on skewed graphs. Aggregate fusion folds a count/sum/min/max directly into the join instead of computing the join first and aggregating after. It has been available since 0.10.0 and applies automatically to the shapes it covers, so the only way to see what it saves is to turn it off and compare — which is what the row above asks you to do. Circuit caching reuses the compiled inference circuit across training steps instead of rebuilding it each time; the 2.74x above is the speedup from turning that reuse on.
A WCOJ speedup is not a constant. It moves with the GPU, the fixture size, and the key width — which is why the super-hub row is a range rather than a single figure. The numbers above are what XLOG measured on the fixtures named beside them; a number for any other hardware has to be measured on that hardware.

​
Neural-symbolic training detail

Two different runs sit behind the neural-symbolic numbers on this page. They used different settings, so read each number against its own run rather than mixing them. The cache ablation — the run the 2.74x comes from — used the 01_minimal example (MNIST addition) over 512 training images for 3 epochs, repeated across 3 seeds (7, 42, 123), once with circuit caching on and once off: The published 2.74x is the mean of the three per-seed ratios (2.94, 2.67, 2.60), which is why it is not exactly 242.90 / 88.89. Its 95% confidence interval is [2.29, 3.18]. Evidence: examples/neural/results/evidence/cache_ablation_20260218.json. Where the time goes within a run. A separate timing run — the same 01_minimal example over 512 images, but 5 epochs at batch_size=64, on development hardware — records the per-epoch shape: These timings describe the current pipeline, and the run behind them is not among this repository’s committed neural artifacts, which all record an earlier generation of the code. The first epoch is slow because XLOG compiles the logic program into a Decision-DNNF — a circuit form that makes exact probability counting cheap to evaluate — and verifies it. Once that circuit is cached, later epochs skip the compile and run in a fraction of a second.

​
Aspirational targets — not measured

The tables in this section are aspirational targets, not measured results (audited 2026-06-10). No run backs them at all. The Criterion harnesses exist (crates/xlog-gpu/benches/, crates/xlog-prob/benches/), but their output is git-ignored and no baseline has been committed. Do not cite these numbers as evidence or state them as achieved. They describe where each benchmark is aimed; only the numbers under Measured results are backed by a run.

​
Transitive closure (targets — unmeasured)

Transitive closure repeatedly follows edges until no new reachable pairs appear.

​
Hash join (targets — unmeasured)

​
Exact inference (targets — unmeasured)

Exact inference computes an exact probability over the compiled circuit, rather than estimating it by sampling.

​
Monte Carlo (targets — unmeasured)

Monte Carlo estimates the same probability by drawing many random possible worlds and averaging, trading exactness for speed on larger problems.

​
Running benchmarks

1

Check prerequisites

  • A CUDA-capable NVIDIA GPU (compute capability 7.0+). Development device: RTX PRO 3000 Blackwell, SM120, 12 GB.
  • The CUDA Toolkit 13.x, with nvcc on PATH, to compile the kernels. Runtime PTX is floored so the compiled artifacts also load on pre-CUDA-13 drivers.
  • Enough GPU memory: 4 GB minimum, 12 GB recommended for neural-symbolic training.
2

Run a suite

The WCOJ benchmarks need a CUDA device. Without one they print a Skipping …: No CUDA device line and exit successfully rather than failing, so check for that line before concluding a run produced nothing.
3

Save and compare a baseline

To catch regressions, save one run as a named baseline and compare later runs against it:

​
Reading Criterion output

A benchmark run prints a block like this for each case:
A run is flagged as a regression when the change lower bound exceeds +5% with p < 0.10. Two common noise sources can distort a run. GPU thermal throttling inflates variance, so let the card cool down between runs. And the first sample is slow because of one-time compilation, which the warm-up phase absorbs.

​
Methodology

​
Measurement approach

Criterion drives every benchmark. The default settings favor catching regressions over reporting a single peak number: Throughput is elements per unit time, where “elements” varies by benchmark: input edges for transitive closure, total input rows for joins, input rows for aggregation, circuit variables for exact inference, and samples × variables for Monte Carlo.

​
Warm-up and reproducibility

GPU benchmarks warm up so the timed region excludes one-time costs: PTX modules compile and cache, memory pools initialize, and the CUDA context is established. All random data uses deterministic seeding — a fixed-seed LCG, no system entropy — so the same seed produces identical graphs across runs. That is what makes a measured number reproducible rather than a one-off.

​
The WCOJ triangle harness

crates/xlog-integration/benches/wcoj_triangle_bench.rs is the harness behind the super-hub row above, and the one you run to measure WCOJ for yourself. It compares the GPU 3-way worst-case-optimal join against the ordinary binary-join chain on identical fixtures across u32, u64, and a Symbol sanity case. Each case runs three modes: Off (binary join), Force (WCOJ always), and Adaptive (classifier-gated, the default). The mode is set explicitly via RuntimeConfig so the timed path never depends on process-global state. Two guards keep the numbers honest:
  • Same answer. Before timing, each case asserts that the binary and WCOJ paths produce identical row sets.
  • Right path. During timing, it asserts the dispatch counter advances by exactly the iteration count on the WCOJ path and by zero on the binary path. A silent fallback anywhere in the hot loop fails the benchmark instead of quietly reporting the wrong path’s time.
The super-hub fixture — which concentrates roughly half the edges on a single key — is the one where WCOJ has the most to gain, and it is where the 10.5x to 33.8x range above was measured. It runs at 10K and 50K rows per relation by default; set WCOJ_BENCH_FULL=1 to add the 100K and 250K sizes. Criterion names each cell <width>-<rows>-<mode>, for example u32-50K-off and u32-50K-force, so the ratio of those two reported times is your local speedup. Nothing about that ratio is portable: report it with the GPU and the fixture size you ran it on.

​
Reference: benchmark environment variables

These variables tune what the benchmarks run and which code path they take. The last two are diagnostics for forcing or disabling the WCOJ triangle route.

​
See also