For contributors — how the rule-learning trainer works internally. If you want to learn rules from data, read Rule learning instead.
XLOG can learn Datalog rules from labelled examples by gradient descent rather than by searching over programs. This page describes how that trainer — the dILP (differentiable Inductive Logic Programming) subsystem — is built. The idea in one paragraph. Compiling every candidate rule into an executable program is too slow to do per training step. So xlog compiles all candidates once into a single program and attaches a learnable weight to each one. During training those weights are turned into a soft on/off mask with Gumbel-Softmax — a way to sample a near-discrete choice that still passes gradients — and the mask is annealed toward picking exactly one rule. At the end, the highest-weighted candidate is the learned rule.

​
Design Goals

  1. Learn rules, not weights — discover symbolic Datalog clauses (e.g., reach(X,Y) :- edge(X,Z), edge(Z,Y).) from data
  2. GPU-resident hot loop — no semantic column downloads in the training step loop
  3. Sparse by default — candidate-indexed soft-probs instead of materializing N³ tensors
  4. Transactional promotion — learned rules pass gate checks before entering the knowledge base
  5. Auditable transfer evidence — learned rules carry fold, held-out-domain, gate, and base-kernel checksum metadata

​
Core Idea: Tensorized Super-Graph Masking

Traditional ILP systems compile candidate rules into executable programs — impossible at millisecond timescales. XLOG instead pre-compiles a super-graph — one program holding every candidate rule at once — and activates individual candidates through continuous mask tensors optimized with Gumbel-Softmax:
At convergence, argmax(W) picks the winning rule. Temperature annealing (τ → τ_floor) drives the soft mask toward a one-hot selection.

​
Architecture Overview

​
Key Entry Points

​
Python (pyxlog.ilp)

​
Rust (xlog-runtime, xlog-cuda)

​
CUDA Kernels

​
Mask Backends

The MaskBackend protocol abstracts how the learnable tensor W is applied to the XLOG executor:

​
SparseMaskBackend (default)

  • Learnable params: C floats (one per candidate rule)
  • Memory: O(C) — typically C < 100
  • Preferred hot-loop path calls set_rule_mask_sparse_selected() on the compiled program
  • Legacy compatibility path set_rule_mask_sparse() remains available when Rust-side ranking is desired

​
DenseMaskBackend (alpha-compatible, debug)

  • Learnable params: N³ floats (N = schema size)
  • Memory: O(N³) — expensive for large schemas
  • Enabled via TrainConfig(debug_dense_mask=True) for parity testing

​
Training Pipeline

​
train_only()

  1. Candidate enumeration — valid_candidates(source, mask_name) returns all syntactically legal body-pair assignments
  2. Multi-start — up to max_attempts independent restarts with fresh logits
  3. Step loop (per attempt, up to step_budget_per_attempt):
    • Apply mask via backend
    • Forward pass: program.evaluate_device() (GPU-only, no host reads)
    • BCE loss between predicted and target fact membership
    • Backward pass: loss.backward() through PyTorch autograd
    • Optimizer step on W
    • Temperature anneal: τ_start → τ_floor (cosine schedule)
    • Optional deterministic controls (deterministic=True) for reproducible attempt seeding
    • Early stopping: when argmax is stable and loss < threshold
  4. Decode — argmax(W) maps to winning candidate → discovered rule string

​
train_and_promote()

  1. Call train_only() — get TrainResult
  2. If not converged → PromotionStatus.NOT_CONVERGED
  3. Trial compile — substitute discovered rule into source, compile via Rust
  4. Promotion gates (all must pass for PROMOTED):
    • Convergence gate — training converged (already checked)
    • Novel-rate gate — fraction of non-example derivations ≤ max_novel_rate
    • Protected-relation gate — no unwanted relation side-effects
    • Holdout F1 gate — F1 on held-out examples ≥ threshold
    • Ambiguity gate — top-M scan (or exhaustive mode) detects no alternative winning candidates
    • Typed-schema gate — optional hard gate requiring relation type metadata (or waiver-driven manual review)
  5. All pass → PromotionStatus.PROMOTED with committed_source

​
External Consumer Training Surface

A higher-level training entry point covers sources that mix neural predicates and trainable symbolic clauses:
The source owns declarative nn(...), trainable_rule(...), and train(...) declarations. The result reports neural gradient norms, symbolic gradients, final symbolic weights, and a RuleInventory suitable for transfer audits. Backbone coupling (since 0.12.0). By default a neural body spec detaches the entity features phi(x) before uploading them for training, so gradient stops at the feature producer. Setting NeuralBodySpec.train_phi_gradient = True leaves phi attached, so gradient flows back into whatever computed the features. Only the autograd linkage changes — the uploaded values are byte-identical either way (crates/pyxlog/python/pyxlog/ilp/neurosymbolic.py:114). Typed network registration (since 0.11.0). CompiledProgram.register_network accepts keyword-only arity=, arg_sorts=, and artifact_hash=. The arity is validated against every nn/4 declaration bound to that network name in the program rather than trusted: a mismatch raises ValueError naming both the declared and the passed arity (crates/pyxlog/src/neural.rs:290). Passing arg_sorts without arity is rejected, as is an arg_sorts whose length differs from arity, or one containing a bool or a non-int element. CompiledProgram.network_metadata(name) reads the registration back as {arity, arg_sorts, artifact_hash, declared}; embedding-declared names are refused, since they carry no registration metadata.

​
Existential-join trainable bodies (Stage B)

A trainable_rule body may join a neural predicate to an ordinary relation on an existential (non-head) variable — the neural predicate is grounded over the real join domain inside the circuit and OR-aggregated at the head:
Here Event appears only in the body. The engine materializes the join domain from pre_before_post’s ground facts, emits one neural leaf per joined event, and the differentiable provenance OR-aggregates the per-event contributions per head binding, yielding P(plastic(Edge)) = σ(w) · (1 − ∏_{e : pre_before_post(e,Edge)} (1 − p_saliency(e))). Gradient flows into the neural predicate (all joined events) and the rule guard, but never into the deterministic join relation. The per-event features arrive through a domain_inputs={"net": features} channel, with a companion domain_ids={"net": ids} naming the domain constant each row holds (see the domain_ids contract below), and examples carry only per-head-binding targets. Because saliency is learned as a function of the event feature (not an id lookup), the trained predicate generalizes to unseen events. Constraints: the join domain must be ground facts (a derived relation is rejected, since its extension is not materialized); head-binding ids must be 0..N-1 row-aligned with targets; a single join network is supported; and the exact d-DNNF compiler — d-DNNF is a circuit form of a Boolean formula whose satisfying assignments can be weighted and summed in one pass instead of enumerated — builds one circuit over all head-binding queries, so the planted graph must stay within the compiler’s fixed buffer (empirically ~6–7 events). Worked example + CUDA-gated recovery test: examples/plasticity_incircuit/ and python/tests/test_plasticity_incircuit.py. Head-variable (“hard filter”) joins remain supported as pre-filters; only the existential-join case is new.

​
Neural join bodies in the joint mixture

A rule that puts a neural predicate on an existential join variable is no longer confined to being the program’s only trainable rule: it now competes as one of several same-head candidates in the joint mixture. Give the mixture a relation vocabulary and it will find which relation the rule joins on, while learning the neural predicate from scratch:
For one candidate rule, xlog needs the probability that its head fires. It gets there in three steps. First, it asks the engine which domain constants actually join to each head binding — the candidate’s join extension. This always comes from the engine’s own relation facts (relation_facts); a caller cannot supply the binding-to-constants map by hand. Second, it runs the network on each of those constants to get a per-constant probability. Third, it combines them. The head fires if any joined constant does, so the mask is one minus the product of the complements — a noisy-OR. xlog computes this in log space for numerical stability:
The domain_ids contract. domain_ids is the one map from a domain constant to its feature row, and both engines — the exact d-DNNF circuit and the torch-side mixture — resolve rows through it:
The ids must be distinct; they may be in any order and need not be contiguous (a joined constant absent from domain_ids is refused, not silently mis-indexed). Omitting domain_ids defaults it to [0 … D-1]. There is no rank-indexing fallback: the map is the only path from a constant to a feature row. Semantics anchor. With a single candidate, the torch-side OR reproduces the exact d-DNNF circuit to ~2e-07 (tolerance 1e-4) on four domain layouts — dense, sparse, superset (rows for constants that are never joined) and shuffled (domain_ids in non-sorted order). See python/tests/test_join_semantics_anchor.py. This is candidate SELECTION, not rule induction. build_join_candidates fills the single free slot of a fixed body template once per relation name supplied by the caller — |R| candidates, no conjunctions, no chaining through an intermediate variable, no recursion, no negation. It is a different and narrower search than the engine-side dILP enumerator (valid_candidates, |R|² chained candidates with recursion), and it does not call it: the two induction paths remain disjoint. What is new here is the neural predicate on an existential variable, trained through the logic — not an enlargement of the hypothesis space. Worked example + CUDA-gated tests: examples/neural_join_discovery/, python/tests/test_join_discovery.py and python/tests/test_join_identifiability.py. Limits — stated plainly:
  1. One join network per program. domain_inputs currently supports a single join network.
  2. Head arity must be 1. The multi-outcome form plastic(Edge, L) :- saliency(Event, L), pre_before_post(Event, Edge). does not compile — the mixture’s eligibility call is fixed at arity 1. Multi-outcome plasticity (a learned label on the head) is not supported and must not be claimed.
  3. The inter-candidate noisy-OR is a modelling choice, not compiled semantics. The anchor above pins the per-candidate mask against the exact circuit. The rule that combines several candidates into one head probability has no exact-circuit counterpart and cannot have one: declaring more than one trainable_rule is precisely what routes execution away from the circuit and into the torch-side mixture.
  4. Saturation. The noisy-OR saturates as the number of joined constants per head binding grows: at the default init (p ≈ 0.5) a binding with k joined constants starts at 1 − (1−p)^k ≈ 1, the gradient to the detector vanishes, and the optimizer lands in a degenerate inverted minimum that more steps do not escape (seed 0, k = 6, bare: loss pinned at the base-rate entropy 0.640 at 1500/3000/6000/12000 steps, with the wrong candidate hardened to 1.0). Shifting the detector’s initial logit for the positive label by −2.0 (a “quiet prior”: an initialization encoding the prior that constants are mostly negative) removes the basin. Measured over 5 seeds at n_edges = 40 — seeds discovering the rule / mean accuracy: Beyond roughly 4–6 joined constants per head binding the detector stops converging reliably without a sparsity prior. Saturation hits the detector before it hits the selection: at k = 16 with the prior all 5 seeds still pick the correct relation, but one never converges its detector (accuracy 0.600).
  5. Identifiability — the mixture cannot rank relations it cannot distinguish. The inter-candidate noisy-OR is monotone and the objective carries no sparsity term at all (no L1, no weight_decay, no simplex over candidate weights). Two candidates with the same extension are therefore exactly degenerate: 1 − (1−w₁m)(1−w₂m) is reachable with the mass split, so the loss is flat between them. Measured (python/tests/test_join_identifiability.py): Three consequences, all load-bearing. (a) argmax over candidate weights is not a selection signal. Python’s max returns the first key holding the maximum, so on indistinguishable relations it reports whichever the caller listed first — a confident wrong answer. Use discovery.select_rule, which claims a rule only when one candidate is both believed (weight ≥ 0.5) and alone at the top (runner-up > 0.01 behind), and abstains otherwise. (b) Accuracy is not evidence that the relation was identified — it is 1.000 in every tied case above. (c) Weight alone catches ambiguity, not degeneracy — in the trivially-true world the wrong candidate can come back believed and alone on weight, so the weight/tie gates pass it through. The fit gate ships in select_rule(fits=...): the joint mixture (neurosymbolic._train_joint_mixture) exports candidate_train_fit, each candidate’s TRAIN-set agreement mean((mask ≥ 0.5) == targets) off the final step’s own mask, independent of its guard weight or its rank among the others. Passing it through (select_rule(weights, fits=candidate_train_fit, min_fit=0.75)) drops any candidate below min_fit before ranking; if none survive, the run abstains and names the fit gate in reason. Measured on the trivially-true world: the recovered seed’s winner fits at ~1.0 and is selected exactly as before; the seed that used to derail selects co_occurs at weight 0.955 but fit 0.500 — a coin flip — and is now caught and abstained on rather than confidently misreported (python/tests/test_join_identifiability.py).
  6. A neural predicate cannot ride in the dILP enumerator’s own candidate space. The mixture’s candidate vocabulary is supplied by the caller. The engine-side enumerator (valid_candidates) and this mixture remain disjoint induction paths.
train_and_promote(...) also accepts training_fold, held_out_domains, base_kernel_checksum_before, and base_kernel_checksum_after. These fields are recorded on PromotionResult.rule_inventory, along with selected and rejected candidate clauses and gate outcomes.

​
Artifact Persistence

LearnedArtifact captures the full training result for reproducibility:
Schema version: beta-v1. Fields: discovered rule, logits, candidate map, config, telemetry, precision/recall, metadata (timestamp, schema version, candidate map hash).

​
GPU Contract

The training step loop obeys XLOG’s GPU-resident contract. Tagged credit below means: for each derived fact, which candidate rules derived it — the coverage matrix the loss gradient is built from.
  • evaluate_device() — no host reads for semantic results
  • batch_fact_membership_device() — returns a CUDA bool mask via DLPack with zero semantic-loop device-to-host transfer
  • batch_tagged_credit_device() — returns CSR-style CUDA credit data via DLPack with zero semantic-loop device-to-host transfer
  • batch_fact_membership() / batch_tagged_credit() remain available when host materialization is desired
  • AtomicU64 device-to-host counter on CudaKernelProvider — hard gate raises if download_column_* is observed during step loop
  • host_transfer_stats() / reset_host_transfer_stats() expose broader host transfer accounting for profiling
  • Legacy set_rule_mask_sparse() still performs a control-plane soft-probability download; the selected-candidate sparse path avoids it

​
Testing

  • 86+ static test functions across ILP Python test files (expanded by parametrized GA/beta gates)
  • Reliability gate: 20 consecutive train_only() runs must all converge (20/20 pass)
  • GA reliability gate: default 50-seed statistical run (test_ilp_ga_reliability.py)
  • GA performance/transfer tests: forward_p95_us + host transfer accounting (test_ilp_performance.py)
  • Dense/sparse parity: every sparse-path test has a debug_dense_mask=True variant
  • Rust-side: ilp_integration_tests.rs, ilp_kernel_tests.rs
  • CUDA certification: extract_nonzero_indices covered by kernel test suite

​
See Also