pathtracer-diff

A differentiable Monte Carlo path tracer as one kernel, loadable through kernels. The forward pass is a megakernel unidirectional path tracer with five materials, area lights, an importance-sampled environment map, an optional homogeneous medium, and multiple importance sampling; the backward pass replays the identical paths with identical counter-based draws and emits analytic gradients. The reference standards are analytic transport results and finite differences along the same paths.

Rendering is a simulation of light, and making it differentiable turns it into an instrument: give it a photograph and unknown scene parameters, and gradient descent recovers the materials, the lights, the fog, and the geometry that produced the image. That normally requires a research framework; here it is one loadable kernel, and inverse rendering is a bare torch loop: render, compare, backward(), step.

THE DRAGON'S HALL (media/make_hero.py): one continuous shot, every gradient, optimized live through the kernel against the fixed TARGET inset. From a flat gray start, Adam through render() recovers the 128x128 marble floor albedo (49,152 unknowns), the gold trim's conductor tint, the pedestal's plastic albedo, the braziers' warm emission (max err 1.75 of 14), and a 16x64 per-texel emissive rune band (3,072 unknowns, mean err 0.15), around a smooth glass slab and the rough-dielectric crystal dragon. Mid-shot, geometry_grad() alone takes over the dragon's position: 5,205 vertices slide 4.60 scene units onto the pedestal by shadow and silhouette alone, final offset 0.004. In the last phase the participating medium rolls in: sigma_a and sigma_s recover from the hazy target to extinction error 0.014. Image loss falls 8.5e-1 to 1.9e-1 at spp 8; the dim-hall floor and trim retain residual texel error where the scene barely observes them. Stanford dragon courtesy of the Stanford Computer Graphics Laboratory.

Usage

import torch
from kernels import get_kernel

ptd = get_kernel("phanerozoic/pathtracer-diff", version=1, trust_remote_code=True)

wall_tex = torch.full((256, 256, 3), 0.5, device="cuda", requires_grad=True)
scene = ptd.Scene(vertices, faces, material_ids,
                  albedo=[torch.tensor([0.73] * 3), wall_tex],
                  emission=[torch.zeros(3), torch.zeros(3)],
                  uvs=uvs, material_types=[ptd.DIFFUSE, ptd.DIFFUSE],
                  roughness=[0.3, 0.3])
cam = ptd.Camera(position=(2.78, 2.73, -8.0), look_at=(2.78, 2.73, 2.8),
                 vfov_deg=39.0)

img = ptd.render(scene, cam, 512, 512, spp=64, max_bounces=4)
loss = (img - target).square().mean()
loss.backward()                       # gradients on wall_tex

grad_verts = ptd.geometry_grad(scene, cam, dloss_dimage)

version selects the release branch; trust_remote_code is required by kernels for publishers without the trusted-publisher mark. A fixed seed renders the same paths every call, so the Monte Carlo objective is a deterministic function of the parameters and gradient descent sees a smooth landscape at any spp.

API

Method

One thread owns one pixel and accumulates its spp samples serially, so the image is bitwise deterministic with no atomics in the forward pass. Diffuse vertices sample the cosine hemisphere; conductors sample the GGX visible-normal distribution with the height-correlated Smith term; smooth dielectrics branch on Fresnel; rough plastic mixes a diffuse base with a GGX coat under a 50/50 lobe-mixture pdf; rough dielectrics sample GGX transmission. Direct light is estimated at every NEE-capable vertex by area sampling over emissive faces and, with an environment map, by a detached mixture of its luminance CDF and the uniform sphere; BSDF-sampled hits combine by the balance heuristic.

The backward pass is exact path replay: sampling distributions depend only on geometry, frozen parameters, and the counter-based Philox stream, never on the differentiable parameters, so the backward kernel re-traces identical paths with no stored path state. Every radiance term is a product of per-bounce factors, each affine in its vertex's albedo texels, times a linear emission or environment texel; the replay differentiates the product in closed form with prefix/suffix exclusion products and scatters each factor's gradient into the four texels of its bilinear footprint. Geometry gradients run a separate pass over the direct-lighting term: forward-mode duals for the smooth part, redner-style edge sampling for shadow and camera silhouettes.

Measured

RTX 6000 Ada, 512x512, MIS, 4 bounces; terrain scenes are displaced grids with a 256x256 checker albedo, a GGX conductor block, and a constant environment, built by the vectorized torch SAH builder:

Correctness

Measured on RTX 6000 Ada from source; the published v1 variants pass the full 22-test suite through get_kernel on a GeForce RTX 3070 Ti (sm86):

• Analytic furnace, zero variance: a closed uniform diffuse box returns exactly E * sum a^k per path; the analytic 0.498046875 is met with worst pixel deviation 3.4e-3.
• Fresnel slabs: a smooth glass slab at normal incidence transmits the internal-bounce series (1-R)/(1+R), measured 0.923141 against 0.923077 (0.007%); a rough-dielectric slab at alpha 0.02 within 0.05%.
• Beer-Lambert: a purely absorbing medium attenuates a uniform emitter to E * exp(-sigma_a d) within 0.035%.
• Estimator agreement: MIS, NEE-only, and BRDF-only means agree within Monte Carlo tolerance across Cornell, GGX-under-environment, and rough-plastic scenes.
• Gradients match finite differences along the same paths: constants 2.2e-4 max relative over nine entries; wall texels 2.6e-3; conductor F0 through GGX and MIS 9.9e-4; environment texels 2.1e-4; emission linear-exact; medium sigmas within 2e-2 gates. Texels no path can see get exactly zero from both sides.
• Geometry gradients match seed-averaged finite differences: light translation within 15%; occluder vertices within 0.23 worst case.
• Inverse rendering: a wall color recovers to max error 0.004 in 80 Adam steps; an 8x8 texture drives the loss from 1.8e-4 to 7.2e-14.
• Deterministic: repeated forward renders are bitwise identical; gradient buffers accumulate through float atomics and are deterministic in expectation but not bitwise.

Requirements and limits

• NVIDIA GPU with compute capability 8.0+; float32 throughout.
• At most 64 materials and 16 bounces; two-sided surfaces; no Russian roulette (the fixed bounce budget keeps the replay exact); pinhole camera.
• render() differentiates albedo, emission, environment texels, and medium sigmas; camera, UVs, roughness, and ior are frozen. geometry_grad() covers the direct-lighting term only.
• The medium is homogeneous, fills the scene, and is mutually exclusive with an environment map; the environment CDF is detached at Scene construction.
• Published variants are Linux x86_64; on Windows load_local.py JIT-builds the same source.

References

Kajiya (1986); Vicini et al., "Path Replay Backpropagation" (2021); Nimier-David et al., "Radiative Backpropagation" (2020); Li et al., "Differentiable Monte Carlo Ray Tracing through Edge Sampling" (2018); Heitz (2014, 2018) on GGX and VNDF; Walter et al. (2007); Veach and Guibas (1995); Möller and Trumbore (1997); Wald (2007) on SAH construction.

License

Apache-2.0.