How can a photorealistic 3D CT scan be viewed in real time on an ordinary computer?

The short answer: bake the rendering into Gaussian splats

A photorealistic 3D CT scan can be viewed in real time on an ordinary computer by converting, or baking, the finished volume rendering into a 3D Gaussian splatting model, a format that lightweight viewers can draw quickly. A 2026 preprint called CTBakeGS, by Haruki Fukuda, Constantin Lux, Wolf Schweitzer, Tadashi Hosoya and Lars Ebert, does this conversion directly, with no neural-network training, in about a minute or less per conversion pass on ordinary computers.

The result is not the CT scan itself but a portable 3D picture of how one specific rendering of it looks, which can be rotated smoothly and shown in a web browser.

Why photorealistic CT rendering is hard to share

A CT scan is a stack of voxels, small 3D cells, each holding a density value. Volume rendering turns that stack into a 3D image using transfer functions: one maps each density value to an opacity, deciding how see-through a tissue looks, and another maps it to a colour. Change the transfer function and the same scan can show bone only, contrast-filled vessels, or skin over bone.

Conventional volume rendering uses a simple lighting model along each viewing ray. Cinematic rendering goes further. A 2016 pictorial review in Insights into Imaging describes it as solving the rendering equation with Monte Carlo path tracing, sampling many random light paths so that shadows, multiple scattering and ambient occlusion look natural. The cost is heavy: the review reports that a final image takes about 5 to 30 seconds and that rotating the view is interrupted by recalculation. The CTBakeGS authors add that this kind of rendering needs the original CT data, specialised software and a powerful GPU, which makes it hard to hand to a student, a colleague or a court.

What 3D Gaussian splatting adds

3D Gaussian splatting, introduced by Kerbl and colleagues in 2023, represents a scene as a large set of small, soft, ellipsoid blobs, each with a position, shape, colour and opacity. Their paper reports real-time novel-view rendering at 30 frames per second or more at 1080p, because the renderer can skip empty space and draw the blobs quickly.

Normally a splat model is learned by optimisation: the blobs are adjusted, often for minutes to hours, until rendered views match reference images. The CTBakeGS authors note that existing medical splatting methods also train each scene for minutes to hours, which is the step they set out to remove.

How the training-free conversion works

According to the preprint's abstract, splats are placed on an adaptive octree of millimetre-sized cells, so detail goes where the anatomy needs it. Each splat's colour and opacity are computed on the GPU using the renderer's own lighting, so the photorealistic shading is baked in. The opacity of each cell is converted analytically into splat opacity rather than learned. Because standard splat rasterisers ignore anything fainter than 1/255 opacity, faint splats are kept stochastically above that floor so thin, translucent tissue does not simply vanish.

The authors tested six public CT scans with 1.5 mm voxels and three transfer functions, comparing against converged reference renderings. With two-million-splat models they report a peak signal-to-noise ratio of about 32.6 dB for bone, 30.1 dB for contrast-enhanced vessels and 32.3 dB for skin and bone, which they judge sufficient for interactive viewing. A model usually needs two or three passes; one pass takes 4 to 30 seconds on a current laptop, and the laptop rendered the result at 68 frames per second at 800 by 800 pixels.

Limits worth keeping in mind

A baked splat model freezes one transfer function and one lighting setup; showing a different tissue means baking again. It is also an approximation of the reference rendering, not the diagnostic data, and the 2016 review cautions that both volume and cinematic rendering can hide findings when rendering parameters are poorly chosen, and that no diagnostic gain from cinematic rendering had been shown. The practical value is communication: teaching, explaining a case, or sharing a 3D view with someone who has no CT workstation.