One cat image,
one 3D splat

Lesson 1.1

A camera move is a learnable effect

MiniMax-H3's reference-to-video mode (Ref2VA) takes up to a few reference pictures and a caption and writes a video with sound. ORB360 is a rank-32 LoRA for it that teaches one camera move: a clockwise orbit at a constant distance that ends exactly where it began, with the subject frozen. Every training clip was rendered in Blender, so the camera's true position is known for every frame. That's what makes the orbit measurable later.

The orbit LoRA never saw a real-looking image or a real scene in training, yet it kept the ridge, the pines and the sky all the way round. It did one thing the caption didn't ask for: this orbit is at the image's own height, roughly eye level. Getting the camera to go higher or lower than the image is the next two sections.

# musubi-tuner (kohya-ss) v0.3.5+, MiniMax-H3 Ref2VA weights, LoRA from huggingface.co/MATLOWAI/MiniMax-H3-ORB360-CardSpin
python src/musubi_tuner/minimax_h3_generate_video.py --task ref2va \
  --dit minimax_h3_ref2va_bf16.safetensors --prune_adaln \
  --video_vae minimax_h3_video_vae_fp16.safetensors --audio_vae minimax_h3_audio_vae_fp32.safetensors \
  --text_encoder qwen3vl_32b_minimax_h3_int8_convrot.safetensors --text_encoder_attn_mode sdpa \
  --lora_weight minimax_h3_orb360_step1500.safetensors --lora_multiplier 1.0 --lora_runtime_attach \
  --video_size 768 1024 --video_length 124 --infer_steps 20 --attn_mode sdpa --seed 20260926 \
  --prompt "$(cat prompts/orbit_realscene_caption.txt)" --ref photo.png --save_path out/

Generate one 124-frame, 24 fps orbit with the ORB360 step-1500 LoRA from a single reference image, using the model card's real-scene orbit caption unchanged. Keep the raw MP4, the exact command, the seed and the LoRA hash next to it. Before using it for 3D, check by eye that it completes one turn, shows the back, never reverses, and that the last frame matches the first.

Lesson 2.1

Show it pictures from above, ask for eye level

For a splat you want views from several heights, including from below. So the test gave the first LoRA (750 steps on four 512×512 orbits) three pictures of a stone cat statue rendered from 45° above, and a caption asking for an orbit at 5°, near eye level. The statue is a free Poly Haven scan that was never in training, and Blender rendered the true 5° orbit to compare against.

It wasn't a one-off. In all four tests where the pictures came from 45° and the caption asked for something else (5° on three held-out objects, −30° on one), the old LoRA orbited at 45°. Its orbits were otherwise excellent: one turn, tight pacing, the right start view. It had been trained on four orbits, each with pictures taken from that same orbit, so "film from wherever the pictures were taken" had always been the right answer.

Lesson 3.1

Render every height, and say it in the caption

The fix was data. 23 objects (Poly Haven CC0 props and a few stylised character models) were rendered in Blender as 345 one-turn orbits at three lengths, each at the largest size that fit in 96 GB: 124 frames at 800×800, 73 at 1024×1024 and 22 at 1600×1600. Camera heights ran from 45° below the subject's centre to 55° above. While planning it, the question came up: what about negative angles, looking up through a transparent floor? Think getting angles for splat. So every orbit below the centre was rendered with the floor switched off.

• Each clip appears in three training rows with different reference layouts (front only, front and back, a turnaround, thirds), and one of the three uses pictures rendered from a different height than the orbit.
• Captions give the height in words and numbers ("a low camera below the subject, looking up, 30 degrees below the subject's centre") and label every picture with its time, frame and angle.
• 960 rows, 1,500 steps of rank-32 LoRA training, about 62 s per step (about 26 hours) on one 96 GB GPU.

Every checkpoint was scored on the same held-out tests. A tracker matched each generated frame to Blender renders of the true orbit at a grid of heights, which gives a height and an angle per frame, good to about ±5–7° in height. One seed per test.

Median measured height per clip. Step 1500 is the first checkpoint that put the statue at the right height and went round once: every earlier ladder checkpoint got the height right and then orbited twice.

Two more failures matter for splats. Changing height during one clip (45° above down to 30° below) made the object tumble instead of the camera moving. And on a real (non-Blender) image, height prompts did nothing yet: the cat asked for an orbit 35° above and 20° below came out at the image's own height both times, on two seeds each. Of the 345 training clips, only 109 had straight-on cross-height pictures with a higher orbit, and none had straight-on pictures with an orbit from below.

Lesson 4.1

Registering every frame isn't the same as recovering the orbit

The 124 frames went to a desktop with two RTX 5090s and the stock tools: Spirula Studio for structure-from-motion and splat training, and COLMAP as a cross-check. A generated orbit has no camera data, so every camera pose has to be solved from the pixels.

• Full frames, Spirula: 124 of 124 frames registered, yet the path covered only 94° and reversed direction 34 times. COLMAP's incremental solver on full frames kept collapsing.
• Masked to the cat: Spirula's built-in SAM 2.1 tracker cut out the cat and hat. With separate horizontal and vertical focal lengths, the path jumped 107° between neighbouring frames.
• Masked, one shared focal length: the smoothest solution. 124 of 124 frames, no reversals, the largest step between frames 4.52°, but only 316.6° of sweep.

The first and last frames are nearly the same picture, so their cameras should nearly coincide. Instead their solved orientations differ by 44.51°, their centres sit 0.67 units apart on an orbit of radius about 0.9, and the two frames share just 3 reconstructed 3D points out of about 430 tracked in each. Orbiting the finished splat by hand, Dylan saw it as the cat's pose jumping where the orbit wraps round.

Colour runs from frame 1 to frame 124.

Why the solver lost 44° is not established. The generated frames are probably not perfectly consistent with one rigid scene, so small errors pile up round the ring and nothing ties the end back to the start strongly enough. That is a hypothesis; it hasn't been tested.

Lesson 5.1

Spread the error round the ring

A short script, close_camera_loop.py, builds a separate experimental camera model and leaves the original alone. It takes the rigid transform that would move the last camera onto the first, converts it to a twist with the SE(3) logarithm, and applies a growing fraction of it to each frame: none at frame 1, all of it at frame 124. COLMAP then re-optimises only the 3D points, with every camera and the focal length held fixed, and the splat is retrained from scratch on the corrected cameras.

T_i' = exp( (i − 1)/123 · log( T_first · T_last⁻¹ ) ) · T_i        # camera-to-world, i = 1…124

The biggest gains are at the two ends of the orbit, where the seam was: frame 1 rose 2.7 dB and frame 121 rose 1.4 dB. Four of the 15 held-out frames got slightly worse (by 0.04–0.43 dB). The reprojection error went up, as expected: the points now have to agree with cameras that were moved, not solved.

Given a COLMAP text model of a generated 360° orbit whose first and last frames show the same view, write a script that creates a NEW model directory (refuse to overwrite) with an experimental loop prior: compute C = T_first · inverse(T_last) on camera-to-world matrices, take its matrix logarithm, and replace each camera T_i by exp(alpha_i · log C) · T_i with alpha_i running from 0 at the first frame to 1 at the last. Check the result stays a rigid transform, the first camera doesn't move and the last equals the first. Then re-triangulate / bundle-adjust points only, with cameras and intrinsics fixed, and train the splat on both models with the same settings. Report the sweep, largest step, reprojection error before and after, and PSNR on the same held-out frames, and label the closure as imposed.

It starts from the image's viewpoint and turns slowly; touch it to take over. Along the orbit it looks like the video. Leave the orbit, go above it or below it, and the ghosting and blur show what a single eye-level ring can't pin down: there were no views from there to learn from.

The file is the trained PLY (63.7 MB, spherical harmonics to degree 3) converted to Niantic's SPZ format with Spark's transcodeSpz: nothing removed, but positions, colours and rotations are quantised to 8–24 bits. The model is shown in its own reconstruction frame, turned so the orbit plane is level.

Lesson 8.1

The 1.37× sharper cat that had stopped orbiting

A later run trained tilted rings and vertical loops on top of step 1500, and its automatic report scored the cat at step 750 as having 1.37× the fine detail of step 1500. Looking at the frames showed why: on that seed the camera stalled at the side view and never reached the back, so the clip was mostly the image's sharp front. A sharpness score rewards a camera that doesn't move.

Two more seeds showed the stall was that seed's bad luck: step 750 orbits on both. The lesson stands either way.