GazeCtrl one photo + one 3-vector → the person turns
GitHub Weights Labels About ↓
generated output source photograph INPUT

target direction

columns = yaw · rows = pitch · arrow keys work

All 60 directions for this person
How it works
target gaze (dx, dy, dz) └─▶ flat RGB map   R=(dx+1)/2   G=(dy+1)/2   B=(dz+1)/2 └─▶ VAE encode ──▶ InstantX ControlNet · TRAINED · 1768M └─ 5 residuals ─▶ Qwen-Image-Edit-2509 · FROZEN ──▶ output source photo ────────────────────────────────▶ in-context reference token (carries identity) text embedding = zero — every control signal arrives through the ControlNet

The condition is a single flat colour, so it says which way to face and nothing about where anything is. That is the point: the claim under test is that a large pretrained editing model already holds the head-pose ↔ body-pose prior, and the adapter only has to index it. No skeleton, no pose predictor, no text prompt.

Every image here: 25 source photographs × 12 yaw × 5 pitch = 1500 generations, one checkpoint (cn_stage2_step040000) at 1024², cn_scale 1.0, 16 denoising steps, one fixed seed — so differences across a grid come from the gaze condition alone. Ten of the sources are photographs the model has never seen.

How well it follows the condition
source groupgenerationsmedian |Δyaw|within 30°id_sim

Read these with the caveat that the instruments are unreliable. Direction is measured by a head-pose estimator (6DRepNet360), which disagrees with eye-gaze estimators by a median 26.7° on real photographs; id_sim is ArcFace cosine, averaged only over generations where a face was detected at all — which excludes the back-facing cases where identity is hardest. Both this model and its alternative sit near ArcFace's own same-person threshold (≈0.28), so identity preservation is marginal, not solved. The pictures carry more evidence than the numbers.

What is honest to say about this

Zero-shot identities degrade but do not collapse. Large rotations are where both direction accuracy and identity are weakest. Supervision is synthetic — turnaround videos generated with Wan2.2 — which holds identity fixed by construction but bakes that generator's artifacts into the targets. Facial detail is dominated by resolution, not by the model: the same weights and seed render an eye across 25 px or 106 px depending only on framing, a finding that arrived after two rounds of loss engineering had already been spent on it. The black bands on some outputs are letterbox from padding portrait photographs to a square canvas; they are left visible rather than cropped away. The repository documents the failures alongside the results.