LO-World

Pretrained PyTorch checkpoints for LO-World, a BEV latent world model for joint future occupancy and LiDAR prediction.

Download

hf download JimmyLiaoo/LO-World \
  bev_world_6f.pth \
  bev_world_12f_autoregressive.pth bev_world_12f_layout_autoregressive.pth \
  bev_world_6f_stc.pth bev_world_6f_waymo.pth occ_lidar_vae.pth SHA256SUMS --local-dir ckpts

Checkpoints

The release contains six checkpoints for inference, fine-tuning and result reproduction.

File	Model
bev_world_6f.pth	6-frame nuScenes World Model; source for STCOcc and Waymo fine-tuning
bev_world_12f_autoregressive.pth	12-frame autoregressive World Model
bev_world_12f_layout_autoregressive.pth	12-frame autoregressive World Model with physical v1 BEVLayout conditioning
bev_world_6f_stc.pth	6-frame nuScenes World Model fine-tuned on STCOcc-conditioned latents
bev_world_6f_waymo.pth	6-frame Waymo World Model
occ_lidar_vae.pth	Joint occupancy-LiDAR tokenizer

Use the corresponding configuration from the LO-World code repository. The Layout checkpoint is the physical v1 fine-tuned epoch 8 EMA and must be paired with the corrected layouts extracted to data/bev_layout/. World Model checkpoints include the tokenizer parameters. Training and evaluation require separately prepared dataset files and tokenizer latents.

Download the STCOcc prediction results, generated with the r50_704_256_16f_occ3d checkpoint from STCOcc, and arrange them as data/nuscenes/stc-results/<scene_name>/<sample_token>/labels.npz. Use configs/tokenizer/occ_lidar_vae_stc_latent.py to prepare the matching input latents.

bev_world_6f.pth is the shared source checkpoint for STCOcc and Waymo fine-tuning. Both adaptations predict six future frames at 2 Hz (0.5–3 seconds). Their configurations load this complete checkpoint through load_from, keeping the tokenizer frozen. The STCOcc fine-tuning configuration is configs/world_model/bev_world_6f_stc_occ.py (8 epochs, AdamW learning rate 1e-5); the Waymo configuration is configs/world_model/bev_world_6f_waymo.py.

For bev_world_6f_stc.pth, use configs/world_model/bev_world_6f_stc_occ.py and matching STCOcc-conditioned validation latents at data/stc_occ_lidar_latent/<sample_token>.npz. This checkpoint's embedded tokenizer parameters are identical to the released occ_lidar_vae.pth. Encode the STCOcc predictions with that tokenizer:

bash tools/dist_test.sh configs/tokenizer/occ_lidar_vae_stc_latent.py \
  ckpts/occ_lidar_vae.pth 8

Evaluation targets remain the original Occ3D labels and LiDAR range images.

Verify downloads

cd ckpts
sha256sum -c SHA256SUMS

nuScenes infos

Prepared train/validation annotation infos are available under nus_infos/. They contain 28,130 training samples and 6,019 validation samples, respectively. Download them into the code repository's data/ directory:

hf download JimmyLiaoo/LO-World --include "nus_infos/*.pkl" --local-dir data

The resulting paths are data/nus_infos/nuscenes_infos_train.pkl and data/nus_infos/nuscenes_infos_val.pkl. Original nuScenes sensor data and Occ3D occupancy labels must be downloaded separately.

BEVLayout conditioning data

The complete 2 Hz train/validation BEVLayout arrays used by the Layout-conditioned models are available under data/bev_layout/. The package contains 34,149 binary uint8 arrays of shape (18, 200, 200): 28,130 train and 6,019 validation samples. It is approximately 22.9 GiB uncompressed and 77 MiB as split Zstandard archives.

Download the package into the LO-World code repository and extract the arrays to data/bev_layout/:

hf download JimmyLiaoo/LO-World --include "data/bev_layout/**" --local-dir .
mkdir -p data/bev_layout
tar --zstd -xf data/bev_layout/2hz/train.tar.zst -C data/bev_layout
tar --zstd -xf data/bev_layout/2hz/val.tar.zst -C data/bev_layout
(cd data/bev_layout && sha256sum -c SHA256SUMS)

These are 2 Hz keyframe layouts, not dense 12 Hz frames. The arrays are derived from nuScenes and are for non-commercial research use subject to the nuScenes terms. See the package card for format, split manifests, checksums, source terms, and citation.

Selected 12 Hz conditions: uniform physical timeline

Updated 2026-10-22 at the existing download paths. The train700 and val150 packages preserve all 850 original target tokens, clip order and seeds, with one 81-frame window per scene (68,850 total Layouts). They use the corrected physical clock t0 + round(k * 1e6 / 12) for ego poses, instance tracks and map rendering, replacing the old sweep-pose clock. The checkpoint and 2 Hz archives are unchanged.

hf download JimmyLiaoo/LO-World --include "data/bev_layout/12hz_train700_81f/**" --local-dir .
hf download JimmyLiaoo/LO-World --include "data/bev_layout/12hz_val150_81f/**" --local-dir .
(cd data/bev_layout/12hz_train700_81f && sha256sum -c SHA256SUMS)
(cd data/bev_layout/12hz_val150_81f && sha256sum -c SHA256SUMS)

Each condition NPZ contains 80 future Layouts, binary uint8 with shape (1,80,18,200,200), ego transforms, uniform timestamps and provenance. Each package now also includes current_layouts/ for frame 0 and selected_12hz_frames.pkl for the same frozen rendering timeline. The manifest, release validation summary and SHA256 checksums travel with each package.

These are selected windows, not full-scene 12 Hz coverage or high-frequency ground truth. Synthetic future tokens differ from the old sweep tokens; regenerate old predictions and videos rather than pairing them with the new annotations. Original RGB acquisition timestamps are retained, without RGB interpolation. Use the updated LO-World code for timeline/source validation.

Use data/bev_layout/12hz_val150_81f as --dataset-dir with --dataset-split val in the batch generator; use the train package with --dataset-split train. The updated preparation shell pipeline also stages these installed packages into its existing work_dirs generation paths.

See the training package and validation package.