OptimusMesh: Compact Autoregressive Mesh Generation from Point Clouds via Sparse Latent Pivots

Mazhar Iqbal¹ Xuanmeng Sha¹ Naoya Chiba¹ Yuki Uranishi¹ Tomohiro Mashita²

¹ The University of Osaka ² Osaka Electro-Communication University

Paper Code Coming soon Video Coming soon Data Coming soon

OptimusMesh maps a point-cloud input to sparse latent pivots, autoregressively decodes vertices, and then generates triangular faces to obtain a compact render-ready mesh.

Abstract

Generating compact and geometrically faithful 3D meshes directly from point clouds remains a fundamental challenge. Point clouds are unordered and sparse, whereas meshes exhibit irregular structure and varying topology. As a result, many existing approaches rely on implicit representations followed by surface extraction or reconstruction. Although effective, these pipelines can produce dense or over-smoothed meshes, often requiring computationally expensive post-processing and simplification. We present OptimusMesh, a framework for direct compact triangle mesh generation from point clouds using sparse latent pivot conditioning. Our key idea is to compress 2,048 oriented input points into only 16 sparse latent pivots, reducing the geometric conditioning set by 128×. These pivots provide a compact structural representation shared across a two-stage autoregressive framework that first generates mesh vertices and then predicts triangular faces conditioned on the generated vertices and the same pivots. Compared with the evaluated recent point-cloud-conditioned autoregressive methods, which use 257 decoder-conditioning tokens, OptimusMesh uses only 16, yielding a 16.1× shorter conditioning sequence. Experiments show that OptimusMesh produces the most compact outputs among the compared recent autoregressive methods, using 25.7%–94.1% fewer faces while maintaining competitive geometric fidelity and distributional quality.

Comparison with Baselines

Same object IDs are shown across methods.

Object	Input PC	OptimusMesh	FastMesh	MeshAnything	NKSR	MeshAnythingV2	MeshRipple	PSR	SAP
Table
Chair
Display
Table

Application

From Point Clouds to a Populated 3D Scene

We use OptimusMesh to populate an indoor scene with compact assets generated directly from point-cloud observations. The video follows the progression from observed points to explicit triangle meshes and then places the generated objects together, illustrating how compact outputs can serve as lightweight, render-ready scene assets.

Each scene object is reconstructed from its point-cloud observation using the same sparse-pivot-conditioned generation pipeline.

Generated Mesh Samples

Rotating examples of generated meshes and compact reference meshes.

Interactive 3D Views

Drag to rotate. Scroll to zoom.

Compact table

Compact chair

Compact laptop

Approach

Method Overview

OptimusMesh compresses an oriented point cloud into 16 sparse latent pivots and uses them to directly generate a compact triangle mesh.

OptimusMesh converts point clouds into compact triangle meshes using sparse latent pivots — From point-cloud observation to a compact, render-ready triangle mesh.

Point-cloud encoder — The point-cloud encoder converts 2,048 oriented points into 16 learned 48-D latent pivots that summarize the object's global structure.

Complete OptimusMesh sparse-pivot-conditioned autoregressive architecture — The sparse pivots condition a vertex Transformer followed by a face Transformer, producing vertices first and triangular connectivity second.

BibTeX

@inproceedings{optimusmesh2026,
  title     = {OptimusMesh: Compact Autoregressive Mesh Generation from Point Clouds},
  author    = {Iqbal, Mazhar and Sha, Xuanmeng and Chiba, Naoya and Uranishi, Yuki and Mashita, Tomohiro},
  booktitle = {Conference},
  year      = {2026}
}