Skip to content

MeshTransform: Slice F — Photo → rigged human (MHR runtime, body-shape sliders, CLI/MCP/GUI) #851

Description

@fernandotonon

Part of #845 · Design: docs/MESH_TRANSFORM_DESIGN.md §3.1 · Gated on Slice E go-decision.

Photo → rigged, skinned, animatable human via SAM 3D Body (ONNX) → MHR parameters → MHR template evaluation in C++. No auto-rig/skin pass needed — MHR ships artist topology + rig.

Pipeline: photo → person matte/crop (existing BackgroundRemover tier) → SAM 3D Body graph(s) → (β shape, θ pose) → MHR eval (blendshapes + LBS) → Ogre entity with skeleton + skin weights.

Product surfaces:

  • Object-mode "AI: Human from photo" (next to "AI: Image → 3D"): posed or rest-pose (θ=0, keep β) output
  • Body shape editor: β sliders Inspector section on MHR-derived entities — the reshapeable "dummy human" (by photo or by hand)
  • MHR→22-joint canonical name table next to canonicalIndexForBone → generated humans play the motion library / text-to-motion out of the box
  • Composition with MeshTransform: Slice D — MeshWrap (native Amberg NRICP + Eigen): qtmesh wrap, GUI, --keep-topology variants #849: wrap the user's own character onto the MHR fit = "my character with this photo's proportions", rig intact
  • CLI qtmesh human photo.jpg [--rest-pose] -o person.glb; MCP generate_human_from_photo (heavy, ONNX-gated)
  • Models on HF repo (human/ subdir), QTMESH_HUMAN_MODEL_BASE_URL/_NO_DOWNLOAD; Sentry ai.assist.human; THIRD_PARTY_AI_MODELS.md + CLAUDE.md entries

Metadata

Metadata

Assignees

No one assigned

    Labels

    ai-assistLocal-AI-assisted 3D workflows (epic prefix: AI:)enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions