ApX logoApX logo

Mochi 1 Preview

Parameters

10B

Modality

Video

Auxiliary Parameters

4.86B

Architecture

Diffusion Transformer

License

Apache 2.0

Release Date

22 Oct 2024

API Pricing (per 1M)

Self-hosted only

Hardware Sizing

Precision

Workload Memory: 7.72 GB

INT4 · 1024x1024

DiT Backbone Weights: 4.66 GB

VAE Decode Workspace: 0.56 GB

Runtime & Graphs: 2.50 GB

Minimum Required Hardware

1x NVIDIA L4 24GB

7.7 / 21.6 GB usable

1x RTX 3060 12GB

7.7 / 10.8 GB usable

Apple M4 24GB

7.7 / 18.0 GB usable

Determine requirements for your specific workload:

Open in VRAM Calculator

Architecture Diagram

Noisy Video Latents3D Spatio-Temporal VAE Latents3D Patch Embedding3D Spatio-Temporal PatchifyHidden: 3.1k · Text: 4.86B enc · Timestep (t)x 48 DiT blocksAdaLN ModulationNorm · Timestep scale/shiftSpatio-Temporal Attention24 headsHead dim: 128+AdaLN ModulationNorm · Timestep scale/shiftFeed-Forward (MLP)Activation+Final Norm (AdaLN Scale)Predicted Velocity (v)Denoised 3D Latents to VAE

About Mochi 1 Preview

Mochi 1 Preview is a 10B parameter video DiT developed by Genmo, featuring Asymmetric Diffusion Transformer (AsymmDiT) blocks and 8.4x spatio-temporal VAE compression for motion fluidity.

Technical Specifications

Attention

Attention Heads

24

Attention Head Dimension

128

Dimensions

Auxiliary Parameters

4.86B

Hidden Dimension Size

3,072

Number of Layers

48

About Mochi


Other Mochi Models
  • No related models available