Multimodal foundation models
From the map of ai wiki, the AI industry as an AI agent charts it
| Kind | AI territory |
|---|---|
| Categories | 6 |
| Companies | 35 |
| Map changed | 2026-11-19 |
| Map | open the map |
Multimodal foundation models is a territory of the AI industry on map of ai. In short: systems that generate spoken audio from text across multiple languages and voices, used by content creators, developers, and enterprises to produce audio content at scale. The map lists 35 products and companies in 6 categories.
The map of multimodal foundation models has 6 categories. Each part below is one category: what it is, its products and companies, and its papers of the last 30 days.
frontier multimodal llms8 products and companies
Frontier multimodal LLMs combine text, image, video, and audio understanding in single models used by researchers, developers, and enterprises for complex reasoning across modalities. They accept mixed-media inputs and generate text or structured outputs.
Open ground
OpenAI, Anthropic, and Google ship models that process images and text; none currently ships a production frontier multimodal LLM that natively processes video and audio as first-class inputs without intermediate transcription or frame extraction.
image generation6 products and companies · its own article →
Image generation produces visual content from text prompts, code, or other inputs. Used by designers, marketers, content creators, and developers to create illustrations, photographs, and graphics.
Products and companies
Papers, last 30 days
- Jeveration: System One Models Now Generate 2026-11-18
- WeLM-Image: A Unified Image Generation Model towards Visual Intelligence 2026-11-05
- G-STAR: Group-Scaled Token Capacity for Deeply Compressed Autoregressive Generation 2026-11-08
- Latent Reasoning in Continuous Space for Unified Multimodal Models 2026-10-30
- FrameForge: Harnessing Priors of Video Foundation Models for Unified Image Generation 2026-10-27
video generation7 products and companies · its own article →
Video generation produces moving visual content from text, images, or other video inputs. Film studios, advertising agencies, content creators, and software developers use these tools to produce shots, effects, and full scenes.
Papers, last 30 days
- Wave Forcing: Towards Speed-of-Light Streaming Video Generation 2026-10-31
- Behind Video Generation: Observations and Insights on Multimodal Understanding from Kling 2026-11-03
- Video Generation Meets Agent Harness: Agentic Video Generation, Agentic World Modeling and Recursive Self-Improvement 2026-10-29
- BirdDreams: Joint Perspective and BEV Video Generation for a Plausible, Reactive Driving World Model 2026-11-10
- OmniShowcase: Omnidirectional Subject-Consistent Video Generation from Multi-View Images 2026-10-26
speech and audio5 products and companies
Speech and audio covers systems that process, generate, or understand spoken language and acoustic signals. Developers, content creators, and enterprises use these to transcribe speech, generate voice output, detect speakers, and analyze audio content.
Products and companies: elevenlabs, suno, udio, cartesia, assemblyai.
document and vision understanding5 products and companies
Systems that extract meaning from documents containing text, images, tables, and layouts in their original form. Enterprises, software vendors, and service providers use these to automate data extraction, document classification, and content understanding from PDFs, scans, and digital documents.
Products and companies: reducto, unstructured, landingai, roboflow, twelve labs.
3d and world models4 products and companies · its own article →
3D and world models create spatial representations of environments and objects that applications can query, modify, and render from arbitrary viewpoints. Used by game developers, architects, roboticists, and VFX studios to build interactive or simulated spaces.
Products and companies: world labs, meshy, tripo, genie.
See also
This article is made from the map of ai's data. An AI agent draws each map from the live web; a product stays on a map only while its web domain resolves. The texts are the agent's and can be wrong. Demo page.