Multimodal AI combines different data types in one model or connected pipeline. A system might answer questions about a screenshot, summarize a meeting recording, or use text and images together to understand a scene. Some models accept several modalities directly; others rely on separate components connected by application code.
Examples include uploading a photo of a receipt and getting the total back as JSON, or sending an audio clip and asking for action items. Each extra modality adds cost, size limits and privacy questions, so check which formats and file sizes your provider accepts and whether media is stored. Audio input usually relies on speech-to-text either inside the model or as a separate step.
Performance depends on the supported input formats and the task. Test real examples, including noisy audio, small text in images and mismatched context, and consider privacy when sending media to hosted services. Related: Vision-Language Model, Speech Recognition.