Choosing the Right Multimodal Model

Audio Input and Transcription

Text-to-Speech / Audio Output

Video Input Handling

Prompting for Image Generation

Image Detail/Resolution Settings

What Multimodal Models Are

Vision-Language Models Overview

Passing Images to a Model

Image Understanding and Captioning

OCR via Multimodal Models

Document and Chart Understanding

Multiple Images in One Request

Multimodal Safety Risks

Selecting a Multimodal Model

Reasoning Models and Extended Thinking

Vision with Function Calling

Multimodal Failure Modes

Long-Context Models

Structuring Multimodal Messages

Image Generation Models Overview

Image Token Cost and Sizing

Common Input Modalities

Multimodal Embeddings (CLIP-style)

Multimodal RAG

Embedding Models for Multimodal Search

Long-Context Models and Their Trade-offs

Reasoning Models and Extended Thinking

Multimodal Model Failure Modes

Context Window and Multimodal Inputs

Token Cost of Image and Audio Inputs

Reading Reasoning Token Usage

Parsing to a Pydantic Schema

JSON Mode Structured Output

Extracting Reasoning Content

Setting Reasoning Effort

Decoding Generated Image Output

Generating Text Embeddings

Requesting Image Generation

Multimodal Prompting Techniques

Document and PDF Understanding

Text-to-Speech Capabilities

Speech-to-Text Transcription Capabilities

Image Generation and Text-to-Image APIs

Image Understanding Use Cases

Passing Images to a Multimodal API

How Vision-Language Models See Images

Common LLM Modalities

Audio as a Base64 Content Block

Image Detail and Token Cost

Vision Input with Tool Calling

Cosine Similarity of Embeddings

What Multimodal Models Are

Mixing Text and Image Blocks

Passing an Image URL

Setting Image Detail Level

Text-to-Speech Output

Audio Transcription Input

Encode Local Image to Base64

Sending Multiple Images

Selecting a Vision Model

Latency and Cost of Multimodal Inputs

Image Generation Models

Video Understanding and Frame Sampling

Multimodal Use Cases in Applications

Long Context Models and Their Capabilities

Limitations of Multimodal Models

Text-to-Image Prompting Basics

Reasoning Models and Extended Thinking

Combining Text and Images in a Prompt

Document and Chart Understanding

Visual Question Answering

How Images Are Tokenized and Cost

Image Detail/Resolution Settings

Passing Images to an API

Text-to-Speech (Audio Generation)

What Multimodal Models Are

Vision-Language Models (VLMs)

Image Input Formats

OCR with Multimodal Models

Audio Input and Transcription

Common Modalities