Choosing the Right Multimodal Model

Long-Context Models

Reasoning Models and Extended Thinking

Selecting a Multimodal Model

Multimodal Failure Modes

Vision with Function Calling

Multimodal Safety Risks

Image Generation Models Overview

Image Token Cost and Sizing

Image Detail/Resolution Settings

Common Input Modalities

Structuring Multimodal Messages

Text-to-Speech / Audio Output

Audio Input and Transcription

Multimodal RAG

Multimodal Embeddings (CLIP-style)

Prompting for Image Generation

What Multimodal Models Are

Vision-Language Models Overview

Video Input Handling

Passing Images to a Model

Image Understanding and Captioning

OCR via Multimodal Models

Document and Chart Understanding

Multiple Images in One Request

Embedding Models for Multimodal Search

Long-Context Models and Their Trade-offs

Reasoning Models and Extended Thinking

Multimodal Model Failure Modes

Context Window and Multimodal Inputs

Token Cost of Image and Audio Inputs

Generating Text Embeddings

Decoding Generated Image Output

JSON Mode Structured Output

Parsing to a Pydantic Schema

Extracting Reasoning Content

Setting Reasoning Effort

Requesting Image Generation

Reading Reasoning Token Usage

Multimodal Prompting Techniques

Document and PDF Understanding

Text-to-Speech Capabilities

Speech-to-Text Transcription Capabilities

Image Generation and Text-to-Image APIs

Image Understanding Use Cases

Passing Images to a Multimodal API

How Vision-Language Models See Images

Common LLM Modalities

Vision Input with Tool Calling

Image Detail and Token Cost

Cosine Similarity of Embeddings

Audio as a Base64 Content Block

What Multimodal Models Are

Selecting a Vision Model

Encode Local Image to Base64

Sending Multiple Images

Audio Transcription Input

Setting Image Detail Level

Text-to-Speech Output

Passing an Image URL

Mixing Text and Image Blocks

Combining Text and Images in a Prompt

Long Context Models and Their Capabilities

Reasoning Models and Extended Thinking

Multimodal Use Cases in Applications

Video Understanding and Frame Sampling

Image Generation Models

Latency and Cost of Multimodal Inputs

Limitations of Multimodal Models

Text-to-Image Prompting Basics

Visual Question Answering

Image Detail/Resolution Settings

Passing Images to an API

Audio Input and Transcription

Common Modalities

OCR with Multimodal Models

Image Input Formats

Vision-Language Models (VLMs)

What Multimodal Models Are

Text-to-Speech (Audio Generation)

Document and Chart Understanding

How Images Are Tokenized and Cost