</>SkillRecallRECALL WHAT YOU KNOW
Prepare for an interviewExplore Library
AI Engineering / Working with LLMsLocal & Open-Source Model Deployment

67 items

1

Streaming Without a Background Thread

Code Quiz
2

AutoTokenizer and AutoModel Loading

Code Quiz
3

4-bit Loading with BitsAndBytesConfig

Code Quiz
4

Generating and Decoding Output

Code Quiz
5

Deploying Open Models on Cloud GPUs

Slides / Video
6

Securing Self-Hosted Model Servers

Slides / Video
7

Device Placement with device_map

Code Quiz
8

Half-Precision Loading with torch_dtype

Code Quiz
9

Configuring Sampling Parameters

Code Quiz
10

Loading a Model with the Pipeline

Code Quiz
11

Applying a Chat Template

Code Quiz
12

Slicing Off the Prompt Tokens

Code Quiz
13

Where trust_remote_code Belongs

Code Quiz
14

Stop Sequence with stop_strings

Code Quiz
15

Local Embedding and Reranker Models

Slides / Video
16

Benchmarking Local Inference Speed

Slides / Video
17

Serving LoRA Adapters Locally

Slides / Video
18

Containerizing Local Model Deployments

Slides / Video
19

Serving Quantized Models: Speed vs Quality

Slides / Video
20

Chat Templates Per Model

Slides / Video
21

OpenAI-Compatible Local API Endpoints

Slides / Video
22

CPU Offloading and Mixed Inference

Slides / Video
23

Model Sharding and Parallelism

Slides / Video
24

Estimating VRAM for a Model

Slides / Video
25

PagedAttention and KV Cache Management

Slides / Video
26

Production Serving Engines for LLMs

Slides / Video
27

Local Inference Runtimes Compared

Slides / Video
28

Model Weight Formats Explained

Slides / Video
29

Landscape of Open Models

Slides / Video
30

Estimating VRAM from Parameter Count

Code Quiz
31

Batched Generation Padding Side

Code Quiz
32

Merging a LoRA Adapter

Code Quiz
33

Loading a Model from a Local Path

Code Quiz
34

Authenticating for Gated HF Models

Code Quiz
35

GPU Offloading with n_gpu_layers

Code Quiz
36

KV Cache During Generation

Code Quiz
37

Local Embeddings with sentence-transformers

Code Quiz
38

max_new_tokens vs max_length

Code Quiz
39

Default context window too small

Code Quiz
40

Loading a GGUF with llama-cpp

Code Quiz
41

vLLM model name mismatch

Code Quiz
42

OpenAI client base_url path

Code Quiz
43

Reading an Ollama response

Code Quiz
44

Missing pad token for GPT-2

Code Quiz
45

Open Model Licensing and Commercial Use

Slides / Video
46

Open-Source vs Open-Weight vs Proprietary

Slides / Video
47

Why Run LLMs Locally

Slides / Video
48

Edge and On-Device Constraints

Flashcard
49

Chat Templates per Model

Flashcard
50

Cloud GPU Providers for Self-Hosting

Flashcard
51

Serving LoRA Adapters Locally

Flashcard
52

Evaluating Quantization Quality Loss

Flashcard
53

Containerizing Model Deployments

Flashcard
54

Security of Self-Hosted Inference

Flashcard
55

Context Length and RoPE Scaling

Flashcard
56

PagedAttention and vLLM

Flashcard
57

Hugging Face Hub Downloading

Flashcard
58

Local Inference Runtimes

Flashcard
59

Memory Offloading

Flashcard
60

Open-Source Model Landscape

Flashcard
61

Open Weights vs Open Source

Flashcard
62

GGUF Quantization Formats

Flashcard
63

Multi-GPU Inference

Flashcard
64

OpenAI-Compatible Local API

Flashcard
65

GPU vs CPU vs Apple Silicon

Flashcard
66

VRAM Estimation

Flashcard
67

Model Weight Formats

Flashcard
Code QuizAdvanced

Merging a LoRA Adapter

A script saves a base model unchanged because the LoRA merge method is never actually called.

Codepython
from peft import PeftModel
from transformers import AutoModelForCausalLM

base = AutoModelForCausalLM.from_pretrained("meta-llama/Llama-2-7b-hf")
model = PeftModel.from_pretrained(base, "./my-lora-adapter")

merged = model.merge_and_unload
merged.save_pretrained("./merged-model")

What is the bug that prevents the adapter from being merged?