How LoRA Actually Works: Low-Rank Decomposition, Weight Merging, and Memory Breakdown Under the Hood

Why full fine-tuning an 8B model requires 130+ GB VRAM, how low-rank matrix decomposition cuts parameters by 99%, and how weight merging enables zero-latency serving.