The author explains how using GPT-4 for a nightly data extraction pipeline caused constant failures due to its non-deterministic nature. Even with strict prompting and temperature settings, the model would occasionally change key names or formatting, breaking the automated workflow. To solve this, the team switched to running smaller local models like Qwen2.5 via Ollama. By using seeded inference on their own hardware, they achieved the consistency needed for a reliable pipeline, finding that while small models lack GPT-4's reasoning depth, they are much better at performing repetitive, structured tasks identically every time.
