Fine tuning vs In Context (src. gemini cookbook)

Use Fine-tuning When:

High Performance and Accuracy are Crucial: For tasks requiring the best possible performance and accuracy in a specific domain, fine-tuning is generally preferred. It allows the model to deeply adapt its parameters to the nuances of your data.
Domain-Specific Tasks: If your application focuses on a narrow domain with specific vocabulary, style, or knowledge (e.g., legal documents, medical records, financial reports), fine-tuning can significantly improve the model’s understanding and output quality.
Consistent Output Format is Needed: Fine-tuning can enforce a specific output format or structure more reliably than in-context learning.
Cost-Effective Inference in the Long Run: While fine-tuning requires upfront computational cost and labeled data, the resulting specialized model can often be smaller and more efficient for inference, leading to lower long-term costs, especially for high-volume applications.

Rapid Prototyping and Experimentation: In-context learning is excellent for quickly testing the capabilities of an LLM on a new task without the time and resource investment of fine-tuning.
Flexibility and Task Switching: If your application needs to handle a wide variety of tasks or adapt to changing requirements on the fly, in-context learning allows you to guide the model with different prompts and examples without retraining.
Limited or No Labeled Data: When you don’t have a large, labeled dataset for your specific task, in-context learning can be a viable alternative by providing a few relevant examples directly in the prompt.
Utilizing General Capabilities of Large Models: If the task can be reasonably accomplished by leveraging the broad knowledge and general language understanding of a powerful pre-trained model with well-crafted prompts.