Fine-Tuning in Deep Learning: Fundamentals, Methodologies, Applications, and Future Trends

Introduction and Fundamentals of Fine-Tuning

Fine-tuning is a fundamental deep learning technique essential for adapting pre-trained models to specific tasks or use cases 1. It is considered a specialized subset of transfer learning 1, serving as a critical bridge between general model knowledge and domain-specific applications.

To fully understand fine-tuning, it is crucial to first define its relationship with transfer learning:

• Transfer Learning: Transfer learning is a machine learning technique where a model trained on one task is reused as the starting point for another, often related, task 2. This approach leverages knowledge captured in a large pre-trained model instead of building a model from scratch 2. It typically involves keeping most of the original parameters fixed and adapting only the final layers, which allows for good results with less data, training time, and computational costs 2.
• Fine-tuning: Fine-tuning is the process of taking a pre-trained model and updating its parameters on a new dataset to enable it to perform well on a specific task 2. This involves adjusting the model's parameters to better suit the nuances of the target data 3. Unlike transfer learning where many weights often remain frozen, fine-tuning allows some or all layers to continue learning during training 2.

Relationship and Differences with Transfer Learning

Fine-tuning extends transfer learning by further adapting a pre-trained model's parameters to a new dataset . The primary distinctions between transfer learning and fine-tuning lie in their training scope, parameter update mechanisms, data requirements, computational cost, and performance trade-offs, as detailed below:

Motivations and Advantages of Fine-tuning

The adoption of fine-tuning is driven by several significant motivations and offers distinct advantages in modern machine learning workflows:

1. Leveraging Pre-trained Models: Fine-tuning capitalizes on the general knowledge and robust feature representations that large models have learned from vast, diverse datasets (e.g., ImageNet for vision, BERT for language) . This approach is considerably more efficient and cost-effective than training a new model from scratch, particularly for deep learning models with millions or billions of parameters 1.
2. Adapting to Specific Tasks and Domains: It enables models to specialize and perform effectively on target tasks that may differ from the original pre-training objective, or to specific domains such as medical imaging or legal text analysis . This adaptation allows models to capture more domain-specific and nuanced features crucial for specialized applications 3.
3. Data Scarcity Mitigation: Fine-tuning proves especially valuable in scenarios where labeled data for the target task is limited, such as in medical imaging or niche text classification . By starting with a model that already possesses strong generalized capabilities, fine-tuning helps mitigate the need for extensive labeled data 3.
4. Achieving Higher Accuracy and Customization: While transfer learning establishes a strong baseline, fine-tuning consistently enhances model performance, leading to significant improvements in various metrics 3. It also allows for deep customization, including conversational tone, illustration style, or the incorporation of proprietary data, which is vital for enterprise and production use cases .
5. Efficiency over Training from Scratch: Compared to training large models from scratch, fine-tuning significantly reduces the computational power and labeled data required 1. It accelerates the training process and lowers the risk of overfitting that might otherwise occur if a large model were trained from scratch on a small dataset 1.

Full Fine-tuning versus Parameter-Efficient Fine-tuning (PEFT)

Modern fine-tuning strategies can generally be categorized into two main paradigms: full fine-tuning and parameter-efficient fine-tuning (PEFT).

1. Full Fine-tuning (Fine-tuning All Parameters) Full fine-tuning involves updating all layers of the pre-trained model during the training process . This method resembles the pre-training process but benefits from pre-initialized weights and typically uses a smaller, task-specific dataset 1. It is particularly effective when the target domain is significantly different from the source domain, allowing the model to fully adapt 3. However, this approach can be computationally expensive and may require substantial annotated data to prevent overfitting 3. To preserve useful pre-trained features and avoid catastrophic forgetting, a smaller learning rate is often employed .

2. Parameter-Efficient Fine-tuning (PEFT) PEFT methods are specifically designed to reduce the number of trainable parameters that need to be updated, making fine-tuning more computationally efficient, especially for very large models with billions of parameters . These techniques update only a small subset of the model's parameters while keeping the majority frozen, thereby reducing compute and memory costs . PEFT methods have demonstrated greater stability compared to full fine-tuning, particularly within Natural Language Processing (NLP) contexts 1. High-level categories of PEFT include:

• Partial/Selective Fine-tuning: This approach involves fine-tuning only specific layers of the pre-trained model, while other layers (typically the earlier, more generic feature-extracting layers) remain frozen . The later layers, which are responsible for capturing more task-specific features, are usually the ones updated .
• Additive Fine-tuning: This method adds extra parameters or new layers to the model, training only these new components while keeping the original pre-trained weights frozen 1. Examples include:
  • Prompt Tuning: This technique trains "soft prompts," which are learnable vector embeddings concatenated to the user's input prompt 1. These prompts effectively guide the frozen model's behavior without altering its core weights 1.
  • Adapters: Adapters inject new, task-specific layers (adapter modules) into the neural network and train only these modules .
• Reparameterization-based methods: Techniques such as Low-Rank Adaptation (LoRA) leverage low-rank transformations to represent updates to the model's weights . LoRA, for instance, optimizes a matrix of updates (delta weights) that is represented as two smaller, lower-rank matrices, dramatically reducing the number of trainable parameters 1.

Types and Methodologies of Parameter-Efficient Fine-Tuning (PEFT)

Parameter-Efficient Fine-Tuning (PEFT) techniques are engineered to adapt large pre-trained models, such as Large Language Models (LLMs), to specific downstream tasks more efficiently than full fine-tuning 5. Given that modern LLMs can contain billions of parameters, full fine-tuning is often computationally expensive, memory-intensive, and susceptible to overfitting or catastrophic forgetting 5. PEFT methods overcome these challenges by either training a small subset of the model's parameters or introducing a few additional trainable parameters, thereby significantly reducing computational costs, memory requirements, and training time while maintaining competitive performance 5.

General Advantages of PEFT over Full Fine-Tuning

PEFT offers several key advantages over traditional full fine-tuning:

• Reduced Computational Costs Requires fewer GPUs and less GPU time, making advanced AI more accessible 5.
• Faster Training Times Accelerates the fine-tuning process 5.
• Lower Hardware Requirements Works with less expensive GPUs and lower VRAM 5.
• Better Modeling Performance Reduces the risk of overfitting by limiting the number of trainable parameters 5.
• Less Storage Only the small, task-specific parameters need to be stored, not entire model copies, significantly reducing checkpoint sizes 5.
• No Catastrophic Forgetting By preserving most of the initial parameters, PEFT safeguards against the model forgetting previously learned knowledge 6.
• More Flexible AI Enables data scientists to customize general LLMs to individual use cases without excessive resource consumption 6.

Prominent PEFT Methodologies

PEFT methods can be broadly categorized into Additive, Selective, and Reparameterization-based techniques 7. The following sections detail prominent methodologies:

1. Adapter Tuning

Adapter tuning introduces small, task-specific neural network modules, known as adapters, into a pre-trained model while freezing most of its original parameters 6. These modules are typically bottleneck-style feed-forward networks inserted within each transformer layer, often after the attention and feed-forward sublayers 8. An adapter layer reduces the input dimensionality, applies a non-linearity, and then restores the original dimensionality 9. Adapters add a small number of trainable parameters, typically comprising 0.1% to 6% of the total model parameters, significantly reducing the parameters needing optimization compared to full fine-tuning 9. This reduction translates to decreased memory usage and computational cost during fine-tuning 7. Adapter methods can achieve performance comparable to full fine-tuning, sometimes reaching 90-98% of its accuracy 6. They are suitable for multi-domain applications, image classification, and general NLP tasks where modularity is desired 7. The primary drawback of adapter tuning is the introduction of additional inference latency because the added adapter layers must be processed sequentially 5. Managing and versioning adapters can also become complex 10.

2. LoRA (Low-Rank Adaptation)

LoRA hypothesizes that the changes in weights during model adaptation have a low intrinsic rank 5. Instead of directly updating the entire weight matrix (W) of a pre-trained model, LoRA introduces two smaller, low-rank matrices (A and B) whose product approximates the weight update (ΔW) 5. The original weights (W) are frozen, and only these low-rank matrices (A and B) are trained 5. This update is injected into existing layers, often targeting attention layer weights 8. After training, ΔW can be merged directly into W to eliminate inference latency 5. LoRA significantly reduces the number of trainable parameters, often to 0.01% to 0.5% of the total, leading to substantial savings in time, memory, and computational cost 5. For GPT-3 175B, it reduced VRAM consumption during training from 1.2TB to 350GB and checkpoint size from 350GB to 35MB 5. LoRA consistently achieves performance comparable to, and sometimes even better than, full fine-tuning across various model architectures and tasks 5. Its performance is stable even with very few trainable parameters 8. LoRA is widely used for LLMs and image generation models like Stable Diffusion 11. It is excellent for multi-domain applications, low-budget teams, and scenarios requiring frequent model adaptation or task-switching due to its efficiency and lack of inference latency 5. LoRA is considered highly efficient and often outperforms other PEFT methods, especially in very large models 7.

3. QLoRA (Quantized Low-Rank Adaptation)

QLoRA extends LoRA by introducing quantization, further reducing memory usage. It freezes a pre-trained language model that has been quantized to 4-bit precision (specifically, 4-bit NormalFloat, NF4) 5. Gradients are then backpropagated into LoRA adapters, which are trained on top of this quantized base model 5. QLoRA employs techniques like Double Quantization and Paged Optimizers to optimize memory 5. While weights are stored in 4-bit NF4, computations typically occur in 16-bit BrainFloat after dequantization 5. QLoRA achieves even higher memory efficiency than LoRA, reducing memory usage by up to 75% compared to FP16 weights 5. It enables fine-tuning models with 10B+ or even 65B parameters on consumer-grade GPUs with limited VRAM 5. This significant memory reduction comes at the cost of slightly reduced training speed 8. Despite being more parameter-efficient, QLoRA preserves high model quality and performance, often on par with or superior to fully fine-tuned 16-bit models 5. It has achieved state-of-the-art chatbot performance comparable to ChatGPT 5. QLoRA is ideal for fine-tuning massive models on limited GPU memory 5. While highly efficient, QLoRA can be more complex to implement than standard LoRA due to the quantization process 11.

4. Prefix Tuning

Prefix tuning keeps all language model parameters frozen and instead optimizes a small, continuous task-specific vector, referred to as a "prefix" 6. This prefix is prepended to the hidden states of the multi-head attention mechanism in each transformer layer 6. These prefix tokens are learnable parameters that condition the LLM's output 8. To handle training instability, prefixes are initially generated through a feed-forward network, which is discarded after training, retaining only the prefixes 7. Prefix tuning stores significantly fewer parameters than fully fine-tuned models, often 0.1% to 4.0% of the total model parameters, leading to reduced computational requirements and memory overhead 6. It achieves performance comparable to full fine-tuning, especially on natural language generation (NLG) tasks and in few-data settings 6. It shows massive gains over P-Tuning and can outperform full fine-tuning in some cases 9. Prefix tuning is primarily designed for NLG tasks 6. A key trade-off is that it reduces the model's usable sequence length 5. The fine-tuning process can also be less stable and difficult to optimize compared to LoRA 8.

5. Prompt Tuning

Prompt tuning is a simpler form of soft prompting where a trainable tensor, or "soft prompt," is prepended to the model's input embeddings 6. During training, only these soft prompt tokens are updated, while the base model's parameters remain completely frozen 7. The soft prompts are continuous vectors automatically optimized through task-specific training data 7. Prompt tuning is highly parameter-efficient, with the learned parameters often being in kilobytes (KBs) 9. This allows for a single base model to be used for multiple tasks, with only the small prompt tokens needing to be stored per task 9. Performance becomes competitive with full fine-tuning primarily for very large models (over 10 billion parameters) 6. For smaller models, its performance may be inferior to traditional fine-tuning 7. Prompt tuning is ideal for classification and text generation tasks, especially when the model already possesses the necessary capabilities and the goal is to optimize how it understands the prompt 9. It is good for prototyping, lightweight deployments, and dynamic personalization 10. The model itself doesn't learn new information; it's purely a prompt optimization task 9, offering limited domain adaptation 10. It can introduce inference overhead due to additional tokens, particularly in transformer models with quadratic complexity 7.

P-Tuning (A Variant of Prompt Tuning)

P-Tuning, a variation of prompt tuning, focuses on Natural Language Understanding (NLU) tasks 6. It introduces trainable continuous vectors ("pseudo tokens") into a prompt template, which are optimized by a prompt encoder, often a Bi-LSTM network 9. The model's original weights remain frozen, and only the prompt encoder and pseudo tokens are trained 9. Prompt tokens can be flexibly positioned within the input sequence 7. P-Tuning is efficient in terms of parameter usage 9. It enables GPT-like models to achieve or surpass BERT-like performance in NLU tasks and can perform better than full fine-tuning on some benchmarks 9. P-Tuning is designed for NLU tasks, knowledge probing, and few-shot learning 6. Similar to prompt tuning, it's an assistive technique; if the base model lacks capability for a given task, P-Tuning won't significantly improve it 9.

Comparative Analysis of Efficiency, Performance, and Applicability

The table below provides a comparative overview of different fine-tuning methodologies:

Identification of Situations Where Certain PEFT Methods Are Preferred

The choice of PEFT method depends heavily on the specific task, available resources, and desired trade-offs:

• Full Fine-Tuning is preferred for tasks requiring a massive domain shift where the new task is fundamentally different from the pre-training data, or in high-stakes, high-budget environments where maximal accuracy is paramount and resource constraints are minimal 10.
• LoRA is widely adopted and highly versatile, suitable for adapting LLMs and vision models for various tasks. It is preferred when balancing efficiency with strong performance and avoiding inference latency 5. It excels in scenarios where a single base model needs to be adapted for multiple distinct tasks without incurring the cost of storing multiple full models 8.
• QLoRA is the optimal choice for fine-tuning extremely large models (e.g., 65B+ parameters) on hardware with limited GPU memory 5. It prioritizes memory efficiency without significantly sacrificing performance 5.
• Adapter Tuning is effective for multi-domain applications and modular workflows, especially when managing specific task adaptations as independent units. It is suitable when moderate efficiency gains are acceptable and the slight inference latency is not critical 10.
• Prefix Tuning is particularly well-suited for Natural Language Generation (NLG) tasks, especially in few-shot learning scenarios, where it can achieve high performance by conditioning the model across multiple layers 7.
• Prompt Tuning is ideal for classification or text generation tasks where the base model already has general proficiency, and the goal is to optimize its understanding of specific prompts rather than teaching it entirely new capabilities. It's excellent for lightweight deployment, prototyping, and when treating the LLM as a black box 9.
• P-Tuning is a strong option for Natural Language Understanding (NLU) tasks, particularly in few-shot settings. It's preferred when optimizing prompt representations to leverage the base model's existing knowledge effectively 6.

Applications and Real-World Impact of Fine-Tuning

Fine-tuning is a fundamental deep learning technique that extends transfer learning, enabling pre-trained models to adapt to specific tasks or use cases 1. While previous sections detailed the technical aspects and various Parameter-Efficient Fine-Tuning (PEFT) methodologies, this section comprehensively explores the profound real-world impact and diverse applications of fine-tuning. PEFT methods, by significantly reducing computational costs, memory requirements, and training times, have democratized access to fine-tuning, especially for large models, thereby facilitating its widespread adoption and impact across numerous AI domains. This adaptation bridges the gap between general knowledge gleaned from vast datasets and the specific expertise required for nuanced tasks, leading to improved accuracy, efficiency, and domain specialization 12.

Fine-Tuning Applications in Natural Language Processing (NLP)

Fine-tuning is extensively employed in NLP to customize Large Language Models (LLMs) and other models for a broad spectrum of text-based tasks. The methodologies include Supervised Fine-Tuning, Reinforcement Learning from Human Feedback (RLHF), and various PEFT techniques such as Low-Rank Adaptation (LoRA), Quantized Low-Rank Adaptation (QLoRA), Prefix tuning, and Prompt tuning, which drastically reduce computational and storage needs 12.

Sentiment Analysis

Fine-tuning plays a critical role in evaluating and identifying the emotional tone in textual data, categorizing sentiments as positive, negative, or neutral, and discerning specific emotions 13.

• Healthcare: Fine-tuned models monitor patient feedback, online reviews, and public sentiment to enhance patient satisfaction and inform health policies 13.
• Financial Markets: Extracting valuable information from news, social media, and reports aids in informed decision-making and risk assessment. FinBERT, a BERT-based language model, leverages domain-specific fine-tuning for financial sentiment analysis 13.
• Customer Support: Analyzing customer interactions (support tickets, chat logs) helps identify problems and improve service quality 13. BERT-based classifiers have proven effective in Portuguese customer support conversations 13.
• E-commerce: Analyzing product reviews and social media provides insights into customer satisfaction and enables real-time product recommendations 13. The ERNIE-BiLSTM-Att model enhances sentiment analysis in Chinese e-commerce product reviews 13.
• Social Media & Product Reviews: Fine-tuning facilitates the classification of emotional tones in posts and comments, and automates sentiment identification in customer feedback to pinpoint areas for enhancement 13.

Specialized LLMs (Medical Domain)

LLMs are fine-tuned for specialized medical use cases, including preconsultation, diagnosis, treatment management, medical education, and medical writing 12. Examples of such specialized LLMs include BioBERT, BioGPT, Med-Alpaca, Med-PaLM, and PMC-LLaMA 12.

• EHR Integration: Fine-tuned LLMs can generate patient summaries from electronic health records (EHRs), assist healthcare providers with decision-making, and perform named entity recognition and predictive diagnosis 12.
  • Case Study: EchoGPT: A Llama-2 model fine-tuned using instruction tuning with QLoRA, EchoGPT generated echocardiography reports that outperformed other LLMs and were rated comparably to human cardiologists 12. QLoRA dramatically reduces GPU memory requirements, exemplified by fine-tuning LLaMA 65B from over 780GB to 48GB 12.
  • Case Study: CohortGPT: Built on ChatGPT and GPT-4, CohortGPT identifies eligible patients for clinical trials from unstructured data by leveraging Chain-of-Thought (CoT) prompting with reinforcement learning 12.

Other NLP Applications

Instruction-tuning fine-tunes pre-trained LLMs to follow specific task instructions, such as translating sentences from one language to another 12. Retrieval Augmented Generation (RAG) also combines fine-tuned LLMs with information retrieval systems to generate contextually rich and accurate responses 12.

Fine-Tuning Applications in Computer Vision (CV)

Fine-tuning in computer vision, particularly with Vision-Language Models (VLMs), is pivotal for enabling advanced decision-making and performance in complex visual tasks.

Decision-Making Agents

Large VLMs fine-tuned with reinforcement learning (RL) serve as effective decision-making agents in multi-step, goal-directed tasks within interactive environments 14. This approach allows VLMs to efficiently learn optimal strategies 14.

• Case Studies: Fine-tuned VLMs have been applied to tasks requiring fine-grained visual recognition and arithmetic reasoning, such as Gym Cards (including NumberLine, EZPoints, Points24, and Blackjack), and in embodied AI environments like ALFWorld, which tests visual semantic understanding 14.
• Impact: Fine-tuning significantly enhances the decision-making abilities of VLMs. Seven billion parameter models, when fine-tuned, have outperformed commercial models like GPT-4V and Gemini in various tasks 14. The integration of Chain-of-Thought (CoT) reasoning is crucial, leading to a 27.1% improvement in arithmetic tasks and a 4.0% improvement in visual semantic tasks compared to supervised fine-tuned models 14.

Fine-Tuning in Speech Recognition (SR) and Other AI Domains

Fine-tuning is essential for developing robust speech technology, especially in low-resource settings and challenging applications.

Automatic Speech Recognition (ASR)

ASR systems have transitioned from traditional hybrid modeling to end-to-end (E2E) encoder-decoder and Transducer-based modeling, with fine-tuning of large pre-trained models being key to overcoming data scarcity 15.

• Contextual Knowledge Integration: Integrating contextual knowledge during decoding and training (e.g., contextual biasing with the Aho-Corasick algorithm) improves ASR model quality and Named Entity (NE) accuracy 15.
• Low-Resource ASR: Fine-tuning approaches, such as pseudo-labeling and knowledge distillation from larger models, help develop streaming ASR models without extensive supervised data 15.

Spoken Language Understanding (SLU)

Fine-tuning also extends to SLU, focusing on comprehending speech, often in a cascaded setting with ASR, and exploring E2E approaches for tasks like intent classification and slot filling 15.

Air Traffic Control (ATC) Communications

Fine-tuning ASR systems for ATC addresses specific challenges such as Named Entity Recognition (NER), callsign recognition, and speaker role detection, which are critical for safety and efficiency 15.

Unified Architectures

Novel architectures like STAC-ST (Speaker-Turn Aware Conversational Speech Translation System) and TokenVerse leverage fine-tuning to seamlessly handle multiple speech and NLP tasks within a single model via special tokens. These architectures support Speaker Change Detection (SCD), Endpointing (ENDP), Named Entity Recognition (NER), ASR, and Speech-to-Text Translation (ST) 15.

Demonstrated Practical Benefits and Impact of Fine-Tuning

The widespread application of fine-tuning translates into several significant practical benefits across various AI domains:

• Accuracy Improvements: Fine-tuning consistently leads to substantial improvements in model accuracy for specialized tasks. For instance, it enhances sentiment classification systems 13 and boosts decision-making capabilities in VLMs, achieving a 27.1% improvement in arithmetic tasks 14.
• Faster Convergence and Resource Efficiency: By starting with a pre-trained model, fine-tuning enables models to train more efficiently, requiring less time and memory compared to training from scratch 12. PEFT techniques, such as QLoRA, dramatically reduce computational resource requirements, making advanced AI more accessible .
• Handling Data Scarcity: Fine-tuning is particularly valuable in scenarios with limited labeled data. By leveraging robust pre-trained models, it performs well even with smaller quantities of task-specific data, which is crucial in data-scarce domains like low-resource speech recognition .
• Domain Adaptation and Specialization: Fine-tuning allows general-purpose models to acquire domain-specific knowledge and optimize for particular tasks, such as specialized medical report summarization or financial sentiment analysis 12. This customization is crucial for enterprise and production use cases .
• Outperforming Commercial Models: Fine-tuned models, such as 7 billion parameter VLMs, have demonstrated the ability to outperform established commercial models like GPT-4V and Gemini in specific decision-making tasks, highlighting the power of specialization 14.

In conclusion, fine-tuning, especially when powered by parameter-efficient methodologies, is a critical process for developing reliable and high-performing real-world AI applications. It ensures that models are optimized for specialized needs while maintaining efficiency and achieving state-of-the-art performance across diverse domains 12.

Latest Developments, Trends, and Research Progress (as of 2025)

The field of fine-tuning, particularly for large language models (LLMs), has seen rapid advancements from late 2023 to mid-2025. Key research directions have focused on parameter efficiency, continual learning, and data efficiency, addressing challenges like computational resource minimization and catastrophic forgetting .

Novel Parameter-Efficient Fine-Tuning (PEFT) Methods and Extensions

Parameter-Efficient Fine-Tuning (PEFT) methods remain central to adapting large pre-trained models to specific tasks by minimizing computational overhead . Recent developments have introduced numerous new variants and improvements:

Reparameterization-based PEFTs:

• DoRA (Differentiable Rank Adaptation), introduced by Liu et al. in 2024, has shown superior performance compared to LoRA .
• LoRA Derivatives continue to evolve, including dynamic rank approaches such as DyLoRA, AdaLoRA, SoRA, CapaBoost, and AutoLoRA. Further improvements encompass Laplace-LoRA, LoRA Dropout, PeriodicLoRA, LoRA+, and MoSLoRA. Multi-LoRA methods like LoRAHub, MOELoRA, MoLORA, MoA, MoLE, and MixLoRA also expand on the foundational LoRA technique, which itself remains widely effective in 2024 studies .

Additive PEFTs:

• Point-PEFT (Tang et al., 2023, 2024) is a novel framework that improves performance for 3D pre-trained models, requiring only 5% of trainable parameters compared to full fine-tuning 16.
• LLaMA-Reviewer (Lu et al., 2023) applies PEFT techniques to the LLaMA model to automate code review, demonstrating strong results in predicting review necessity, generating comments, and refining code 16.
• Prompt-based PEFTs have seen advancements through methods like SPoT, TPT, InfoPrompt, PTP, DePT, SMoP, and IPT, all aimed at enhancing prompt tuning stability, convergence speed, and parameter efficiency. Prompt tuning, which involves trainable input embeddings, is also a subject of 2024 research .
• (IA)3 (Infused Adapter by Inhibiting and Amplifying Inner Activations) utilizes learnable rescaling vectors for key, value, and FFN activations to achieve high parameter efficiency. Its effectiveness is being evaluated in 2024 studies for unit test generation .
• SSF (Scale and Shift Factor) is another additive method that performs linear transformations on model activations, with SSF-ADA layers merged into model weights during inference to avoid overhead 16.
• IPA (Inference-Time Policy Adapters) aligns LLMs with user-specific requirements by combining outputs from a base LLM and a smaller adapter model during decoding, without altering the base model parameters 16.

Hybrid PEFT Methods: These combine principles from multiple PEFT categories, including UniPELT, S4, MAM Adapter, NOAH, AUTOPEFT, LLM-Adapters, and S3PET 16. For instance, UniPELT integrates adapters, prefixes, and LoRA, while Compacter combines adapters with low-rank decomposition 17. P Tuning is also recognized as a notable PEFT approach in a 2024 survey 18.

Strategies and Breakthroughs for Fine-Tuning Extremely Large Models (100B+ Parameters)

Fine-tuning ultra-large models presents significant computational challenges. PEFT provides a practical solution by drastically reducing the number of tunable parameters, thereby minimizing RAM consumption and training time . For example, PEFT can reduce the training cost of LLaMA-3.1 405B from 40 million GPU hours to 400K GPU hours or less 18.

Notable Model Releases (2024-2025):

• DeepSeek-V3 (December 2024), a 671B-parameter Mixture-of-Experts (MoE) LLM, has demonstrated state-of-the-art performance with FP8 precision, achieving significantly lower training and inference costs compared to models like LLaMA 3-405B and Claude 3.5 Sonnet. Its successor, DeepSeek-R1 (2025), is a reinforcement learning-focused model exhibiting comparable reasoning performance to OpenAI-o1 at 2% of the computational cost 18.
• LLaMA-3.1 (April 2024, 405B) and LLaMA-3.2 (September 2024, including 1B/3B text-only and 11B/90B vision models for mobile) illustrate ongoing efforts to scale models efficiently 18.
• The OpenAI o1 family (September 2024), comprising o1-preview and o1-mini, excels in complex reasoning tasks. o3-mini (January 2025) offers a highly cost-efficient version tailored for technical domains 18.

Technical Approaches:

• Quantization, specifically using reduced precision such as 8 or 4-bit, is an efficient design choice for fine-tuning larger models 17.
• LoRA's Scalability has been demonstrated effectively on models containing up to 175 billion parameters 19.

Fine-Tuning for Continual Learning and Mitigating Catastrophic Forgetting

Catastrophic forgetting, where models lose previously acquired knowledge when fine-tuned on new tasks, remains a core challenge in continual learning . PEFT methods address this by updating only a small subset of parameters, thereby preserving the bulk of the pre-trained model's knowledge .

Parameter-Efficient Continual Fine-Tuning (PECFT): This emerging paradigm, highlighted in a 2025 survey, integrates Continual Learning (CL) with PEFT to enable continuous adaptation without the traditional trade-offs in forgetting and computational cost 20.

Gradient-Based Approaches (2023-2025): These methods adjust gradient updates to minimize interference with knowledge from previous tasks 18.

• Utility-based Perturbed Gradient Descent (UPGD) (Elsayed et al., 2023) preserves crucial weights while perturbing less critical ones 18.
• Adversarial Augmentation with Gradient Episodic Memory (Adv-GEM) (Wu et al., 2024) enhances data diversity using episodic memory to improve continual reinforcement learning 18.
• Asymmetric Gradient Distance (AGD) and Maximum Discrepancy Optimization (MaxDO) (Lyu et al., 2023) reduce training conflicts and forgetting in Parallel Continual Learning 18.
• Unified Gradient Projection with Flatter Sharpness for Continual Learning (UniGrad-FS) (Li et al., 2024) applies efficient gradient projection in minimal conflict regions 18.
• Continual Relation Extraction via Sequential Multi-task Learning (CREST) (Le et al., 2024) employs a customized multi-task learning framework to mitigate forgetting 18.
• Continual Flatness (C-Flat) (Bian et al., 2025) balances sensitivity to new tasks with stability by promoting a flatter loss landscape 18.
• VERSE (Banerjee et al., 2024) processes each training example once using virtual gradients to adapt and preserve generalization 18.
• Radian Weight Modification (RWM) (Zhang et al., 2024) is a CL approach for audio deepfake detection that guides gradient modification based on class distributions 18.
• TS-ACL (Fan et al., 2024) provides an analytical framework for time-series data 18.
• Multi-task Fine-tuning through simultaneous training on diverse tasks can also effectively prevent catastrophic forgetting 21.

Research to Enhance Data Efficiency in Fine-Tuning (Few-shot, Zero-shot)

Fine-tuning with limited data is prevalent in low-resource languages and specialized domains 17. PEFT methods are inherently sample-efficient, capable of matching or surpassing full fine-tuning performance with fewer than 100K samples 17.

• In-Context Learning (ICL), a key emergent ability of LLMs, allows models to acquire new task capabilities with a small set of in-context examples, bypassing the need for additional training or gradient updates 18.
• Prompt Tuning, an additive PEFT method, is highly cost-effective, especially for larger models, and can achieve strong performance with relatively few examples .
• Instruction Tuning enhances generalization by exposing decoders to diverse instruction styles and datasets, fostering robust zero- and few-shot instruction-following abilities. Identifying influential data (10-15K samples) is critical for efficient specialization 17.
• Active Learning (AL) reduces annotation costs by selecting the most informative examples for labeling, integrating well with PEFT methods to boost sample efficiency in specialized tasks 17.
• Intermediate Fine-Tuning for encoder models, involving fine-tuning on related, label-rich tasks before the target task, can significantly improve transfer 17.
• While model size generally impacts performance more than fine-tuning or prompt tuning data size, and scaling PEFT parameters offers marginal effectiveness, data efficiency for out-of-distribution or highly specific tasks remains crucial in the early training phase 17.

Other Significant Research Directions and Trends

• Application-Specific Fine-Tuning: PEFT is increasingly applied across diverse domains, including Video Text Generation, Biomedical Imaging, Protein Models, Code Review Generation, 3D Pretrained Models, and Speech Synthesis 16. For example, LoRA has shown superiority in Speech Emotion Recognition (SER) 16. A 2025 review specifically examines fine-tuning LLMs for specialized medical use cases 22.
• Sustainability of AI Research: PEFT significantly reduces the energy consumption and carbon footprint associated with training large models, promoting more sustainable AI research practices 18.
• Benchmarking and Evaluation: Challenges persist in establishing standardized PEFT benchmarks and conducting in-depth studies on hyperparameters and interpretability, partly due to discrepancies between reported and actual trainable parameter counts 19.
• Privacy-Preserving PEFT: Research is exploring PEFT methods capable of operating on sensitive data, such as biomedical images, by integrating techniques like federated learning or homomorphic encryption 16.
• Interpretability: Understanding the impact of PEFT on model interpretability, particularly for protein models, is an ongoing research area 16.
• Retrieval Augmented Generation (RAG): RAG is emerging as a significant alternative or complement to fine-tuning, grounding LLMs with external, up-to-date knowledge sources, which is especially valuable for facts that evolve over time 21.
• Small Language Models (SLMs): Fine-tuning for SLMs is a growing trend in 2025, valued for its practicality and easier implementation for smaller businesses or developers 21.
• Preference Alignment (PA): Beyond traditional fine-tuning, methods like Reinforcement Learning from Human Feedback (RLHF), Direct Preference Optimization (DPO), and other reference-free alignment techniques are crucial for aligning model outputs with nuanced human preferences. These methods are typically applied after supervised fine-tuning 17.
• Dynamic Environments: Continual Learning (CL) addresses the challenge of models learning sequentially from non-stationary data while retaining previously acquired knowledge, which is essential for dynamic real-world scenarios 18.