A curated list of resources on Federated Large Language Models (FedLLMs) – an emerging paradigm that trains or adapts large language models (LLMs) across distributed data sources while preserving privacy.
- Surveys
- Federated Fine-Tuning
- Federated Prompt Learning
- Potential Directions
- LLMs for Federated Learning
- Applications
| Title | Year | Link |
|---|---|---|
| Federated Large Language Models: Current Progress and Future Directions | 2024 | arXiv:2409.15723 |
| Area | Topic |
|---|---|
| Prompt Generation | FedTPG · Code, TPFL |
| Few-Shot | FedFSL |
| Chain-of-Thoughts | Fed-SP-SC, FedLogic |
| Personalization | pFL (pFedPrompt), FedLogic, Fed-DPT |
| Multi-Domain | FedAPT, Profit |
| Parameter Efficient | FedPepTAO, FedLoRA |
| Communication Efficient | FedPrompt |
| Blackbox | FedBPT |
| Retrieval-Augmented | FeB4RAG |
| Applications | Multilingual (Breaking Borders 2023), Recommender (Guo 2024), Medical VQA (Zhu 2024), Weather Forecasting (Chen 2023), Virtual Reality (Zhou 2024) |
| Area | Topic |
|---|---|
| Real-World Deployment | Personalized FL on Confidential Data; Collaborative FL |
| Multi-Modality | Modality Co-optimization |
| Federated Pre-Training | Efficient Data Exchange; Model Architecture Design |
| Federated Inference | Real-time On-device Inference |
| LLMs for FL | Synthetic Data Generation; Capacity-Augmented FL; Responsible and Ethical LLM4FL |
| Area | Contribution |
|---|---|
| Synthetic Data Generation | Use LLMs to generate diverse training data to mitigate scarcity |
| Capacity-Augmented FL | Leverage LLM knowledge distillation and prompt engineering |
| Responsible & Ethical LLM4FL | Ensure compliance with privacy, fairness, and law |
| Domain | Example Works |
|---|---|
| Multilingual | |
| Recommender Systems | |
| Medical VQA | |
| Weather Forecasting | |
| Virtual Reality |