Nouvelles et reportages
Chargement

Blog de recherche Microsoft
Dion: the distributed orthonormal update revolution is here
| Kwangjun Ahn et John Langford
Dion is a new AI model optimization method that boosts scalability and performance over existing leading methods by orthonormalizing only a top rank subset of singular vectors, enabling more efficient training of large models such as LLaMA-3 with reduced overhead.