Abstract: The size of pre-trained models has continuously increased to support growing demands for solving more complex problems. Especially, mixture-of-experts (MoE) model has become the most popular ...