Lucas Caccia

Senior Researcher @ Microsoft Research

prof_pic.jpg

Microsoft Research

Mile-Ex, Montréal

My current research centers on building agents that are modular and composable. I believe that modularity can enable decentralized, continual, and collaborative model development. One successful instantiation of this is model MoErging, where different (independently trained) expert models are combined into an MoE-style architecture. To this end, I have done work both on the learning better experts and routing among available experts.

Prior to that, I completed in November my Ph.D. at McGill and the Quebec Artificial Intelligence Institute (Mila), where I was advised by Joelle Pineau. My PhD thesis focused on enabling efficient and robust Continual Learning in neural networks.

news

Jul 09, 2024 Our paper on building and reusing a library of PEFT experts was presented at ICML
Jun 01, 2024 I am now a Senior Researcher at Microsoft Research in Montréal!
Nov 09, 2023 I successfully defended my PhD 😁. Thanks to my wonderful committee (Siva Reddy, Adrian Popescu, Steve Liu and Doina Precup) for the great discussions during my defense, and of course, big thank you to Joelle for supporting me throughout my PhD.
Sep 15, 2023 Starting my post-doc at MSR Montréal! I will be continuing my work on efficient adaptation
Sep 01, 2023 Our paper on multi-head routing strategies for MoEs has been accepted at NeurIPS 2023

latest posts

Jul 01, 2020 EWC derivation