Lucas Caccia
Senior Researcher @ Microsoft Research
Microsoft Research
Mile-Ex, Montréal
My current research centers on building agents that are modular and composable. I believe that modularity can enable decentralized, continual, and collaborative
model development. One successful instantiation of this is model MoErging, where different (independently trained) expert models are combined into an MoE-style architecture. To this end, I have done work both on the learning better experts and routing among available experts.
Prior to that, I completed in November my Ph.D. at McGill and the Quebec Artificial Intelligence Institute (Mila), where I was advised by Joelle Pineau. My PhD thesis focused on enabling efficient and robust Continual Learning in neural networks.
news
Jul 09, 2024 | Our paper on building and reusing a library of PEFT experts was presented at ICML |
---|---|
Jun 01, 2024 | I am now a Senior Researcher at Microsoft Research in Montréal! |
Nov 09, 2023 | I successfully defended my PhD 😁. Thanks to my wonderful committee (Siva Reddy, Adrian Popescu, Steve Liu and Doina Precup) for the great discussions during my defense, and of course, big thank you to Joelle for supporting me throughout my PhD. |
Sep 15, 2023 | Starting my post-doc at MSR Montréal! I will be continuing my work on efficient adaptation |
Sep 01, 2023 | Our paper on multi-head routing strategies for MoEs has been accepted at NeurIPS 2023 |
latest posts
Jul 01, 2020 | EWC derivation |
---|