About me

I am a researcher at Salesforce Research, I work on large multimodal models. Before joining Salesforce, I obtained my Ph.D. degree in Computer Science from the University of Maryland, College Park (UMD) in 2024, advised by Tom Goldstein. I mainly worked on research topics relevant to AI/ML safety and trustworthiness during my Ph.D. years.

During my years at UMD, I have interned at Nvidia, Salesforce, and Google as a research intern, where I have collaborated with many awesome professors and researchers. Before that, I obtained my bachelor’s degree in information security at the University of Science and Technology of China (USTC) in June 2019.

News

• [08/2024] Technical report out: xGen-MM (BLIP-3): A Family of Open Large Multimodal, along with xGen-MM-v1.5 model release. Been driving the instruction tuning efforts in model development. [arXiv] [X Live with @AK], [🤗 Model card]
• [05/2024] Released Salesforce’s multimodal large language models - xGen-MM, the enhanced continuation of our BLIP series. [Twitter], [🤗 Model card]
• [05/2024] Gave an oral presentation at ICRA’24 on a previous internship project about 3D object detection. [paper] [slides]
• [02/2024] Two preprints out (done at UMD). One studied data poisoning on vision-language models, another explored adversarial attacks on LLMs.
• [01/2024] Joined Salesforce Research. Relocated to Palo Alto, CA.
• [11/2023] Defended my Ph.D. dissertation 💐 👩🏻‍🎓
• [09/2023] One paper accepted at NeurIPS. We studied a novel vulnerability of aligned language models from the perspective of data security. [paper][code]
• [11/2022] In New Orleans attending NeurIPS. Presented the work done at Nvidia about prompt tuning for vision-language models. (Excited to attend my first in-person academic conference. I wish I had printed a bigger poster.)[paper]project page][code]

Selected Publications

For the complete list of publications, please refer to my google scholar page

• On the Exploitability of Instruction Tuning
  M. Shu, J. Wang, C. Zhu, J. Geiping, C. Xiao, T. Goldstein
  to appear at NeurIPS 2023
  [Preprint] [Code]

• On the Reliability of Watermarks for Large Language Models
  J. Kirchenbauer*, J. Geiping*, Y. Wen, M. Shu, K. Saifullah, K. Kong, K. Fernando, A. Saha, M. Goldblum, T. Goldstein
  Under review
  [Preprint] [Code]

• Test-Time Prompt Tuning for Zero-Shot Generalization in Vision-Language Models
  M. Shu, W. Nie, D.A. Huang, Z. Yu, T. Goldstein, A. Anandkumar, C. Xiao
  NeurIPS 2022
  [Paper] [Code] [Project page]

• Where do Models go Wrong? Parameter-Space Saliency Maps for Explainability
  R. Levin*, M. Shu*, E. Borgnia*, F. Huang, M. Goldblum, T. Goldstein
  NeurIPS 2022
  [Paper] [Code]

• The Close Relationship Between Contrastive Learning and Meta-Learning
  R. Ni*, M. Shu*, H. Souri, M. Goldblum, T. Goldstein
  ICLR 2022
  [Paper] [Code]

• Encoding Robustness to Image Style via Adversarial Feature Perturbation
  M. Shu, Z. Wu, M. Goldblum, T. Goldstein
  NeurIPS 2021
  [Paper] [Code]

• Adversarial Differentiable Data Augmentation for Autonomous Systems
  M. Shu, Y. Shen, M.C. Lin, T. Goldstein
  ICRA 2021
  [Paper] [Code]

• Model-Agnostic Hierarchical Attention for 3D Object Detection
  M. Shu, L. Xue, R. Mart'in-Mart'in, C. Xiong, T. Goldstein, J.C. Niebles, R. Xu.
  Under review
  [Preprint]

Services

Conference reviewer: NeurIPS, ICML, ICLR, CVPR, ICCV, IROS
Journal reviewer: IJCV

More about me (misc)

Things I like: Sun, seaside, going for a walk, yoga, weightlifting, civ 6, the legend of Zelda - BotW…