About me

🔍 I am a Research Scientist at Meta in the Applied AI (AAI) org, working on the post-training of Meta’s frontier large language models, the Muse family. My contributions span:

  • 🧠 LLM research: post-training, data, and evaluation methods that improve reasoning, instruction-following, and overall model quality.
  • 🎙️ Speech, audio & music: my current focus, advancing Muse’s understanding and generation across the audio domain.
  • 💻 Coding: expert-level tasks, benchmarks, and training data that strengthen the models’ software engineering abilities.

Before joining AAI in June 2026, I was on Meta’s Media Foundation Video team (AI), building transformer- and diffusion-based models for video enhancement, compression, and quality optimization across Meta’s Family of Apps (Facebook, Instagram, WhatsApp, Messenger), and collaborating with the Meta Superintelligence Lab on large vision models for video generation and multimodal understanding.

🎓 I completed my Ph.D. at Rochester Institute of Technology (RIT) in 2025, conducting research at the Document and Pattern Recognition Lab (DPRL), under the mentorship of Dr. Richard Zanibbi. My dissertation, Parsing of Math Formulas and Chemical Diagrams using Graph-Based Representation and Attention Models, designed fast, efficient, and interpretable parsers for recognizing math formulas and chemical diagrams across PDFs, typeset images, and handwritten strokes, using graph attention in a multi-task learning framework. Earlier, I was an Applied Scientist Intern at Amazon (Alexa Speaker Understanding AI), improving speaker identification with semi-supervised learning.

🧑‍⚖️ I serve as a peer reviewer for Pattern Recognition and IJDAR, and on the ICDAR program committee (2023, 2025, 2026), where I was named an IAPR Outstanding Reviewer in 2025. I am also a Full Member of Sigma Xi. See my review service.

💻 Ph.D. Projects:

  • Multimodal Chemical Search: A multimodal search tool for retrieving chemical reactions, molecular structures, and associated text from scientific literature, linking visual and textual representations of chemical information.
  • ChemScraper: A molecule diagram parser for extracting molecular graphics from PDFs.
  • MathDeck: A math-aware search system supporting formulas & text.

🌐 Research interests: Large language models (post-training and evaluation), speech, audio and music AI, multimodal deep learning, code generation, computer vision, video understanding and generation, pattern recognition, and recognition of graphical structures.

📰 News

2026 4 updates
September 19, 2026

Gave a guest lecture, “Agentic AI in Industry: Context, Autonomy, and What Works”, for the Fusemachines AI Fellowship 2026 (online, Kathmandu, Nepal), covering context engineering, autonomy and what actually works when deploying AI agents. See all my talks.

June 1, 2026

Moved to Meta’s Applied AI (AAI) org to work on the post-training of Meta’s frontier large language models, the Muse family, contributing to LLM research, speech, audio and music, and coding capabilities.

May 2026

Recognized by the editors of Pattern Recognition (Elsevier) for 7 reviews (Nov 2024 – May 2026), and served on the ICDAR 2026 Program Committee. See my review service.

January 30, 2026

Elected a Full Member of Sigma Xi, The Scientific Research Honor Society.

2025 9 updates
September 16, 2025

Named an IAPR Outstanding Reviewer at the 19th International Conference on Document Analysis and Recognition, ICDAR 2025, Wuhan, China.

July 21, 2025

Started as Research Scientist at Meta, working on the Media Foundation Video team (AI).

June 27, 2025
May 23, 2025

Featured in the April-May 2025 MMLI Spotlights (Student Spotlight). News published on the DPRL’s recent highlights: Dr. Zanibbi received RIT’s highest research award and gave an invited talk at the Perimeter Institute for Theoretical Physics, which featured AI scientists including 2024 Nobel Laureate, Geoffrey Hinton.
Likewise, DPRL students achieved major milestones including top internships and industry placements—demonstrating the lab’s growing impact through MMLI collaborations.

April 15-16, 2025

Presented a poster on “ChemScraper: Pipeline for Parsing Raster and Vector Molecule Diagrams from PDFs” at the Molecule Maker Lab Institute (MMLI) Symposium 2025 at University of Illinois Urbana-Champaign (UIUC).

April 4, 2025

Our paper on “Multimodal Search in Chemical Documents and Reactions” has been accepted for publication at Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, in SIGIR ’25, Padua, Italy.
The system is available online at ReactionMiner. ReactionMiner searches reaction descriptions in PDF documents using text and chemical structures provided as SMILES/SMARTS strings. Extracted reaction records associated with passages can also be browsed and exported.

March 27, 2025

A step-by-step video tutorial on the AlphaSynthesis tool, ChemScraper, is available on YouTube. ChemScraper is a fast and accurate molecule diagram parser using characters and graphics extracted from PDF files and raster images.

February 18, 2025

Submitted our paper on the “Multimodal Search in Chemical Documents and Reactions” for publication at Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, in SIGIR ’25, Padua, Italy. The system is available online at ReactionMiner.

January 28, 2025

Gave a talk on “Reaction Extraction from PDF: ChemScraper and ReactionMiner Integration” at the inaugural Molecule Maker’s Playhouse (MMLI all-institute meeting), Champaign, IL. Slides

2024 4 updates
December 19, 2024

Successfully defended and passed my Ph.D. dissertation proposal on “Parsing of Math Formulas and Chemical Diagrams using Graph-Based Representation and Attention Models”.

September 3, 2024

Gave an oral presentation on “ChemScraper: Leveraging PDF Graphics Instructions for Molecular Diagram Parsing” at the 18th International Conference on Document Analysis and Recognition, ICDAR 2024, Athens, Greece.

June 2024

Presented “Integration of ReactionMiner and ChemScraper: Mining Chemical Reactions from Literature” at the MMLI All-Institute cross-thrust meeting, and co-presented the poster “ChemScraper: Extracting and Parsing Molecular Diagrams from PDFs and Images” at the MMLI NSF Site Visit poster session, UIUC. Also gave an online webinar, “Intro to AI: Exploring Fundamentals, Research, and Career Opportunities”, hosted by PANA Academy, Kathmandu, Nepal.

May 2024

A revised paper on ChemScraper has been published at the 18th International Conference on Document Analysis and Recognition, ICDAR 2024, Athens, Greece – Journal Track. The paper describes (1) a fast and accurate technique for parsing born-digital (vector) PDF images, and (2) its use to create training data for a new approach to visual parsing of molecule diagrams in raster images (i.e., pixel-based such as from PNGs). Code is available and the system is online at ChemScraper.

2023 6 updates
November 14, 2023

A paper describing ChemScraper parser for molecular diagrams in PDF drawing instructions (‘born-digital’) is available on arXiv here. The system can also generate annotated training data for visual parsers that recognize raster images (i.e., pixel-based, such as PNG). A link to associated code is provided in a footnote in the paper.

September 12, 2023

Co-presented a poster on “ChemScraper: Extracting Molecule Diagrams from PDF Vector and Raster Images with CDXML and SMILES Output” at the Molecule Maker Lab Institute (MMLI) All-Institute Retreat at University of Illinois Urbana-Champaign (UIUC).

August 22-23, 2023

Gave a poster presentation talk at Poster session 1 and doctoral consortium at the 17th International Conference on Document Analysis and Recognition, ICDAR 2023, San José, California.

June 28, 2023

Co-presented a poster on “ChemScraper: Extracting Molecule Diagrams from PDF Vector Images with Page-Level CDXML (ChemDraw) and SMILES Output” at the NSF Annual Review Meeting at University of Illinois Urbana-Champaign (UIUC).

April 17, 2023

Gave a Research Idea Ring (RIR) talk on “Line-of-sight with Graph Attention Parser (LGAP) for Math Formulas” at RIT.

April 2023

Our paper on the “Line-of-sight with Graph Attention Parser (LGAP) for Math Formulas” has been accepted for publication at the 17th International Conference on Document Analysis and Recognition, ICDAR 2023, San José, California.

2022 5 updates
September 27-28, 2022

Co-presented a poster on “Reconstructing the Structure of Molecular Diagrams in PDF Documents using a CNN-Attention-Based Parsing Model” at the Molecule Maker Lab Institute (MMLI) All-Institute Retreat at University of Illinois Urbana-Champaign (UIUC).

Sepember 5, 2022

Gave a guest lecture on “Bayesian Decision Theory” for RIT’s undergraduate course - Intro to Machine Learning (40 students).

August 28, 2022

Successfully completed Applied Scientist Intern at Amazon (Alexa AI). Started as Graduate Teaching Assistant (GTA) for the undergraduate course CSCI-335 Machine Learning.

May 23, 2022

Started as Applied Scientist Intern at Amazon (Alexa AI). Worked on the Alexa Perceptual Technologies - Speaker Understanding team to improve speaker identification in Alexa devices.

April 7, 2022

Gave a Research Idea Ring (RIR) talk on “A Fast and Interpretable Context-aware Parser for Isolated Formulas and Chemical Diagrams” at RIT.

2021 4 updates
September 9, 2021

Gave a poster presentation talk on the MathSeer extraction pipeline at the 16th International Conference on Document Analysis and Recognition, ICDAR 2021, Lausanne, Switzerland virtually.

September, 2021

MathSeer extraction pipeline released. This tool extracts formula locations and content in PDF documents. The pipeline is available from GitLab, and includes improved versions of SymbolScraper, ScanSSD (now, ScanSSD-XYc), and QD-GGA. The pipeline was prepared by Ayush K. Shah, Abhisek Dey, Matt Langsenkamp, and Prof. Zanibbi.

May 13, 2021

Successfully defended and passed my Ph.D. Research Potential Assessment (RPA) on “Mathematical Formula Recognition”. Slides

April 2021

Our paper on the “MathSeer formula extraction and evaluation pipeline” has been accepted for publication at the 16th International Conference on Document Analysis and Recognition, ICDAR 2021, Lausanne, Switzerland.

2020 2 updates
August 2020

Joined Rochester Institute of Technology (RIT) for Ph.D. in Computing and Information Sciences. Started as Graduate Research Assistant (GRA) at the Document and Pattern Recognition Lab (DPRL) under Prof. Richard Zanibbi.

January 2020

Promoted to Machine Learning Engineer Level 1 at Fusemachines Nepal.

2019 3 updates
August 2019

Promoted to Machine Learning Engineer Associate at Fusemachines Nepal.

August 2019

Graduated from Kathmandu University as a Computer Engineer.

June 2019

Started working as a Machine Learning Engineer Trainee at Fusemachines Nepal.