Manan Dey

At Salesforce, I lead DX Workspaces — a platform that provides isolated, sandboxed cloud environments for hosting untrusted workloads. My team builds the infrastructure powering Agentforce Vibes IDE, Codey Studio, and Vibes as a Service. Previously at Salesforce, I led the AI Data Seeding and Simple Deploy in DX Inspector initiatives.

I love contributing to open-source AI research. I have contributed to the BigScience workshop, where I worked on BLOOM, and initiatives like BigCode, MTEB, and Google BIG-bench. I was a primary contributor to the Data Provenance Initiative, focusing on consent and provenance tracking in AI training data.

My research interests include large language models, multilingual NLP, bias evaluation, and data provenance.

Email  /  Google Scholar  /  Twitter  /  Github  /  LinkedIn

Research

I'm interested in machine learning, deep learning, and natural language processing. My work focuses on large language models, multilingual NLP, bias evaluation and mitigation, and prompt engineering.

Large-scale benchmark for evaluating video embedding models across diverse tasks and datasets.

Comprehensive multilingual text embedding benchmark evaluating models across 168 datasets spanning 113 languages.

Investigates critical gaps in data provenance tracking across modalities and proposes frameworks for responsible AI data curation.

Multilingual framework for systematically assessing social stereotypes in LLMs across diverse languages and cultural contexts.

Documents the rapid decline of the AI data commons, with over 25% of high-quality sources restricting AI access since 2023.

Benchmark for testing LLMs' ability to generate compositional structured outputs requiring multi-step reasoning.

Second-generation code LLM trained on The Stack v2 with advanced filtering and deduplication techniques.

Open-source 15.5B code generation model trained on 1 trillion tokens from 80+ programming languages with fill-in-the-middle capabilities.

Efficient 1.1B parameter code model demonstrating that aggressive data filtering enables smaller models to outperform larger ones.

Open-access 176B multilingual LLM trained on 1.6TB ROOTS corpus covering 46 natural and 13 programming languages.

1.6TB multilingual dataset spanning 59 languages with comprehensive documentation, opt-out processes, and PII redaction for BLOOM training.

Zero-shot debiasing method for neural machine translation using contextual sentences during inference without fine-tuning.

Examines bias evaluation methodologies in multilingual contexts and how frameworks fail to capture cultural nuances.

Demonstrates that multitask prompted training enables zero-shot generalization; T0 model shows task diversity can compensate for model size.

Platform for collaborative creation and sharing of natural language prompts, hosting 2,000+ prompts across 170+ datasets.

Historical survey tracing tokenization evolution from pre-neural to modern deep learning approaches including BPE, WordPiece, and SentencePiece.

Evaluation methodology for measuring gender bias in NLI by pairing gender-neutral premises with gender-specific hypotheses.

System for detecting depressive content in YouTube videos using transcript analysis and CES-D scores from viewer comments.

Open Source Projects

Tracks consent, licensing, and attribution for AI training datasets across text, speech, and video modalities.

Investigates how metadata incorporation during pretraining enhances LLM capabilities for zero-shot performance and structured generation.

Patents

ML-based system for generating structured synthetic data for testing and development.

Method for generating diverse simulated datasets for comprehensive software testing.

Template-driven framework for efficient test data generation and management.

Automated system for detecting and repairing corrupted or invalid test data.

Self-healing test automation system that automatically updates UI element identifiers.

Template from Jon Barron's website