Intro
I am Atharva Naik, a PhD student advised by Prof. Carolyn Rose at the Language Technologies Institute (LTI), Carnegie Mellon University (CMU).
I also collaborate closely with Prof. Daniel Fried and Prof. David Mortensen.
My research focuses on easy-to-hard generalization for LLM agents. I study how to improve agent capabilities through controllable-difficulty synthetic data generation, post-training with verifiable or meta-level supervision, and inference-time optimization. I apply these methods to coding agents for code quality (code review, linting, refactoring), data science agents (e.g., Kaggle-style challenges), and computational linguistics (e.g., forward reconstruction).
This summer I'm interning with the Copilot Tuning Team at Microsoft, where I am exploring how to improve the performance of coding agents for hard data science tasks using process reward models to provide feedback at inference time.
Recently, I developed ReaComp, a framework that compiles LLM reasoning traces into reusable symbolic solvers, enabling coding agents to bypass LLM calls at inference time to reduce cost while improving performance in a hybrid solver-first, LLM-fallback setting, particularly on hard problems.
I also developed MetaLint (COLM 2026), which reframes code linting as instruction following over natural-language best-practice specifications, enabling easy-to-hard generalization. Part of this work was conducted during my Summer 2025 internship at Oracle Labs. In addition, I created PBEBench (ACL Findings 2026), a dynamic, contamination-free benchmark that synthesizes controllable-difficulty data for evaluating knowledge-free inductive reasoning through multi-step string rewrites.
Previously, I developed CRScore (NAACL 2025, Oral), a reference-free metric for evaluating code review comments using LLM-generated code quality rubrics, static analysis, and embedding models. I also conducted an investigative study (ACL 2025, Oral, top 10%) that framed sound law induction as a programming-by-example problem and analyzed how different synthetic data distributions affect downstream performance on low-resource linguistic tasks.
Beyond my published research, I led the CMU team to the finals of the Amazon Nova AI Challenge (2025), where we trained LLMs to generate secure code by combining GRPO, deliberative alignment, and static analysis tools including AWS CodeGuru, Bandit, and CodeQL to produce safe, CWE-free code. I have also explored the use of LLMs to generate reflection questions for collaborative SQL programming. Our first study received Best Paper and Best Student Paper nominations at AIED 2024, and our follow-up study was published in the British Journal of Educational Technology (BJET).
(* - indicates equal contribution)
(* - indicates equal contribution)
