Nikhil Reddy Billa

Nikhil Reddy Billa

Applied Scientist, Agentic AI · SAP Labs, Palo Alto

I'm an applied scientist on the agentic AI team at SAP Labs, where I own model research for the SAP Joule agent that turns natural-language requests into OData queries against S/4HANA, Ariba, and SuccessFactors. That covers synthetic training data, SFT and RL, distillation, evaluation, and serving.

Until May 2026 I was a master's student in Computer Engineering at Virginia Tech and a graduate research assistant in Ruoxi Jia's Responsible Data Science (ReDS) Lab. My research there was on LLM safety and privacy. One project showed that new jailbreaks mostly recombine older attack tactics, and used that to train defenses that hold up on attacks the model has never seen (ICLR 2026). Another pulled verbatim training data out of Llama models by driving them into a high-entropy state (EMNLP 2025).

Before grad school I spent two years at NCR Voyix in Hyderabad putting ML into production. I built a LayoutLM classifier for scanned documents and ran MLOps on Azure AKS, including the retraining pipeline. I also did computer vision research at IIIT Hyderabad and, as an undergrad, at NIT Rourkela, where I earned my B.Tech in Electrical Engineering.

Portrait of Nikhil Reddy Billa

News

Older news
  • Joined the ReDS Lab at Virginia Tech as a graduate research assistant.
  • Started my M.S. in Computer Engineering at Virginia Tech.
  • Our retinal OCT domain adaptation abstract with the University of Leicester appeared in IOVS (ARVO 2024).
  • First-author paper on resizing forensics for double-compressed JPEGs published in JISA, vol. 81.
  • IDD-AW appeared at WACV 2024.

Research Google Scholar ↗

I like finding where models break and fixing it at the data level. My LLM papers are about jailbreak robustness and training-data leakage, and at SAP I now work on post-training and evaluation for LLM agents. Earlier I worked on computer vision for driving in bad weather and on image forensics. The highlighted papers are the best place to start.

Schematic: thousands of raw jailbreak skills compressed into a small dictionary of primitives SDL 16,901 skills 397 primitives

Adversarial Déjà Vu: Jailbreak Dictionary Learning for Stronger Generalization to Unseen Attacks

Mahavir Dabas, Tran Huynh, Nikhil Reddy Billa, Jiachen T. Wang, Peng Gao, Charith Peris, Yao Ma, Rahul Gupta, Ming Jin, Prateek Mittal, Ruoxi Jia

ICLR 2026 International Conference on Learning Representations

Paper arXiv
BibTeX 
@inproceedings{dabas2026adversarial,
  title     = {Adversarial D{\'e}j{\`a} Vu: Jailbreak Dictionary Learning
               for Stronger Generalization to Unseen Attacks},
  author    = {Dabas, Mahavir and Huynh, Tran and Billa, Nikhil Reddy and
               Wang, Jiachen T. and Gao, Peng and Peris, Charith and
               Ma, Yao and Gupta, Rahul and Jin, Ming and Mittal, Prateek
               and Jia, Ruoxi},
  booktitle = {International Conference on Learning Representations (ICLR)},
  year      = {2026}
}

Most "new" jailbreaks recombine tactics from earlier ones. We extracted 16,901 adversarial skills from 32 jailbreak papers, and sparse dictionary learning reduces them to a dictionary of 397 primitives. Training on compositions of those primitives (ASCoT) gives an open-weight 8B model robustness comparable to Claude Sonnet-4-Thinking. It also beats o4-mini on most seen and unseen jailbreak families, with competitive over-refusal.

Schematic: token-level entropy rises and stays high, then the model emits memorized training text high entropy verbatim leak entropy tokens →

Retracing the Past: LLMs Emit Training Data When They Get Lost

Myeongseob Ko, Nikhil Reddy Billa, Adam Nguyen, Charles Fleming, Ming Jin, Ruoxi Jia

EMNLP 2025 Main conference, pp. 35316–35337

Paper arXiv
BibTeX 
@inproceedings{ko2025retracing,
  title     = {Retracing the Past: {LLM}s Emit Training Data When They Get Lost},
  author    = {Ko, Myeongseob and Billa, Nikhil Reddy and Nguyen, Adam and
               Fleming, Charles and Jin, Ming and Jia, Ruoxi},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in
               Natural Language Processing (EMNLP)},
  pages     = {35316--35337},
  year      = {2025}
}

An LLM that is about to recite memorized text usually shows a sustained spike in token-level entropy first. The Confusion-Inducing Attack (CIA) optimizes an input snippet to recreate that state and reaches a 22.2% verbatim extraction rate on Llama-2 70B. Aligned models resist it, so for them we first fine-tune on mismatched question-answer pairs, which weakens alignment enough for the attack to land.

Schematic: an iterative loop of synthetic data, fine-tuning, red teaming, and patching synthetic data fine-tune red-team patch </> secure code

Data is All You Need (Almost): Iterative Synthetic Instruction Tuning for Secure Code Generation

Team HokieTokie, Virginia Tech (faculty advisor: Ruoxi Jia)

Nova AI Challenge 2025 Amazon Nova AI Challenge proceedings, Trusted AI track

Paper

We took a pretrained CodeGen model with no safety training and taught it to write secure code mostly by fixing its data. Taxonomy-guided synthetic instructions came first, and after each tournament round we patched whatever the red teams had broken. My part was evaluation and fine-tuning: a containerized 13-metric evaluation pipeline where LLM judges run alongside CodeQL, Amazon CodeGuru, and Semgrep, and LoRA/QLoRA and GRPO runs that traded code utility against security.

A snowy mountain road from IDD-AW, shown as an RGB frame next to its near-infrared pair RGB NIR

IDD-AW: A Benchmark for Safe and Robust Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather

Furqan Ahmed Shaik, Abhishek Reddy, Nikhil Reddy Billa, Kunal Chaudhary, Sunny Manchanda, Girish Varma

WACV 2024 IEEE/CVF Winter Conference on Applications of Computer Vision, pp. 4614–4623

Paper arXiv Project page
BibTeX 
@inproceedings{shaik2024iddaw,
  title     = {IDD-AW: A Benchmark for Safe and Robust Segmentation of Drive
               Scenes in Unstructured Traffic and Adverse Weather},
  author    = {Shaik, Furqan Ahmed and Reddy, Abhishek and Billa, Nikhil Reddy
               and Chaudhary, Kunal and Manchanda, Sunny and Varma, Girish},
  booktitle = {Proceedings of the IEEE/CVF Winter Conference on Applications
               of Computer Vision (WACV)},
  pages     = {4614--4623},
  year      = {2024}
}

5,000 pairs of RGB and near-infrared images with pixel-level labels, shot in rain, fog, low light, and snow in unstructured traffic, with a four-level label hierarchy. We also propose Safe mIoU, a metric that penalizes dangerous mispredictions that standard mIoU counts like any other error.

A test image and its high-pass residual, before and after resizing between two JPEG compressions

CNN based image resizing forensics for double compressed JPEG images

Nikhil Reddy Billa, Bibhash Pran Das, Mrutyunjay Biswal, Manish Okade

JISA 2024 Journal of Information Security and Applications (Elsevier), vol. 81, 103693

Paper Code
BibTeX 
@article{billa2024cnn,
  title   = {CNN based image resizing forensics for double compressed
             JPEG images},
  author  = {Billa, Nikhil Reddy and Das, Bibhash Pran and
             Biswal, Mrutyunjay and Okade, Manish},
  journal = {Journal of Information Security and Applications},
  volume  = {81},
  pages   = {103693},
  year    = {2024},
  doi     = {10.1016/j.jisa.2023.103693}
}

Detects whether a JPEG was resized between its two compressions and estimates the resize factor. A preprocessing layer of 20 fixed high-pass filters suppresses scene content, so the CNN learns from resampling residuals. It outperformed four earlier methods on RAISE images on both tasks.

Abstracts

International Multi-Centre Validation of Unsupervised Domain Adaptation for Precise Discrimination between Normal and Abnormal Retinal Development

Zhanhan Tu, Nikhil Reddy, Helen Kuht, Gail D. E. Maconachie, Jinu Han, Line Kessel, Jing Jin, Ha-Jun Yoon, Callum Hunt, Prateek Pani, Ravi S. Purohit, Irene Gottlob, Girish Varma, Mervyn G. Thomas

IOVS 2024 Investigative Ophthalmology & Visual Science 65(7):2148, ARVO 2024 abstract

Experience Full CV (PDF) ↗

May 2026 – now

Applied Scientist, Agentic AI

SAP Labs · Palo Alto, CA

  • Own model research for the SAP Joule NL-to-OData agent, from synthetic data and SFT/RL to distillation, evaluation, and serving.
  • Terminator, the synthetic data engine I built, has LLMs generate schema-grounded agent trajectories from real OData $metadata. It targets the query patterns the student model fails on, with a multi-stage pipeline plus online RL.
  • Built the agent evaluation harness on about 1,100 tiered queries across S/4HANA, Ariba, and SuccessFactors. Execution-based row and count matching runs next to LLM judges, and every checkpoint gets sorted into failure buckets we can diagnose.
  • A pass@k analysis based on RLVR incentivization found an RL capability ceiling and moved the roadmap to SFT on reasoning traces.
  • Designed an execution-grounded RL reward that uses the live OData service as the terminal reward, and built a failure-to-correction corpus that trains self-correction into the model weights.
  • Distilled a Gemma 31B teacher into Gemma 4 E4B with LoRA, quantization, and offline DAPO on teacher rollouts. The student comes close to the teacher's accuracy and runs inference 7x faster.
Oct 2024 – May 2026

Graduate Research Assistant

ReDS Lab, Virginia Tech · Blacksburg, VA · advisor Ruoxi Jia

  • Diagnosed multi-agent failures on GAIA and SWE-bench by walking OpenTelemetry execution traces back through long agent trajectories to find and categorize the span that caused each failure.
  • Amazon Nova Trusted AI Challenge, Team HokieTokie: built the LoRA/QLoRA and GRPO fine-tuning pipeline that traded code utility against security, plus a containerized 13-metric evaluation pipeline where LLM judges run alongside CodeQL, Amazon CodeGuru, and Semgrep.
  • Wired LLM-driven jailbreak attackers (PAIR, AutoDAN-Turbo, Crescendo-MT) into a red-teaming CI/CD pipeline, so each checkpoint was attacked again between training runs.
  • Adversarial Déjà Vu (ICLR 2026, with Princeton and Amazon AGI) compressed 16,901 adversarial skills from 32 jailbreak papers into 397 primitives. Training on their compositions cut StrongREJECT attack success on unseen post-cutoff attacks to 0.04–0.11.
  • Retracing the Past (EMNLP 2025, with Cisco Research) introduced the Confusion-Inducing Attack, which probes training-data memorization by steering LLMs into spikes of token-level entropy. It reached 22.2% verbatim extraction on Llama-2 70B.
Jul 2022 – Aug 2024

Software Engineer, Machine Learning

NCR Voyix · Hyderabad, India

  • Built a LayoutLM classifier for 450 OCR-noisy scanned documents in 9 classes. Fine-tuning the layout and token embeddings together took accuracy to 96%, up from 62% for the rule-based extractor it replaced.
  • Wrote a Python log-analysis tool that clusters log spans and surfaces the first anomalous one. Localizing a regression in our globally distributed CI/CD went from about 50 minutes to about 4.
  • Ran MLOps on Azure AKS with Spark, Kafka, and Redis data pipelines, and brought pipeline latency down by about 40%.
  • Automated retraining and model versioning with MLflow and Airflow CI/CD. Retrain-to-deploy went from days to hours.
  • Built Python REST services with monitoring and alerting for the production ML stack, and handled incident response for those services.
Feb 2022 – Jul 2023

Research Assistant

Machine Learning Lab, IIIT Hyderabad · advisor Girish Varma

  • IDD-AW (WACV 2024): worked on the 5,000-pair NIR-RGB dataset and designed the Safe mIoU metric.
  • Graded arrested retinal development from OCT scans, using unsupervised domain adaptation so the model held up across clinical centers and OCT manufacturers. This was a collaboration with the University of Leicester.
Mar – Jun 2021

Computer Vision Intern

KoiReader Technologies · Bengaluru, India

  • Trained LayoutLM document classifiers on a custom dataset I built from raw business documents.
Oct 2020 – Jan 2022

Research Assistant

Intelligent Systems Lab, NIT Rourkela · advisor Manish Okade

  • Image resizing forensics for double-compressed JPEGs, which became my first-author JISA paper.

Projects GitHub ↗

Agentic RAG code assistant with MCP

Answers questions about local codebases and GitHub repos, commit history included. A LangGraph agent calls Filesystem and GitHub MCP tools and searches a FAISS store of chunk embeddings, with models served locally on vLLM. Moving from fixed-size to AST-aware chunking and re-selecting the embedding model raised recall@k and MRR on a held-out question set.

  • Python
  • LangGraph
  • FAISS
  • MCP
  • vLLM

RGB-NIR fusion for segmentation in bad weather

ECE 5554 course project, Virginia Tech, with Akash Reddy Mallepally

Stacked RGB and near-infrared frames into a 4-channel input for InternImage on IDD-AW. The fused model reached 66.7 mIoU, against 62.2 for RGB alone and 60.0 for NIR alone.

  • PyTorch
  • MMSegmentation
  • InternImage

FinQuest

A personal-finance web app for students. It pulls transactions through the Plaid sandbox, suggests saving strategies with Gemini, and runs what-if spending simulations. React and Tailwind in front, FastAPI on Google Cloud Run behind it.

  • React
  • FastAPI
  • LangChain
  • Gemini API
  • Cloud Run

Education

  • M.S., Computer Engineering, Virginia Tech Aug 2024 – May 2026 · GPA 3.67/4.0
  • B.Tech, Electrical Engineering, NIT Rourkela Jul 2018 – Jun 2022

Service & awards

Skills

Agentic systems
  • Tool calling
  • Multi-agent systems
  • LangGraph
  • MCP
  • RAG
  • Context engineering
  • Langfuse
  • vLLM
  • SGLang
Post-training
  • SFT
  • RLHF
  • GRPO
  • DAPO
  • GiGPO
  • LoRA / QLoRA
  • Distillation
  • Quantization
  • RL reward design
  • Synthetic data generation
Agent evaluation
  • Eval harnesses
  • LLM-as-a-judge
  • pass@k analysis
  • Benchmark design
  • Failure analysis
  • OpenTelemetry
  • Red teaming
Machine learning
  • PyTorch
  • Hugging Face
  • Transformers
  • NLP
  • Reinforcement learning
  • Deep learning
  • Experimental design
  • A/B testing
Languages
  • Python
  • C++
  • SQL
  • Bash
Infrastructure
  • Docker
  • Kubernetes
  • Azure (AKS)
  • AWS (EC2, S3, Bedrock)
  • Kafka
  • Spark
  • MLflow
  • Airflow
  • CI/CD
  • Git