StorytellingSerious

Research & Publications

I study how AI systems reason, perceive, and work in real settings. My research spans agentic AI, multimodal learning, and biomedical applications, with an emphasis on careful evaluation.

Publications

18 matching publications

2026

FLARE: Physics-Grounded Evaluation of Language-Model Agents for Feeder Restoration with Degraded Observations

BHVSP Subrahmanyam, Neeraj Choudhary

FLARE tests how language-model agents restore electrical feeders with incomplete observations. In simulation, shadow replay found that 96.4% of 15,706 guard-rejected actions would have violated a physical constraint.

Abstract & detailsHide detailsDetails for FLARE: Physics-Grounded Evaluation of Language-Model Agents for Feeder Restoration with Degraded Observations

NeurIPS · AI Foundations for Power Grids Workshop

Abstract

Restoring service after distribution-grid faults requires decisions from incomplete and conflicting information while respecting electrical constraints. Addressing this need, we introduce FLARE (Feeder-Level Agent Restoration Evaluation), a physics-grounded benchmark for post-isolation feeder restoration. FLARE contains 800 AC-feasible cases across five observation settings and allows agents to gather evidence, evaluate switches, and perform restoration actions under fixed operational limits. We evaluate five instruction-tuned models over 78,750 episodes, including topology, distributed-generation, and loading shifts. Mean normalized regret ranges from 0.082 to 0.148. Although every model has zero median regret, P95 regret ranges from 0.814 to 1.000, revealing an important failure tail. Shadow replay shows that 96.4% of 15,706 guard-rejected actions would have caused a physical violation. FLARE enables reproducible evaluation of language-model-based restoration support while emphasizing the need for external safety checks. Code is available at https://github.com/dr-pandit-69/FLARE-neurips.

Key results
  • 78,750 evaluation episodes across 800 AC-feasible cases
  • 96.4% of 15,706 guard-rejected actions would cause physical violations in shadow replay

Topic Agentic AI

Study notes
  • Evaluates five instruction-tuned models across 78,750 episodes and 800 AC-feasible cases.
  • In simulation, shadow replay found a physical violation for 15,142 of the 15,706 guard-rejected actions (96.4%).
  • All five models had zero median regret, yet their 95th-percentile regret ranged from 0.814 to 1.000.

My contribution Led the research and experimental development of this project.

Beyond Marginal Influence: Interaction-Aware Memory Selection for LLM Agents

BHVSP Subrahmanyam, Neeraj Choudhary

Can an agent reach the same decision with fewer memories? On 6,912 exact benchmark conditions, JointCore found a lowest-cost set that preserved the full-memory decision every time.

Abstract & detailsHide detailsDetails for Beyond Marginal Influence: Interaction-Aware Memory Selection for LLM Agents

NeurIPS · Personalized, Aligned, Long-Term Memory for AI Systems (PALM) Workshop

Abstract

LLM-agent memory selection often uses retrieval rankings based on record relevance or individual influence. Such rankings cannot identify sufficient subsets under redundant or jointly necessary evidence. We study per-decision memory sufficiency: selecting a small subset that preserves full-memory behavior. To separate search error from model error, we introduce JointCoreBench, a benchmark with nine interaction families, six workflow domains, natural-language decisions, and exact sufficient-set oracles, and JointCore, a relevance/cost-ordered group-deletion baseline that tests blocks before individual records. On 6,912 exact conditions, cost-ordered JointCore recovers a minimum-cost sufficient set in 100.0% of cases, reduces subset evaluations from 30.00 to 11.13 (62.9%), and achieves 87.8% active-memory savings. Across four 12B–70B model families and 432 tasks per model, its unweighted mean is 10.28 versus 16.00 model queries and 50.6% versus 43.1% minimum-cost recovery relative to cost-aware singleton deletion. Both methods preserve full-memory behavior under the deterministic primary interface. Tests with physical deletion, UNKNOWN masks, PENDING sentinels, counterfactual values, and alternative labels identify interface-dependent behavior. JointCore is an efficient benchmark baseline; the study does not establish deployment-ready memory compression. Code is available at https://github.com/dr-pandit-69/jointcore-neurips.

Key results
  • 62.9% fewer subset evaluations in the exact benchmark
  • 87.8% active-memory savings in the exact benchmark

Topic Agentic AI

Study notes
  • On 6,912 exact conditions, cost-ordered JointCore recovered a minimum-cost sufficient set in every case.
  • Subset evaluations fell from 30.00 to 11.13, a 62.9% reduction, with 87.8% active-memory savings in the exact benchmark.
  • Across four model families and 432 tasks per model, average minimum-cost recovery was 50.6% for JointCore and 43.1% for deletion one record at a time.

My contribution Led the research and experimental development of this project.

Lost in Visual Translation: A VLM-Assisted Perceptual-Semantic Coherence Framework for EEG-to-Image Reconstruction

Sukriti Tiwari, BHVSP Subrahmanyam, Nidhi Goyal, Sai Amrit Patnaik

An image reconstructed from brain signals can preserve meaning even when it looks different.

Abstract & detailsHide detailsDetails for Lost in Visual Translation: A VLM-Assisted Perceptual-Semantic Coherence Framework for EEG-to-Image Reconstruction

IJCAI–ECAI · 5th Workshop on Human Brain and Artificial Intelligence (HBAI)

Abstract

EEG-to-image evaluation should distinguish visual fidelity from recoverable meaning. Yet EEG-derived reconstructions are blurry, distorted, and low-detail, causing SSIM, LPIPS, and CLIP to penalize semantically recoverable outputs or reward plausible but incorrect ones. We analyze 6,855 ground-truth/reconstruction pairs from ATM, ENIGMA, BrainVis, and DreamDiffusion using semantic probes, caption harshness and blind-spot rates, and controlled degradations. Pixel metrics show near-zero correlation with semantic consistency, while representation metrics conflate perceptual and semantic errors. We therefore introduce a BCI-aware framework in which four VLMs assess image pairs through structured questions, producing Tolerant Perceptual Alignment Scores (T-PAS) and Tolerant Semantic Alignment Scores (T-SAS). Their consensus is distilled into the BCI-Coherence Score (BCS), a compact evaluator achieving T-PAS MAE 0.079 (r = 0.700) and T-SAS MAE 0.082 (r = 0.850) on our data. Human validation shows highly reliable joint coherence judgments (κ = 0.882 ± 0.174, α = 0.882), supporting perceptual–semantic recoverability over generic visual similarity. Code and resources are available at https://sukt03.github.io/BCS/.

Key results
  • 0.079 MAE perceptual alignment on the study data
  • 0.082 MAE semantic alignment on the study data

Topic EEG & multimodal AI

Study notes
  • Analyzes 6,855 image pairs from ATM, ENIGMA, BrainVis, and DreamDiffusion.
  • The distilled BCI-Coherence Score achieves MAE 0.079 for perceptual alignment and 0.082 for semantic alignment on the study data.
  • Human validation supports evaluating recoverable meaning alongside visual fidelity.

Controlled Evaluation of Cross-Lingual TTS Backends for Indic-to-English Dubbing

Jandhyala Dhanya, Laasya Reddy Dendi, Dandu Ashish Varma, BHVSP Subrahmanyam, Neeraj Choudhary

CLF5 better preserves the speaker’s voice, while XTTS-v2 dubs the same clips faster.

Abstract & detailsHide detailsDetails for Controlled Evaluation of Cross-Lingual TTS Backends for Indic-to-English Dubbing

29th International Conference on Computer and Information Technology (ICCIT 2026), Bangladesh

Abstract

Dubbing systems trade intelligibility, speaker similarity, timing, and compute against each other. We compare Cross-Lingual F5-TTS (CLF5) and XTTS-v2 inside one Indic-to-English pipeline, holding separation, recognition, diarization, translation, reference selection, duration fitting, and remixing fixed, and we measure rather than assume that the shared stages held. The evaluation uses 18 public Hindi, Telugu, and Tamil clips totalling 79.2 minutes on one NVIDIA T4. Two of five metrics show no backend effect: mixed-dub WERtts is 0.187 for CLF5 and 0.181 for XTTS-v2 with a zero-spanning interval, and Speech Coverage sits at ceiling (99.94%) for both under the no-drop policy. CLF5 attains higher similarity to its conditioning reference (0.867 versus 0.822, p = 0.0003; generic-voice floor 0.555) and higher ratings from 17 listeners for naturalness (3.98 versus 3.73) and perceived speaker similarity (3.94 versus 3.69). XTTS-v2 reduces end-to-end RTF from 1.55 to 1.25, a 19% reduction. Disabling silence borrowing and the no-drop policy reduces coverage to 95.6% and 81.9% on a six-clip subset, indicating that the shared fitting stage, not the backend, sets speech completeness. Backend choice therefore trades reference similarity against throughput while leaving intelligibility and completeness unchanged.

Key results
  • 0.867 vs. 0.822 similarity to the reference voice for CLF5 and XTTS-v2, respectively
  • 19% lower end-to-end real-time factor with XTTS-v2

Topic Speech synthesis

Study notes
  • The controlled comparison used 18 Hindi, Telugu, and Tamil clips totaling 79.2 minutes.
  • Both systems reached 99.94% speech coverage under the shared no-drop policy. The confidence interval for their word error rate difference included zero.
  • In ratings from 17 listeners, CLF5 scored higher for naturalness and perceived speaker similarity.

How Prompt Injection Changes LLM-Generated Peer Reviews: A Discourse Analysis

BHVSP Subrahmanyam, Saiankit Shankar, Appana Saketh Krishna Rao, Neeraj Choudhary

Generic prompt injection raised scores and reduced criticism in AI-generated peer reviews.

Abstract & detailsHide detailsDetails for How Prompt Injection Changes LLM-Generated Peer Reviews: A Discourse Analysis

29th International Conference on Computer and Information Technology (ICCIT 2026), Bangladesh

Abstract

Large language models are increasingly used to support peer review, yet the manuscripts they assess may contain instructions that redirect the review process. Such prompt injections can affect not only numerical recommendations but also the evidence, criticism, and reasoning expressed in generated reviews. We investigate this risk through a controlled study of 288 ICLR 2024 papers reviewed by three open-weight models under clean, explicit-generic (EG), targeted-specific (TS), and targeted-misdirect (TM) conditions, yielding 10,352 successful reviews. We characterize output changes with transparent discourse representations: D32 deterministic features, T18 sentence-graph features, and their F50 combination. EG raises the mean score by 2.11 points and sharply reduces critique density and weakness proportion from 10.23 to 2.83 and from 0.193 to 0.025, respectively. D32 and F50 achieve 0.991 AUROC for clean-versus-EG separation; a length-and-format baseline reaches 0.985, while residualized D32 retains 0.812 after word-count adjustment. TS and TM produce small score shifts and near-chance separation. These findings show how discourse analysis reveals changes in review reasoning and provides a transparent basis for auditing the integrity of LLM-assisted peer review.

Key results
  • 10,352 successful reviews generated for 288 ICLR 2024 papers
  • 0.812 AUROC for discourse-based detection of generic prompt injection after adjusting for word count

Topic AI safety

Study notes
  • Generic prompt injection raised mean review scores by 2.11 points and reduced the share of text devoted to weaknesses from 0.193 to 0.025.
  • Deterministic discourse features distinguished clean reviews from reviews affected by generic injection with 0.812 AUROC after word-count adjustment.
  • The two targeted conditions produced small score shifts, and the discourse features separated their reviews from clean reviews at near-chance levels.

MINDSET: Energy-based Memory Evolution for Long Conversational Agents

Sujato Dutta, Sreekruthy Tummala, Shashank Vanga, Ayushmi Pavani, BHVSP Subrahmanyam, Neeraj Choudhary

MINDSET keeps track of changing facts and the conversations that support them.

Abstract & detailsHide detailsDetails for MINDSET: Energy-based Memory Evolution for Long Conversational Agents

arXiv preprint · arXiv:2610.08586 [cs.AI]

Abstract summary

When someone gives an assistant a second address, they might have moved, or they might be describing where they work. MINDSET uses a local energy score to decide whether a new detail confirms an existing memory, replaces it, applies in a different context, or belongs in a separate memory. Each version points back to the conversation that supplied the evidence. The system can then retrieve what was true at the time a question asks about. In tests on LoCoMo, it ranked relevant evidence higher. The same settings also carried over to questions about changed facts on LongMemEval.

Read the full abstract on arXiv.

Key results
  • 0.4280 answer F1 on 700 LoCoMo questions with GPT-OSS-120B
  • 69.23% judge accuracy on 78 LongMemEval knowledge-update cases without retuning

Topic Agentic AI

Study notes
  • On 700 LoCoMo questions, MINDSET scored 0.4280 in answer F1 with GPT-OSS-120B.
  • Using the same controller settings on LongMemEval-S, the substitute judge Gemini-3.1-Flash-Lite accepted 54 of 78 knowledge-update answers (69.23%).

Generate, Verify and Select: Evolutionary Fitness Shaping for Safe Clinical Text Generation

BHVSP Subrahmanyam, Appana Saketh Krishna Rao, Sukriti Tiwari, Neeraj Choudhary

Checking candidate discharge summaries against clinical constraints reduced medication hallucinations.

Abstract & detailsHide detailsDetails for Generate, Verify and Select: Evolutionary Fitness Shaping for Safe Clinical Text Generation

IEEE Congress on Evolutionary Computation (CEC) · IEEE WCCI

Abstract

Automatic generation of clinical discharge summaries, particularly the Brief Hospital Course (BHC) and Discharge Instructions (DI), has gained renewed attention with the release of large-scale benchmarks such as Discharge Me. Although recent large language models achieve strong performance on overlap-based metrics, they frequently hallucinate clinically critical details, including medication names, dosages, numeric quantities, and negations. These errors are weakly penalized by standard evaluation metrics, yet they pose substantial risks in real-world clinical deployment. Motivated by population-based optimization and evolutionary selection principles, we propose Generate–Verify–Select (GVS), an inference-time framework that treats candidate generations as a population evaluated under explicit safety and feasibility constraints. GVS first produces a diverse candidate pool, applies deterministic hard safety gates targeting medication hallucinations, numeric inconsistencies, and negation flips, and filters contradictions using sentence-level natural language inference. Final selection is guided by evolutionary fitness shaping, where a linear fitness function is learned via Differential Evolution to balance multiple, competing quality and safety objectives during output selection. We instantiate GVS for both BHC and DI generation on the Discharge Me benchmark. Without retraining or modifying the underlying generators, GVS improves official automatic evaluation scores (0.323 for BHC and 0.378 for DI) while substantially reducing clinically meaningful hallucinations. GVS cuts medication hallucinations from 20.4–27.8% to 9.2–11.1%, while also halving numeric and negation errors, yielding substantial safety improvements independent of the base model. These results demonstrate that evolutionary selection in output space offers a principled and practical mechanism for enforcing safety and reliability in large language model–based natural language generation, particularly in high-stakes domains.

Key results
  • 20.4–27.8% → 9.2–11.1% medication hallucinations on the Discharge Me benchmark
  • Approximately halved numeric and negation errors, without retraining the generators

Topic Clinical NLP

Study notes
  • Evaluated on Brief Hospital Course and Discharge Instructions generation in the Discharge Me benchmark.
  • Medication hallucinations decrease from 20.4–27.8% to 9.2–11.1% across the reported generators.
  • Numeric and negation errors are approximately halved; the underlying generators are not retrained.

Continuous Coordinate-Space Diffusion for Probabilistic Cephalometric Landmark Localization

Kushal, Appana Saketh Krishna Rao, BHVSP Subrahmanyam, Neeraj Choudhary

A diffusion model refines landmark coordinates directly on craniofacial X-rays.

Abstract & detailsHide detailsDetails for Continuous Coordinate-Space Diffusion for Probabilistic Cephalometric Landmark Localization

IEEE International Symposium on Computer-Based Medical Systems (CBMS)

Abstract

The detection of cephalometric landmarks is pivotal to craniofacial analysis, requiring models that capture structured spatial dependencies arising from radiographic variability. In this paper, we propose a diffusion-based framework that formulates landmark prediction as probabilistic inference in continuous anatomical coordinate space, enabling iterative estimation instead of one-shot predictions. Reverse diffusion is performed directly on landmark coordinates conditioned on image features, avoiding discretized heatmap representations while remaining computationally efficient. Modeling coordinates as a distribution rather than point estimates promotes stable performance under dataset variability. Experiments on the ISBI 2015 and ISBI 2023 benchmarks demonstrate state-of-the-art performance, achieving 0.82 mm MRE on ISBI 2015 and 0.692 mm MRE on ISBI 2023, respectively.

Key results
  • 0.82 mm mean radial error on ISBI 2015
  • 0.692 mm mean radial error on ISBI 2023

Topic Biomedical AI

Study notes
  • Reports mean radial error of 0.82 mm on ISBI 2015 and 0.692 mm on ISBI 2023.
  • Reverse diffusion operates directly in continuous landmark-coordinate space.

Beyond Heatmaps: Self-Evolving Graph Neural Fields for Robust Cephalometric Landmark Detection

Appana Saketh Krishna Rao, Kushal, BHVSP Subrahmanyam, Acchugatla Sravanthi, Neeraj Choudhary

One model locates craniofacial landmarks across different annotation protocols.

Abstract & detailsHide detailsDetails for Beyond Heatmaps: Self-Evolving Graph Neural Fields for Robust Cephalometric Landmark Detection

IEEE International Symposium on Computer-Based Medical Systems (CBMS)

Abstract

Automated cephalometric landmark detection still constitutes a critical component of orthodontic diagnosis and treatment planning. Nevertheless, the current state of the art in heatmap regression and coordinate prediction is challenged by the presence of low-contrast images, highly variable anatomy, and the need to model complex spatial dependencies between landmarks. To overcome these difficulties, we propose Self-Evolving Graph Neural Fields (SE-GNF), a new framework that combines implicit neural models with graph-based spatial reasoning to support accurate, continuous, and sub-pixel landmark localization. SE-GNF consists of three main components: a neural-field module that maps sampled coordinates to smooth spatial representations, a Graph Attention Network encoder that encodes anatomical dependencies between craniofacial structures, and a cross-attention-based detector with learnable landmark-specific queries. Unlike discrete heatmap-based methods, SE-GNF provides differentiable coordinate predictions while maintaining anatomical relationships. The method generalizes to different cephalometric protocols without requiring any changes to the architecture. Experimentally, SE-GNF achieves mean radial errors of 0.98 mm (Test1) and 1.27 mm (Test2) on the ISBI 2015 challenge dataset (19 landmarks) and 0.70 mm on the ISBI 2023 challenge dataset (29 landmarks), thus providing a clinically sufficient level of precision for orthodontic landmark detection and treatment planning.

Key results
  • 0.98 / 1.27 mm mean radial error on ISBI 2015 Test1 / Test2
  • 0.70 mm mean radial error on ISBI 2023’s 29-landmark protocol

Topic Biomedical AI

Study notes
  • Reports mean radial errors of 0.98 mm and 1.27 mm on ISBI 2015 Test1 and Test2.
  • Reports 0.70 mm on ISBI 2023, covering a different 29-landmark protocol.

Subject-Independent ECG Reconstruction from SCG Using Dual-Domain Adversarial ODE Regularization

Ritika Buchupalli, V. V. Lakshmi Vyshali, BHVSP Subrahmanyam, Sreeja Pamu, Sathwika Alla, Kushal, Neeraj Choudhary

A model reconstructs ECG waveforms from mechanical signals recorded from the heart.

Abstract & detailsHide detailsDetails for Subject-Independent ECG Reconstruction from SCG Using Dual-Domain Adversarial ODE Regularization

IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB)

Abstract

Seismocardiography (SCG) offers a practical, electrode-free alternative to electrocardiography (ECG) for wearable cardiac monitoring, yet its susceptibility to posture, breathing, and sensor-placement variability limits direct clinical use. We propose a subject-independent SCG-to-ECG reconstruction framework combining a convolutional-transformer generator with dual time- and frequency-domain discriminators, trained under an ODE-based physiological regularization loss that enforces cardiac rhythm consistency and phase alignment beyond what purely data-driven objectives can achieve. Unlike prior GAN-based methods evaluated under subject-dependent settings, we validate our framework using a strict Leave-One-Subject-Out protocol across all three recording conditions of the CEBSDB dataset, demonstrating robust generalization to unseen individuals. The reconstructed ECG signals achieve a mean Pearson correlation of 0.984, a mean heart-rate error of 0.043 bpm, and a Bland–Altman bias of 0.035 bpm, accurately recovering key morphological features including P-waves, QRS complexes, and T-waves. These results confirm that physiologically consistent, clinically relevant ECG can be reliably reconstructed from compact wearable SCG sensors without subject-specific calibration.

Key results
  • 0.984 mean Pearson correlation under leave-one-subject-out evaluation on CEBSDB
  • 0.043 bpm mean heart-rate error under the same subject-held-out evaluation

Topic Biomedical AI

Study notes
  • Uses Leave-One-Subject-Out evaluation across all three recording conditions of CEBSDB.
  • Reports mean Pearson correlation of 0.984 and mean heart-rate error of 0.043 bpm.

Anatomical Slot Attention for Cephalometric Landmark Detection

Vijaya Sai Chigurupati, Vedha Srilaxmi Mudhana, Kushal, BHVSP Subrahmanyam, Neeraj Choudhary

Attention slots locate 19 anatomical landmarks on lateral skull X-rays.

Abstract & detailsHide detailsDetails for Anatomical Slot Attention for Cephalometric Landmark Detection

IEEE Conference on Computational Intelligence in Bioinformatics and Computational Biology (CIBCB)

Abstract

Cephalometric landmark detection is crucial for clinical diagnosis and treatment plans in orthodontics, while tedious manual annotation processes often lead to inconsistent annotations. In this paper, we propose an Anatomical Slot Attention based framework that automatically detects 19 landmarks on the lateral cephalogram images. Specifically, our framework considers the task as a feature binding problem where each landmark is represented as a slot that competes with other slots while attending to spatial features. The multi-scale anatomical representation extracted by the high-resolution HRNet backbone is fed into the slot attention module to enforce competition and reduce slot overlaps. Meanwhile, a lightweight relational bottleneck is applied to facilitate interactions between different slots to leverage the anatomical consistency of craniofacial structure. Finally, both coordinate estimation and uncertainty estimation tasks are performed via regression heads. Experiments are conducted using the publicly available IEEE ISBI 2015 benchmark dataset. We have obtained a mean radial error of 1.041 ± 0.706mm and 1.459 ± 0.922mm on Test Set 1 and Test Set 2, respectively, with a success detection rate of 94.95% and 84.32% at the 2.5mm threshold.

Key results
  • 1.041 ± 0.706 mm mean radial error on ISBI 2015 Test Set 1
  • 1.459 ± 0.922 mm mean radial error on ISBI 2015 Test Set 2

Topic Biomedical AI

Study notes
  • Reports mean radial error of 1.041 ± 0.706 mm and 1.459 ± 0.922 mm on ISBI 2015 Test Sets 1 and 2.
  • Success detection rates at 2.5 mm are 94.95% and 84.32%, respectively.

Multimodal Image Retrieval Through Image-Text Query Fusion Using CLIP

Kushagra Pathak, Alok Kumar Kamal, Appana Saketh Krishna Rao, BHVSP Subrahmanyam, Neeraj Choudhary

A reference photo and a written change guide a CLIP model to the target fashion image.

Abstract & detailsHide detailsDetails for Multimodal Image Retrieval Through Image-Text Query Fusion Using CLIP

International Conference on Emerging Techniques in Computational Intelligence (ICETCI)

Abstract

Multimodal image retrieval involves retrieving a target image with a reference image and a text modification. This task requires proper fusion of image and textual cues for successful performance. In this paper, we propose a two-staged CLIP-based pipeline where we first fine-tune the image/text encoders according to our retrieval goals. Then, we design a composition network to fuse the image and text representations and perform target image retrieval. Our proposed model is evaluated on the FashionIQ dataset via Recall@K evaluation metric. We obtain average Recall@10 = 30.12 and Recall@50 = 55.06 for our composed model. These values outperform baseline methods such as summation (Recall@10 = 21.17 / Recall@50 = 45.62) and weighted summation (Recall@10 = 27.12 / Recall@50 = 51.94). Moreover, we achieve improvements in mean recall by replacing the ResNet50 backbone with RN50x4 (31.64 → 33.36). These results show that a lightweight CLIP-based composine strategy can enhance retrieval performance while maintaining simplicity.

Key results
  • 30.12 average Recall@10 on FashionIQ
  • 55.06 average Recall@50 on FashionIQ

Topic Multimodal retrieval

Study notes
  • Reports average Recall@10 of 30.12 and Recall@50 of 55.06 on FashionIQ.
  • Outperforms simple and weighted-summation fusion baselines in the reported evaluation.

MAT-GAN: Adversarial Image Steganography with Secret-Conditioned Generation

Shalini Galingula, Appana Saketh Krishna Rao, BHVSP Subrahmanyam, Nam Pranav Rajiv Reddy, Neeraj Choudhary, Alok Kumar Kamal

The study tests how each part of MAT-GAN’s training objective affects image structure.

Abstract & detailsHide detailsDetails for MAT-GAN: Adversarial Image Steganography with Secret-Conditioned Generation

International Conference on Emerging Techniques in Computational Intelligence (ICETCI)

Abstract

Image steganography requires models that preserve secret-conditioned structure while producing outputs aligned with the statistics of plausible carrier imagery. We present MAT-GAN, a transformer-guided adversarial framework for secret-conditioned carrier-domain image generation. MAT-GAN combines a convolutional encoder-decoder, a tokenized self-attention bottleneck, and a configurable discriminator with an objective that balances adversarial alignment and reconstruction regularization. We evaluate the method on a RESISC45-derived benchmark that groups urban scenes as carrier-like targets and natural scenes as secret-like inputs, and compare it against DualGAN, CycleGAN, and WGAN under a common reduced training schedule. Across models, DualGAN achieves the strongest common carrier-domain FID of 344.29, while MAT-GAN reaches 473.33. However, ablation within the MAT-GAN family shows that the full model consistently outperforms all partial variants, improving FID to 463.76, reducing proxy MAE to 0.0870, and raising proxy PSNR to 19.27 dB. Detailed quantitative, qualitative, and training-curve analyses indicate that reconstruction regularization is the clearest isolated gain and that transformer guidance becomes most effective when paired with the full objective. These results position MAT-GAN as a compact steganography-oriented adversarial framework for studying secret-conditioned carrier-domain generation.

Key results
  • Full objective improves over partial MAT-GAN variants in ablation studies
  • Baseline comparison DualGAN retains the strongest common carrier-domain FID

Topic Generative AI

Study notes
  • Evaluated against DualGAN, CycleGAN, and WGAN on a RESISC45-derived benchmark.
  • DualGAN has the strongest common carrier-domain FID; MAT-GAN does not outperform every baseline.
  • Within MAT-GAN ablations, the complete model outperforms partial variants, with reconstruction regularization providing the clearest isolated gain.

Hard Negative-Aware Fine-Tuning for Multi-Modal Image Retrieval

Appana Saketh Krishna Rao, BHVSP Subrahmanyam, Jyotsna Pandranki, Neeraj Choudhary, Alok Kumar Kamal

BLIP and OpenCLIP use hard negative examples to learn to retrieve similar fashion images.

Abstract & detailsHide detailsDetails for Hard Negative-Aware Fine-Tuning for Multi-Modal Image Retrieval

International Conference on Emerging Techniques in Computational Intelligence (ICETCI)

Abstract

The notion of multi-modal image retrieval involves retrieving semantically relevant images with a textual query from a very large image gallery. However, most of the current methods based on single modality fail to address complex relationships that exist between visual and textual content. In this paper, we propose a framework to learn embeddings by contrasting image representations with textual query representations in order to perform effective multi-modal retrieval in the fashion domain. We investigate two latest state-of-the-art vision-language models, i.e., OpenCLIP and BLIP, on the FashionIQ dataset which is made up of fashion images with detailed human annotations in their captions. To improve the performance of our framework, hard negative samples are used in the contrastive loss function to increase the discriminative power of learning embeddings, and then progressive fine-tuning of image and text encoders is explored on different architectures. As our experimental results show, when tested on three fashion categories, viz. Dress, TopTee and Shirt, the recall@50 scores are 50.13% and 46.08% respectively for BLIP and OpenCLIP, outperforming other baselines such as TIRG, ComposeAE, and CIRPLANT.

Key results
  • 50.13% BLIP Recall@50 on FashionIQ
  • 46.08% OpenCLIP Recall@50 on FashionIQ

Topic Multimodal retrieval

Study notes
  • Reports Recall@50 of 50.13% for BLIP and 46.08% for OpenCLIP across the evaluated fashion categories.
  • Combines hard-negative contrastive learning with progressive encoder fine-tuning.

Multi-View Contrastive and Semantically-Aligned EEG Emotion Recognition with Topology and Asymmetry-Aware Adaptation

Sukriti Tiwari, BHVSP Subrahmanyam, Nidhi Goyal

A model uses electrode positions and differences between brain hemispheres to recognize emotion from EEG.

Abstract & detailsHide detailsDetails for Multi-View Contrastive and Semantically-Aligned EEG Emotion Recognition with Topology and Asymmetry-Aware Adaptation

International Conference on Emerging Techniques in Computational Intelligence (ICETCI)

Abstract

A reliable emotion recognition approach using EEG signals is crucial for affective human-computer interactions, but developing a reliable subject-independent decoder is very difficult. Despite previous research exploring either supervised deep learning or self-supervised contrastive approaches, no studies consider integrating neurophysiological structural knowledge into the problem via semantic supervision. This paper presents an integrated multi-objective framework incorporating focal classification, NT-Xent multi-view contrastive learning, and semantic alignment based on a language model. In addition to the two types of contrasts, we introduce the Topology and Asymmetry-Aware Multi-view Adapter (TAMA), which incorporates neurophysiological structural priors such as electrode topography, brain hemispheric asymmetry, and regional power spectral density into the backbone representations using topology-masked self-attention and gated residual fusion. Meanwhile, semantic supervision is conducted through the frozen BERT encoder during training, so our framework is deployable in an EEG-only mode. The experiments are performed on the DEAP dataset using an 8-fold leave-subjects-out cross-validation scheme. Our cnn_full_tama variant achieves a mean macro F1 of 0.475±0.033, making it the best-performing model among all variants; interestingly, it is also the only model that shows positive mean MCC at +0.013±0.082. Therefore, we explore the influence of different forms of inductive bias using the proposed framework. Contrastive learning appears to be critical for cross-subject generalisation, whereas TAMA boosts class-balanced discrimination.

Key results
  • 0.475 ± 0.033 mean macro F1 · cnn_full_tama · DEAP 8-fold leave-subjects-out
  • +0.013 ± 0.082 mean MCC for the same variant and evaluation

Topic EEG & multimodal AI

Study notes
  • Uses 8-fold leave-subjects-out cross-validation on DEAP.
  • The cnn_full_tama variant achieves mean macro F1 of 0.475 ± 0.033 and mean MCC of +0.013 ± 0.082.
  • Contrastive learning supports cross-subject generalization, while TAMA improves class-balanced discrimination in the reported variants.

Basilectal-Inspired Health Questions Expose Robustness Gaps in Small Medical QA Models

Saiankit Shankar, BHVSP Subrahmanyam, Neeraj Choudhary

Fluent answers can lose medical detail when a health question is phrased in nonstandard English.

Abstract & detailsHide detailsDetails for Basilectal-Inspired Health Questions Expose Robustness Gaps in Small Medical QA Models

IJCAI–ECAI · Workshop on Foundation Models for Social Good

Abstract

Small local language models are attractive for medical question answering in settings where connectivity, cost, and privacy limit cloud-based use. Yet users in these settings may express health questions in forms that move away from benchmark-standard English, including compressed, phonetic, and basilectal-inspired phrasing. We present a controlled robustness study using 102 TREC LiveQA Medical questions, paired L0–L3 synthetic basilectal-inspired variants, six small instruction-tuned models, and 7,416 free-form generations. Across the baseline setting, answer quality is stable under mild variation but degrades sharply in the L3 basilectal-inspired condition: BERTScore-style F1 falls from 0.707 at L0 to 0.614 at L3, ROUGE-L from 0.136 to 0.095, and Medical Concept Overlap recall from 0.129 to 0.046. The dominant failure mode is fluent incompleteness: models often continue producing plausible medical text while preserving fewer medically relevant concepts from trusted reference answers. We also examine prompt intervention, quantization, feature ablations, behavioral proxies, and claim-level support. The study is intentionally bounded: it is a controlled robustness benchmark, not a clinical safety certification, not a naturally collected dialect corpus, and not a claim about any specific speech community. Code is available at https://github.com/ankitsblade/Basilectal-Robustness-in-SLMs

Key results
  • 0.129 → 0.046 Medical Concept Overlap recall from standard to most perturbed phrasing
  • 7,416 responses across six models and 102 TREC LiveQA Medical questions

Topic Clinical NLP

Study notes
  • Evaluates 102 TREC LiveQA Medical questions, six models, and 7,416 generated responses.
  • Medical Concept Overlap recall drops from 0.129 to 0.046 between the standard and most perturbed conditions.
  • The study uses synthetic phrasing variants; it is neither a clinical safety certification nor a claim about a specific speech community.

2025

Temporal Attention-Based Multimodal Networks for Real-Time Proctoring of Online Examinations

BHVSP Subrahmanyam, Krishnakant Sharma, Shivam Saraswat, Satyam Pandey

The study combines gaze, head pose, and facial cues to analyze online exam recordings.

Abstract & detailsHide detailsDetails for Temporal Attention-Based Multimodal Networks for Real-Time Proctoring of Online Examinations

International Journal of Human Computer Interaction and Data Mining

Abstract

The growth of online education has driven the need for technologies that uphold academic integrity in remote online assessments. While traditional online proctoring methods may not be the most resource-efficient. This paper presents a multi-modal neural network based online proctoring that works on webcam recordings to detect suspicious device usage, This Neural Network has been trained on a dataset with 18 subjects, the videos have been annotated cheating & non-cheating labels manually, and various components have been extracted such as Head pose via 6DRepNet, Eye gaze using L2CS-Net, Facial landmarks using Media pipe library and cropped faces using the latest YOLO-v11, Subject Embeddings have been generated using ResNet50 to reduce the computational complexity along with using techniques like PCA. Our Multi Modal Neural Network achieves an overall accuracy of 85.74% with the late-fusion accuracy of our overall system components being 94%.

Abstract from ICETESS 2025 conference abstract · proceedings, page 46.

Key results
  • 85.74% neural-network accuracy · conference abstract’s 18-subject evaluation
  • 94% late-fusion accuracy · same conference abstract and evaluation

Topic Computer vision

Study notes
  • The conference abstract reports 85.74% neural-network accuracy and 94% late-fusion accuracy on its 18-subject dataset.
  • These figures describe the conference abstract; they do not establish performance beyond that evaluation.

2024

AI based Online Proctoring Remote Monitoring Intruder, Emotion Detection and Distance Estimation

BHVSP Subrahmanyam, Chandrasekhar Balabhadrapatruni

Face, gaze, and audio cues support identity checks and monitoring during remote exams.

Abstract & detailsHide detailsDetails for AI based Online Proctoring Remote Monitoring Intruder, Emotion Detection and Distance Estimation

International Journal of Scientific Research in Engineering and Management

Abstract

AI based online learning has undeniably surged in popularity over recent years. The COVID-19 pandemic has further accelerated the transition to online education, heightening the need for secure methods to authenticate and proctor online students. Today, a range of technologies offers varying levels of automation. In this paper, we present a comprehensive analysis of a specific solution that integrates multiple automated authentication technologies with an automatic proctoring system. The parameters that we used to achieve our goal are face detection, eye gaze tracking, multiple person detection, emotion detection, distance estimation and background noise detection. All these components help to maintain the exam integrity and to mitigate the existing limitation of e-exam proctoring software’s.

Key results
  • Automated authentication face-based identity checks for remote examinations
  • Multimodal monitoring face, gaze, multiple-person, emotion, distance, and background-noise analysis

Topic Computer vision

Awards

  • Best Poster Award

    IJCAI–ECAI · 5th Workshop on Human Brain and Artificial Intelligence (HBAI)

  • Best Presentation Award

    International Conference on Emerging Techniques in Computational Intelligence (ICETCI)

  • Silver & Elite Distinction

    NPTEL · IIT Ropar

  • National Bronze & Zonal Silver

    38th AIU Western Instrumental (Solo)

  • Two-time winner

    Spandan · Central University of Haryana

What are you in the mood for?