Sandeep Nagar | Homepage

About Me

Hi! I am a final year PhD student under the guidance of prof. Girish Varma at Machine Learning Lab, IIIT-Hyderabad, India. Previously, I have completed research internships at Computer Vision and Robotics Laboratory, University of Illinois, under the guidance of prof. Narendra Ahuja in collaboration with Dr David Beiser and Dr David Chestekat at College of Medicine at University of Illinois Chicago . I have also completed a research internship at School of Mathematics and Statistics, University of New South Wales Sydney, under the guidance of Prof Rohitash Chandra and Dr. Ehsan Farahbakhsh in collaberation with School of Geosciences, The University of Sydney, Australia.

I hold a BTech degree in Computer Science Engineering from Harcourt Butler Technical University, Kanpur, India. I am from Greater Noida, India. I am passionate about creating and building ML ideas. My main research interests include Computer Vision, Robotics, Theoretical Machine Learning, and Deep Learning.

I also read novels, find a list of interesting books here. Sometimes I ramble through the neural net of my brain about random thoughts, for preview click here.

Updates & News:

04-2025, Selected for the “COSPAR Outstanding Paper Award for Young Scientists, 2024” for our Geo-SAE paper.
02-2025, Awarded ACM India-IARCS travel grant
01-2025, Inverse-Flow accepted at AISTAT 2025
10-2024, Pre-print R2I-rPPG: A Robust Region of Interest Selection Method for Remote Photoplethysmography to Extract Heart Rate
09-2024, Paper accepted at ICPR'24, Competition on Safe Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather Conditions
06-2024, I will be attanding Climate Change AI Summer School 2024
04-2024, We are organizing a Comptetion on Safe Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather Condition co-located with ICPR'24
02-2024, Selected for Oxford Machine Learning Summer School 2024.
01-2024, Selected for Research Internship , Sony Research India, Bangalore
09-2023, Paper accepted at IEEE ICETCI'23, Adaptation of the Super Resolution SOTA for Art Restoration in Camera Capture Images.
08-2023, Won 1st Place in NVIDIA Competitions, ICETCI 2023 Reviving Cultural Heritage In Images using Image-restoration SOTA.
08-2023, Awarded one year iHub-Data PhD Fellowship
07-2023, I'll interning at Advance Research Group, Samsung R&D Institute-Bangalore (SRI-B).Project: Generative Models for Images Super-Resolution.
06-2023, Won the C4MTS: CHALLENGE for MISSING TRAFFIC SIGNs, NCVPRIPG 2023, IIT Jodhpur.
03-2023, Won Cash Prize: Hackathon, ML4Science 2023 , GitHub repo link
03-2023, Selected for Oxford Machine Learning Summer School 2023.
01-2023, Selected for Research Week with Google 2023
02-2023, I will present "Normalizing Flow models for 3D molecular generation" at ML4Science 2023
02-2023, FInC Flow accepted at VISAPP'23 with oral presentations.
02-2022, Awarded one year iHub-Data MS Fellowship.

Latest Research Articals:

Parallel Backpropagation for Inverse of a Convolution with Application to Normalizing Flows (AISTATS'25)

We give a fast parallel backpropagation algorithm with running time O(√n) for a square image and provide a GPU implementation of the same. Inverse Convolutions are usually used in Normalizing Flows in the sampling pass, making them slow. We propose to use Inverse Convolutions in the forward (image to latent vector) pass of the Normalizing flow. Since the sampling pass is the inverse of the forward pass, it will use convolutions only, resulting in efficient sampling times.

New

We use our parallel backpropagation algorithm for optimizing the inverse convolution layer resulting in fast training times also. We implement this approach in various Normalizing Flow backbones, resulting in our Inverse-Flow models. We benchmark Inverse-Flow on standard datasets and show significantly improved sampling times with similar bits per dimension compared to previous models.

paper and code

R2I-rPPG: A Robust Region of Interest Selection Method for Remote Photoplethysmography to Extract Heart Rate (Pre-print)

An rPPG ROI selection method based on 3D facial landmarks and patient head yaw angle. We then demonstrate the robustness of this ROI selection method when coupled to the Plane-Orthogonal-to-Skin (POS) rPPG method when applied to videos of patients presenting to an Emergency Department for respiratory complaints.

New

We identify ROI in 3D, using landmarks in a 3D representation of a face. Our ROI selection algorithm dynamically adapts to head rotation by resorting to a region from one of cheeks if forehead gets occluded. Our HR extraction algorithm works in real-time on a 2.60GHz CPU with 4GB RAM, and proposed R2I-rPPG HR extraction pipeline is effective under real-world clinical conditions.

paper and code

ICPR 2024 Competition on Safe Segmentation of Drive Scenes in Unstructured Traffic and Adverse Weather Conditions

A key aspect of the competition was the use and improvement of the Safe mean Intersection over Union (Safe mIoU) metric, designed to penalize unsafe incorrect predictions that could be overlooked by traditional mIoU. This innovative metric emphasized the importance of safety in developing autonomous driving systems. The competition showed significant advancements in the field, with participants demonstrating models that excelled in semantic segmentation and prioritized safety and robustness in unstructured and adverse conditions.

New

The results of the competition set new benchmarks in the domain, highlighting the critical role of safety in deploying autonomous vehicles in real-world scenarios. The contributions from this competition are expected to drive further innovation in autonomous driving technology, addressing the critical challenges of operating in diverse and unpredictable environments.

paper and code

Adaptation of the Super Resolution SOTA for Art Restoration in Camera Capture Images (IEEE Xplore, ICETCI'23 )

We adapt the current state-of-art for the image super resolution based on Diffusion Model (DM) and fine-tune it for Image art restoration. Our result show that the instead of fine-tunning multiple different model for different kind of degradation, fine- tunning one super-resolution.

New

We propose to use the super resolution SOTA (StableSR) to restore the art. Further, we fine tune the StableSR model for the art restoration and reconstruction.

paper and code

FInC Flow: Fast and Invertible k × k Convolutions for Normalizing Flows (accepted: VISAPP'23)

We propose a k × k convolutional layer and Deep Normalizing Flow architecture which i.) has a fast parallel inversion algorithm with running time O(nk**2) (n is height and width of the input image and k is kernel size), ii.) masks the minimal amount of learnable parameters in a layer. iii.) gives better forward pass and sampling times comparable to other k × k convolution-based models on real-world benchmarks.

New

As With a parallel inversion approach, we present a k × k invertible convolution for Normalizing flow models. We utilize it to develop a model with highly efficient sampling pass, normalizing flow architecture.

paper and code

Remote sensing framework for geological mapping via stacked autoencoders and clustering (Appected Journal: Advances in Space Research 2024)

Remote sensing data clustering using autoencoder networks for lithological mapping.

New

As Auto-Encoder are able to model the complex high-dimensional distributions of real-world data, they offer a promising approach to address this challenge. In this work, we propose an anomaly detection method using AEs and clustering algorithms. We also compare our results with the Principal component analysis (PCA), Independent Component Analysis (ICA), and Minimum Noise Fraction (MNF) dimensionality reduction techniques known as multivariate statistical methods that convert a set of observed and correlated input variables into uncorrelated or independent components.

paper and code

Estimating Heart Rate from Videos of Emergency Department Patients with Non-Caucasian Skin Tones (Illinois College of Emergency Physicians Conference-22)

remote photoplethysmography (rPPG)

New

The rapid expansion of telemedicine during the COVID-19 pandemic has highlighted some important limitations of virtual care, such as the lack of access to patient vital signs during the visit. Estimation of specific vital signs through remote video-based photoplethysmography (rPPG) has been demonstrated. We are investigating how to estimate heart rate remotely.

paper and code

CInC Flow: Characterizable Invertible 3×3 Convolution (accepted: 4th Workshop on Tractable Probabilistic Modeling, UAI 2021)

Invertible 3x3 Convolution

New

We study conditions such that 3x3 CNNs are invertible, allowing them to construct expressive normalizing flows. We derive necessary and sufficient conditions on a padded-CNN for it to be invertible.

paper and code

Automated Seed Quality Testing System using GAN & Active Learning (accepted: PReMI'21)

solving the problem of labelling the images and dataset imbalanced

New

We build a novel seeds image acquisition setup, which captures both the top and bottom views.Dataset collection for this problem has challenges of data annotation costs/timeand class imbalance. We address these challenges by i.) using a Conditional Generative Adversarial Network (CGAN) to generate real-looking images for the classes with lesser images and ii.) annotate a large dataset with minimal expert human intervention by using a Batch Active Learning based annotation tool. We benchmark different image classification models on the dataset obtained. We have introduced a new Dataset of the images of corn seed with a total of 26,802 images.

paper and code

Deep Web Crawling using Reinforcement Learning (Independent Research Project)

Solving the problem of deep web crawling (which links to follow) using RL to choose next url for crawling and text/image/urls extraction.

New

Build a novel scrapy-based crawler model which uses Reinforcement Learning (A2C, ϵ-greedy) methods to learn links to follow. Start a crawl where reward is given by a classifier that returns a score with 'extracted text' and the 'wiki-page text' match score calculated using cosine similarity after tokenizing.

code

Corn Seeds Images Dataset

To attract the focus of the vision and machine learning on the application and automation of agriculture, we have introduced a new image dataset.

New Dataset

Build a corn seeds dataset for classification and detection. We use the top and bottom images of seeds as individual images, which can get two independent predictions, increasing the accuracy. To solve the problem of dataset imbalance, we trained a Cond. GAN (BigGAN) to generated fake corn seed images and added them to the dataset. We trained an Active Learning model (BatchBALD) for automatic labelling images and labelled 9K new captured images.

webpage

Fine Grained Clasification

This project aims to show deep models like VGG16, Inception V3, and ResNet can be used on minimal size data with a total of 12607 images, including 10K training and 2607 validation images, without severe overfitting and with outstanding performance.

Find out more

Groot - Text-Extraction

Computer Vision course project-: Extracting the book and writer name from the cover image of book (English), using the classical image processing methods to improve the edges/features of text under the supervision of prof. Ravi Kiran Sarvadevabhatla.

Find out more

Low-cost Four wheels Payload Mobile Robot (AARG, IIIT-Hyderabad)

Designing of the Payload Mobile Robot with DC motor and improved battery life, making a robot work in more different environments like slop of 20 degrees, potholes, and damaged floor/road under the supervision of prof. Kamal Karlapalem.

Find Out More

Other Projects

Kinematic analysis of tendon-driven robotic mechanisms (Course project: Robotics Mechanism)

The kinematic structure of tendon-driven robotic mechanisms is investigated with the aid of oriented graphs, under the supervision of Dr Abhishek Sarkar at Robotics Research Center, IIIT Hyderabad.

Swedish Wheels Mobile Robot (Course project: Adv. Robotics)

Designing and path planning of a mobile robot with the four Swedish/Mecanum wheels, ROS, Arduino, RealSence for the path planning and vision feedback, Wheel Encoder. Project under the supervision of Dr Abhishek Sarkar at Robotics Research Center, IIIT Hyderabad.

Face-Image Classification (Course project: Statistical Methods in AI)

Performed PCA of all images combined of a dataset then classified using Naive Bayes Multi-class and Linear Multi-class Classifier (using python from scratch ). Project under the supervision of prof. C. V. Jawahar

Chroma key compositing (Course project: Digital Image Processing)

Implementation of chroma key technique, Compare global thresholding and locally adaptive thresholding (using python from scratch ). Project under the supervision of prof. Ravi Kiran Sarvadevabhatla

Panoptic Segmentation

Implementation of Panoptic Segmentation paper on the IDD dataset Comparison to Existing Metrics(Semantic segmentation metrics, Panoptic quality) using Pytorch from scratch. Project under the supervision of prof. Girish Varma

Work Experience

Research Intern - Advance Research Group, Samsung R&D Institute, Bangalore, India (Aug. 2023 - Jan 2024)

Research project:- Generative Models for Image Super-Resolution and Enhacement.

Manager:- Mr Raj Narayana Gadde and Dr. Sthya Reddy

Research Assistant - Dept. of Ophthalmology and Genomic Medicine, University of Leicester (Sep. 2022 - July, 2023)

Research projects: Machine learning based classification of foveal development

Supervisor:- Dr Mervyn G Thomas

Research Intern - Lab: Computer Vision and Robotics Laboratory,Dept: Electricaland Computer Engineering (ECE), University of Illinois at Urbana-Champaign (UIUC) (June 2021 - Oct. 2021)

Research Project:- Remote Sensing of Physiological Measurements Using Camera.

Supervisor:- Prof. Narendra Ahuja

Research Intern - Department: School of Mathematics and Statistics, University of New South Wales (UNSW) Sydney, Australia. (Nov. 2021 - Feb. 2022)

Research project:- Deep Variational Autoencoder for Geo-chemical Anomalies Detection.

Supervisor:- Prof. Rohitash Chandra in collaboration with Dr. Ehsan Farahbakhsh

Research Assistant - DevaLab, IIIT-Hyderabad (Nov. 2021 - May 2022)

Research projects:- Generative models for 3D molecules generation, and Generate molecular graphs using Energy-based Models.

Supervisor:- Prof. U. Deva Priyakumar and Prof. Girish Varma

Research Assistant - Machine Learning Lab, IIIT-Hyderabad (July 2019 - October 2021)

Research projects: Normalizing Flow, Machine Learning, Computer Vision, Generative Models

Supervisor:- Prof. Girish Varma

Research Assistant - Agents and Applied Robotics Group, IIIT-Hyderabad (Jan - April 2019)

Projects: Work on Obstacle avoidance for a group of Mobile Robots. Designing of Low-cost Four wheels robot for Indoor Payload(25kg) Transportation with efficient Battery utilization and High Lifetime. Work on Indoor Localization of a Group of Mobile Robots Using the Triangulation Algorithm, DecaWave-DW1000, Gyro and Ultrasonic Sensor, ROS, Arduino.

Supervisor:- Prof. Kamalakar Karlapalem

Computer Vision & Machine Learning Summer School- IIIT-Hyderabad, India (July 01, 2019 - July 15, 2019)

The curriculum consists of a series of lectures and lab sessions conducted by leading faculty from various disciplines, bringing their diverse experiences to the topic of AI. It also includes interactive talks with leading academicians in the field of AI and several experienced industry practitioners from top organizations across the world. The program is also designed to create connectivity and networking among participants through peer collaboration, simulation, and engagement. Special focus on recent advancements in fields of AI, Computer Vision and Machine Learning.Won cash prize [top-20/400].

Teaching Assitance - Great Learning, Hyderabad, India (June 2021 - present)

Post Graduate Certificate in Software Engineering for Data Science: To help professionals become data-proficient and build career critical competencies, Great Learning has collaborated with IIIT - Hyderabad to offer the Post Graduate Certificate in Software Systems for Data Science. The program enables participants to gain an in-depth understanding of data science and software systems that are widely used by companies. It takes a very practical approach to impart industry-relevant skills, and enables participants to become job-ready.

Teaching Assitance - IIIT-Hyderabad, India (Jan 2019 - May 2021)

- Computer Programming, Instructor: Dr Shantanav Chakraborty, Dr Abhishek Deshpande, and Dr Girish Varma, Monsoon-22
- Maths for CS: Probability and Statistics, Instructor: Dr Girish Varma, Monsoon-21
- Linear Algebra, Instructor: Dr Girish Varma, Spring-21
- Probability and Statistics, Instructor: Dr Pawan Kumar, Monsoon-20
- Probability and Statistics, Instructor: Dr Girish Varma, Monsoon-19
- Computer Network , Instructor: Prof Shatrunjay Rawat, Spring-19

A Paper A Day

150 Successful Machine Learning Models: 6 Lessons Learned at Booking.com Machine Learning Lab, IIIT-Hyderabad, India (May 29, 2021)

Machine Learning; Data Science; Business Impact; Product Development; Experimentation; E-commerce

DRAW: A Recurrent Neural Network For Image Generation -Machine Learning Lab, IIIT-Hyderabad, India (May 28, 2021)

DRAW networks combine a novel spatial attention mechanism that mimics the foveation of the human eye, with a sequential variational auto-encoding framework that allows for the iterative construction of complex images.

Zero-Shot Text-to-Image Generation - Machine Learning Lab, IIIT-Hyderabad, India (May 27, 2021)

A simple approach for this task based on a transformer that autoregressively models the text and image tokens as a single stream of data. With sufficient data and scale, our approach is competitive with previous domain-specific models when evaluated in a zero-shot fashion.

Normalizing Flows for Probabilistic Modeling and Inference -Machine Learning Lab, IIIT-Hyderabad, India (May 26, 2021)

Review Paper: normalizing flows, invertible neural networks, probabilistic modeling, probabilistic inference, generative models.

WASSERSTEIN AUTO-ENCODERS Machine Learning Lab, IIIT-Hyderabad, India (May 25, 2021)

A new algorithm for buildinga generative model of the data distribution. WAE minimizes a penalized form of the Wasserstein distance between the model distribution and the target distribution, which leads to a different regularizer than the one used by the Variational Auto-Encoder (VAE).