Rutwik S. Kulkarni
Robotics Graduate Student
About Me.
I am currently a Master's student in Robotics Engineering, in my second semester at Worcester Polytechnic Institute. My journey in this field began with obtaining a Bachelor's in Mechanical Engineering from the KK Wagh Institute of Engineering Education and Research in India. A significant milestone during my undergraduate studies was leading Team Vector as its captain for the ABU Robocon competition hosted by IIT Delhi. This leadership role notably enhanced my understanding and skills in robotics, marking a pivotal point in my academic and professional development.
After completing my undergraduate degree, I ventured into the professional sphere as a Robotics Software Engineer at Armstrong Dematic in India. This position allowed me to further refine my technical skills and gain industry experience.
My interests and career goals are firmly anchored in the fields of Robotics and Engineering, with a specific focus on Computer Vision, Robotic Perception, Simultaneous Localization and Mapping (SLAM), and Motion Planning. At present, I am deeply involved in projects that delve into these areas. My aim is to not only expand my knowledge and expertise in these domains but also to contribute meaningful innovations and advancements to the field of robotics.
Experience.
Graduate Researcher, (Dec 23 - Present)
Perception and Autonomous Robotics Lab (Worcester, MA)
Robotics Department.
Developing a Hardware in Loop (HIL) UAV simulation framework that merges the tangible physics of real quadrotors with the sensory perceptions derived from photorealistic virtual environments.
Robotics Software Engineer, (Jun 22 - Dec 22)
Armstrong Dematic (Nashik, MH, India)
Robotics Department.
Contributed to developing the Perception and Planning Stack for an Industrustrial Autonomous Mobile Robot (AMR) for Intralogistical Automation.
Graduate Apprentice Trainee, (Sept 22 - Dec 22)
Rucha Yantra LLP (Aurangabad, MH, India)
Mechanical Design Department.
Contributed to Material Handling System Development and Sensor Integration projects in automotive manufacturing.
Projects.
Sequence 01_7.mp4
VizFlyt: Perception-centric Framework For Autonomous Aerial Robots
Abstract: Hands-on aerial robotics courses are pivotal in training the next-generation workforce to meet the growing market demands. Such an efficient and compelling course depends on a reliable testbed. This paper presents VizFlyt, an open-source perception-centric Hardware-In-The-Loop (HITL) photorealistic testing framework for aerial robotics courses. We utilize pose from an external localization system to hallucinate real-time and photorealistic visual sensors using 3D Gaussian Splatting. This enables stress-free testing of autonomy algorithms on aerial robots without the risk of crashing into obstacles. We achieve over 100Hz of system update rate. Lastly, we build upon our past experiences of offering hands-on aerial robotics courses and propose a new open-source and open-hardware curriculum based on VizFlyt for the future. We test our framework on various course projects in real-world HITL experiments and present the results showing the efficacy of such a system and its large potential use cases.
Perception Stack for Autonomous Drone Racing
Developed a DNN-based perception stack, proficient in accurately detecting and segmenting windows under various angles and lighting conditions, leveraging a Blender-created dataset and a UNET for high-accuracy window detection, validated by Dice score metrics.
Implemented algorithms for nearest window selection and 3D pose estimation using Perspective-n-Point (PnP) methods, complemented by precise camera calibration using Zhang’s method with a checkerboard pattern.
Navigating Unknown Environments using Optical Flow
Engineered a resource-efficient perception and planning stack for DJI Tello’s monocular camera, optimized for the Jetson Nano, enabling autonomous navigation through windows of various shapes and textures.
Implemented SPyNet-based deep neural network for optical flow detection; processed flow maps for accurate gap detection and segmentation. Applied Image-Based Visual Servoing (IBVS) for dynamic control, successfully guiding through the largest gaps; rigorously tested and validated in both Blender simulations and real-world scenarios.
Building Built in Minutes: SfM and Novel View Synthesis using NeRF.
Successfully reconstructed a 3D scene and accurately determined the camera poses from a series of images by leveraging feature point correspondences. This was achieved through advanced techniques including Linear/Non-Linear triangulation, Perspective-n-Points solutions and refined through Bundle Adjustment for Optimal Accuracy and detail.
Implemented the foundational Neural Radiance Fields (NeRF) approach enabling the synthesis of novel viewpoints of intricate scenes. This was accomplished by optimizing a continuous volumetric scene representation, utilizing a sparse collection of input views to generate detailed and dynamic 3D visualizations.
Decoupled Kinodynamic Path Planning for a Quadrotor
Implemented a comprehensive two-phase approach for a quadrotor: Initiated with RRT* and Informed RRT* for path planning and kinematic feasibility, followed by dynamic trajectory optimization focusing on minimizing snap(i.e. 4th Order Derivative of Position).
Validated the system through detailed Blender simulations and practical testing on a DJI Tello EDU drone, achieving efficient and precise navigation in both simulated and real-world environments.
Classical and Deep Learning Approaches to Image Stitching using Homography
The objective of this project is to merge two or more photographs into a single panorama image by identifying the Homography that connects these images. This Project is executed in two phases:
Classical Approach: Using traditional for local feature matching to identify corresponding points accross the image, then computing homography to homography to stitch the image seamlessly.
Supervised and Un-Supervised Approach: Employing HomographyNet, a neural network designed to specifically for estimating Homography.
Deep UnDeep Visual Inertial Odometry
Abstract: — This work presents a deep learning-based approach to compute visual, inertial, and visual-inertial odometry for a Micro Aerial Vehicle (MAV) using a custom and synthetic dataset generated from simulation tools. We explore architectures that combine convolutional neural networks (CNN) and long short-term memory (LSTM) networks to learn spatial and temporal features from the visual and inertial data. Convolutional and LSTM networks are trained independently on the visual and inertial data, and their outputs are fused to estimate the MAV’s pose. The effectiveness of our approach is evaluated on the custom dataset, demonstrating the potential of deep learning techniques in enhancing visual-inertial odometry for MAVs under simulated conditions.
Einstein Vision: Traffic Scene Regeneration using Monocular Camera of a Car
Executed lane detection using Mask RCNN, CLRNet, and Bezier curve fitting; achieved zero-shot metric depth using the Marigold; deployed Detic for instance segmentation of traffic objects such as cars, stop signs, etc.
Automated traffic scene reconstruction with JSON intermediate representation and Blender scripting, resulting in a visualization like Tesla's dashboard, with pedestrian poses determined via a Component Aware Transformer.
Advanced Sampling-based Path Planners
Implemented BFS, DFS, A*, Dijkstra, RRT, RRT*, PRM (Uniform, Random, Gaussian, Bridge Sampling), D*, and Informed RRT* algorithms for motion planning.
Defined and constructed configuration and state spaces; applied algorithms to rigid objects and kinematic chains, achieving effective navigation and obstacle avoidance.
Attitude Estimation using 6-DoF IMU
Implemented quaternion-based attitude estimation for a 6-DoF IMU, employing Madgwick Filter and Unscented Kalman Filter (UKF).
Conducted benchmarking of attitude estimation against ground truth data obtained from the Vicon motion capture system.
Quadrotor Controller Design for UAV Interception in Restricted Airspace.
Implemented an advanced Linear Quadratic Regulator (LQR) control system for a quadrotor drone, designed for efficient airspace monitoring and precise interception of unauthorized UAVs in restricted zones
Resume.
rutwik-kulkarni-resume-general.pdf