\text{Stereo Vision and Depth Estimation}
\textbf{Naresh Kumar Devulapally}
\text{CSE 4/573: Computer Vision and Image Processing}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Lectures 6, 7, 8: June 12, 17, 24 - 2025}
Slides adapted from Dr. Chen Wang CSE 573 lecture slides
\text{Stereo Vision}
- Recap of Perspective Projection
- Introduction to Stereo Vision
- Example Projects for Depth Estimation
\( \text{Agenda of this Lecture:}\)
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Recap - Perspective Projection}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Recap - Perspective Projection}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Recap - Perspective Projection}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Recap - Perspective Projection}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Why Stereo Vision?}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Stereo Vision}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Stereo Vision}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Stereo Vision - Why Multiple Views?}
Structure and depth are inherently ambiguous from single views.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Stereo Vision - Why Multiple Views?}
Structure and depth are inherently ambiguous from single views.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation cues}
- Shading
- Focus/Defocus
- Texture
- Perspective
- Motion
- Occlusion
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Shading}
Prados & Faugeras, 2006
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Focus/Defocus}
- Same point of view, different camera parameters
- 3D shape / depth estimation
H. Jin & P. Favaro, 2002
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Texture}
A.M. Loh
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Perspective Projection}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Motion}
L. Zhang
Keunhong Park, Nerifies, 2021
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Stereo photography \& Stereo Viewers}
http://www.johnsonshawmuseum.org
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Stereo Vision}
We will need to consider:
- Info on camera pose ("Calibration")
- Image point correspondences (feature detection/matching)
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Stereo Vision}
Two cameras, simultaneous views
Single camera moving in a static scene
Structure from Motion:
- Shape from "motion" between two images.
- Infer 3D shape of scene from two (multiple) images from different viewponts.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Stereo Vision}
Stereo Correspondence:
Given a point in one of the images, where could its corresponding points be in the other images?
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Stereo Vision}
Motion (Camera Calibration):
Given a set of corresponding points in two or more images, compute the camera parameters.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Stereo Vision}
Structure (Depth in the Scene):
Given projections of the same 3D point in two or more images, compute the 3D coordinates of that point.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Stereo Vision}
Human eyes fixate on point in space - rotate so that corresponding images form in centers of fovea.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - using Disparity}
Disparity occurs when eyes fixate on one object; that appears at different visual angles.
d = r - l = D - F
We will be using Disparity for depth estimation.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Camera Parameters (Recap)}
We will assume for now that these parameters are given and fixed.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Simple Stereo Vision}
Assuming parallel optical axes, known camera parameters (i.e., calibrated cameras).
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Simple Stereo Vision}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - using Disparity}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - using Disparity}
General
Case
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - using Disparity}
General
Case
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - using Disparity}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - using Disparity}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Epipolar Geometry}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Epipolar Geometry}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Epipolar Geometry}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Rectification}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Cross Product - Recap}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Essential and Fundamental Matrices
}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Essential and Fundamental Matrices
}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Essential and Fundamental Matrices
}
\( 2T + 3R\)
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Essential and Fundamental Matrices
}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Essential and Fundamental Matrices
}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Essential and Fundamental Matrices
}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - 8 point algorithm}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - 8 point algorithm}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - 8 point algorithm}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Correspondence search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Effect of Window size}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Effect of Window size}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Occlusion Challenge}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Ordering Challenge}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Results with Window Search}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Results with Window Search}
Use Live Code in the course website to play around with the implementation of depth estimation using window search.
Depth map \( \rightarrow \)
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Results with Window Search}
Use Live Code in the course website to play around with the implementation of depth estimation using window search.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Results with Window Search}
Use Live Code in the course website to play around with the implementation of depth estimation using window search.
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Better Solutions}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Better Solutions}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Epipolar Geometry - Energy Minimization}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Better Solutions}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Data-driven solutions}
\text{June 12, 2025}
\text{Naresh Kumar Devulapally}
\text{CSE 4/573: CVIP, Summer 2025}
\text{June 12, 17, 24 - 2025}
\text{Depth Estimation - Data-driven solutions - Interesting Projects}
- [ICCV 2025] StereoGen: Towards Open-World Generation of Stereo Images and Unsupervised Matching.
- Depth Anything V2: https://depth-anything-v2.github.io/
- Video Depth Anything: https://huggingface.co/spaces/depth-anything/Video-Depth-Anything
Lecture 6, 7, 8: Stereo Vision and Depth Estimation
By Naresh Kumar Devulapally
