Stereo DIC¶

Stereo Digital Image Correlation (Stereo DIC) extends 2D DIC by using two synchronised cameras instead of one. The idea is to measure the same material point in the left and right image, then use the calibrated camera geometry to triangulate its 3D position. Repeating this for each deformed image gives the 3D displacement field.

In Pyvale, Stereo DIC still uses the same subset matching, shape functions, correlation criteria, interpolation, and Levenberg-Marquardt optimization covered in the DIC theory overview. The extra pieces are the stereo camera geometry, the left-right matching strategy, and the conversion from pixel coordinates to world coordinates.

Stereo Processing Strategy¶

The stereo workflow can be thought of as two linked DIC problems. The left camera is used for the usual temporal correlation between the reference and current deformed image. Once the left-image displacement is known, Pyvale uses that result to find the matching subset in the right camera image.

For a subset with reference left-image centre \((x_l^0,y_l^0)\), the left camera temporal DIC gives a deformed left-image centre

\[x_l^k = x_l^0 + u_l^k, \qquad y_l^k = y_l^0 + v_l^k,\]

where \(u_l^k\) and \(v_l^k\) are the 2D DIC displacement components for image \(k\). The stereo match then searches for the corresponding point in the right image,

\[x_r^k = x_l^0 + u_r^k, \qquad y_r^k = y_l^0 + v_r^k.\]

Here \(u_r^k\) and \(v_r^k\) are stored as the stereo pixel displacement components. They describe where the matching right-image subset is relative to the original left-image subset grid.

Camera Geometry¶

Stereo DIC needs a calibration for both cameras. Pyvale uses each camera’s intrinsic matrix,

\[\begin{split}\mathbf{K} = \begin{bmatrix} f_x & f_s & c_x \\ 0 & f_y & c_y \\ 0 & 0 & 1 \end{bmatrix},\end{split}\]

where \(f_x\) and \(f_y\) are the focal lengths in pixels, \(f_s\) is the skew term, and \((c_x,c_y)\) is the principal point. The relative camera pose is defined by a rotation matrix \(\mathbf{R}\) and a translation vector \(\mathbf{t}\) from the left camera to the right camera. In the current implementation the rotation matrix is built from the calibration Euler angles using the order

\[\mathbf{R} = \mathbf{R}_z\mathbf{R}_y\mathbf{R}_x.\]

From this calibration, Pyvale forms the fundamental matrix

\[\mathbf{F} = \mathbf{K}_1^{-\mathsf{T}}[\mathbf{t}]_\times \mathbf{R}\mathbf{K}_0^{-1},\]

where \([\mathbf{t}]_\times\) is the skew-symmetric matrix for the stereo translation. The fundamental matrix defines the epipolar constraint between the left and right images.

Epipolar Constraint¶

For a point in the left image written in homogeneous coordinates as \(\mathbf{x}_l = [x_l, y_l, 1]^\mathsf{T}\), the corresponding point in the right image must lie on the epipolar line

\[\mathbf{l}_r = \mathbf{F}\mathbf{x}_l.\]

If \(\mathbf{l}_r = [a,b,c]^\mathsf{T}\), then any matching right-image point \((x_r,y_r)\) satisfies

\[a x_r + b y_r + c = 0.\]

This is useful because it reduces the initial stereo search from a 2D image search to a search along a line. Pyvale computes the closest point on the epipolar line to the left-image subset centre,

\[\begin{split}\mathbf{P} = \begin{bmatrix} x_l \\ y_l \end{bmatrix} - \frac{a x_l + b y_l + c}{a^2+b^2} \begin{bmatrix} a \\ b \end{bmatrix},\end{split}\]

and the unit direction of the epipolar line,

\[\begin{split}\mathbf{d} = \frac{1}{\sqrt{a^2+b^2}} \begin{bmatrix} -b \\ a \end{bmatrix}.\end{split}\]

This line direction is also used to create a local rectified search coordinate system. The perpendicular direction is

\[\begin{split}\mathbf{d}_{\perp} = \begin{bmatrix} d_y \\ -d_x \end{bmatrix}.\end{split}\]

Initial Guess From Rectified FFT¶

Just as 2D DIC benefits from a good initial guess, stereo matching also needs a starting point that is close to the correct right-image subset. Pyvale does this by building a small rectified search window around the epipolar line.

The left subset is placed in the centre of an FFT correlation window. The right image is then sampled in a coordinate system aligned with the epipolar line,

\[\mathbf{s}(i,j) = \mathbf{P} + i\mathbf{d} - j\mathbf{d}_{\perp},\]

where \(i\) moves along the epipolar line and \(j\) moves perpendicular to it. This gives an unrectified patch from the right image, sampled as if the local search region had been rectified.

FFT cross-correlation is then used to estimate the translation between the left subset and this rectified right-image window. If the correlation peak is \((\Delta i, \Delta j)\), the estimated right-image point is

\[\mathbf{x}_r = \mathbf{P} + \Delta i\mathbf{d} - \Delta j\mathbf{d}_{\perp}.\]

This gives the initial rigid translation

\[p_0 = x_r - x_l, \qquad p_1 = y_r - y_l.\]

For affine or quadratic shape functions, the initial parameters also include the local coordinate transformation implied by the epipolar-line basis,

\[\begin{split}\begin{bmatrix} 1+p_2 & p_3 \\ p_4 & 1+p_5 \end{bmatrix} = \begin{bmatrix} d_x & -d_{\perp,x} \\ d_y & -d_{\perp,y} \end{bmatrix}.\end{split}\]

This is only an initial estimate. The final match is still obtained using the same nonlinear subset optimization described in the 2D DIC theory guide.

Reliability-Guided Stereo Matching¶

Pyvale uses a reliability-guided strategy for stereo matching in the same spirit as RG-DIC. Seed subsets are matched first, then the solution expands through the subset grid using previously matched neighbours as initial guesses.

The algorithm proceeds as follows:

The left-image temporal DIC result is used to locate the current subset centre in the deformed left image.
The current left subset is built using the temporal left-camera shape-function parameters.
For each seed point, an initial stereo guess is estimated from rectified FFT correlation along the epipolar line.
The optimizer refines the left-to-right subset match in the right image.
Neighbouring subsets are added to a queue ordered by their correlation cost.
When a subset is processed, Pyvale uses a successful neighbouring stereo result as the initial guess where possible. If the neighbour did not pass the threshold, the epipolar FFT estimate is used again.
The process expands through the active subset grid until all reachable subsets have been attempted.

For deformed images after the first one, the stereo shape-function parameters are composed with the left-camera temporal parameters. This means the stored stereo parameters describe the mapping from the original left reference subset through to the current right-image subset, rather than only the incremental left-to-right match for that frame.

Triangulation¶

Once a left-image point and right-image point have been matched, the 3D position can be calculated. Pyvale first undistorts both image points using the calibrated radial and tangential distortion parameters. The distorted pixel point is converted to normalized camera coordinates, and then an iterative update is used to remove distortion.

The normalized left and right image points are written as

\[\mathbf{x}_l = [x_l, y_l, 1]^\mathsf{T}, \qquad \mathbf{x}_r = [x_r, y_r, 1]^\mathsf{T}.\]

Pyvale then uses linear triangulation with projection matrices

\[\mathbf{P}_0 = [\mathbf{I}\mid\mathbf{0}], \qquad \mathbf{P}_1 = [\mathbf{R}\mid\mathbf{t}].\]

The world point \(\mathbf{X} = [X,Y,Z,1]^\mathsf{T}\) is found by solving the homogeneous DLT system

\[\mathbf{A}\mathbf{X} = \mathbf{0},\]

where

\[\begin{split}\mathbf{A} = \begin{bmatrix} x_l\mathbf{P}_{0,3} - \mathbf{P}_{0,1} \\ y_l\mathbf{P}_{0,3} - \mathbf{P}_{0,2} \\ x_r\mathbf{P}_{1,3} - \mathbf{P}_{1,1} \\ y_r\mathbf{P}_{1,3} - \mathbf{P}_{1,2} \end{bmatrix}.\end{split}\]

Here \(\mathbf{P}_{m,n}\) denotes row \(n\) of projection matrix \(\mathbf{P}_m\). The solution is the right singular vector corresponding to the smallest singular value of \(\mathbf{A}\). After converting from homogeneous coordinates, Pyvale stores the 3D coordinates \((X,Y,Z)\). If the stereo translation is supplied in millimetres, these coordinates are also in millimetres.

3D Displacement¶

For the first image pair, the triangulated coordinates are stored as the stereo reference coordinates and the world displacement is set to zero. For later image pairs, the 3D displacement is calculated by subtracting the reference stereo coordinates from the current triangulated coordinates,

\[u_X^k = X^k - X^0, \qquad u_Y^k = Y^k - Y^0, \qquad u_Z^k = Z^k - Z^0.\]

These are the displacement components written by Pyvale as the stereo millimetre displacement fields. Subsets that do not pass the temporal or stereo correlation threshold are not used for the world-coordinate calculation.

Practical Notes¶

Stereo DIC is more sensitive to calibration quality than 2D DIC. A poor intrinsic calibration, stereo rotation, or translation vector will give poor epipolar lines and therefore poor initial guesses. It will also directly affect the triangulated world coordinates. Spend time ensuring your calibration is of good quality - a lot of stereo DIC issues stem from a poor camera calibration.

Good speckle texture remains important. The left and right cameras see the same surface from different viewpoints, so subsets should have enough unique texture to match reliably even when perspective and lighting change slightly. Avoid regions with repetitive patterns that may cause ambiguous left-to-right matches.

The epipolar FFT estimate is used to get close to the correct match, but the final result still depends on the nonlinear subset optimizer. As with standard DIC, the threshold, subset size, subset step, shape function, and interpolation routine all influence the final result. Parameters may need to be adjusted differently for stereo DIC compared to 2D DIC due to the additional complexity of the stereo matching process.

When using calculate_3d(), ensure that the epi_distance parameter is set appropriately for your stereo baseline and resolution. This parameter controls the search distance along epipolar lines and affects both computation time and matching robustness. The default value of 300 pixels is suitable for typical stereo setups, but may need adjustment for very wide or very narrow baselines.

For incremental stereo DIC, Pyvale carries forward the previous stereo match as the updated reference for 3D reconstruction. This helps maintain accuracy across large deformation sequences while ensuring displacements remain relative to the original reference position.