The angular error-based triangulation method and the parallax path method are both high-performance methods for large-scale multi-view sequential reconstruction that can be parallelized on the GPU. We map parallax paths to the GPU and test its triangulation timing and accuracy performance for the first time. To this end, we compare it the angular method on the GPU for both performance and accuracy. Furthermore, we improve the recovery of path scales and perform more extensive analysis and testing compared with the original parallax paths method. Although parallax paths requires sequential and piecewise-planar camera positions, in such scenerios we can achieve a speedup of up to 14x over angular triangulation, while maintaining comparable accuracy.
Structure-from-Motion (SfM) applications attempt to reconstruct the three-dimensional (3D) geometry of an underly- ing scene from a collection of images, taken from various camera viewpoints. Traditional optimization techniques in SfM, which compute and refine camera poses and 3D structure, rely only on feature tracks, or sets of corresponding pixels, generated from color (RGB) images. With the abundance of reliable depth sensor information, these optimization procedures can be augmented to increase the accuracy of reconstruction. This paper presents a general cost function, which evaluates the quality of a reconstruc- tion based upon a previously established angular cost function and depth data estimates. The cost function takes into account two error measures: first, the angular error between each computed 3D scene point and its corresponding feature track location, and second, the difference between the sensor depth value and its computed estimate. A bundle adjustment parameter optimization is implemented using the proposed cost function and evaluated for accuracy and performance. As opposed to traditional bundle adjustment, in the event of feature tracking errors, a corrective routine is also present to detect and correct inaccurate feature tracks. The filtering algorithm involves clustering depth estimates of the same scene point and observing the difference between the depth point estimates and the triangulated 3D point. Results on both real and synthetic data are presented and show that reconstruction accuracy is improved.
A comprehensive uncertainty, baseline, and noise analysis in computing 3D points using a recent L1-based triangulation algorithm is presented. This method is shown to be not only faster and more accurate than its main competitor, linear triangulation, but also more stable under noise and baseline changes. A Monte Carlo analysis of covariance and a confidence ellipsoid analysis were performed over a large range of baselines and noise levels for different camera configurations, to compare performance between angular error-based and linear triangulation. Furthermore, the effect of baseline and noise was analyzed for true multi-view triangulation versus pairwise stereo fusion. Results on real and synthetic data show that L1 angular error-based triangulation has a positive effect on confidence ellipsoids, lowers covariance values and results in more-accurate pairwise and multi-view triangulation, for varying numbers of cameras and configurations.
Given the recent advances in both photogrammetry and structure-from-motion, a pipeline that captializes on the strengths of both fields is now possible. This paper presents a hybrid system that uses photogrammetric information to improve the accuracy of structure-from-motion, which in turn provides a more dense reconstruction. The procedure maintains the required metrological accuracy and permits measurements between points with no coressponding targets. The paper provides an analysis of the effects of various camera parameters to determine optimal scene configurations. Results generated by the hybrid system for real and synthetic data demonstrate that both more accurate and more dense reconstructions are obtained than with structure-from-motion alone.
We present the general idea of a computer vision structure-from-motion framework that makes use of sensor fusion to provide very accurate and efficient multi-view reconstruction results that can capture internal geometry. Given the increased ubiquity and cost-effectiveness of embedding sensors, such as positional sensors, into objects, it has become feasible to fuse such sensor data and camera-acquired data to vastly improve reconstruction quality and enable a number of novel applications for structure-from-motion. Application areas, which require very high accuracy, include medicine, robotics, security, and additive manufacturing (3D printing). Specific examples and initial results are discussed, followed by a discussion on proposed future work.
This paper presents a framework for GPU-accelerated N-view triangulation in multi-view reconstruction that improves processing time and final reprojection error with respect to methods in the literature. The framework uses an algorithm based on optimizing an angular error-based L1 cost function and it is shown how adaptive gradient descent can be applied for convergence. The triangulation algorithm is mapped onto the GPU and two approaches for parallelization are compared: one thread per track and one thread block per track. The better performing approach depends on the number of tracks and the lengths of the tracks in the dataset. Furthermore, the algorithm uses statistical sampling based on confidence levels to successfully reduce the quantity of feature track positions needed to triangulate an entire track. Sampling aids in load balancing for the GPU's SIMD architecture and for exploiting the GPU's memory hierarchy. When compared to a serial implementation, a typical performance increase of 3-4× can be achieved on a 4-core CPU. On a GPU, large track numbers are favorable and an increase of up to 40× can be achieved. Results on real and synthetic data prove that reprojection errors are similar to the best performing current triangulation methods but costing only a fraction of the computation time, allowing for efficient and accurate triangulation of large scenes.
This paper presents a novel framework for practical and accurate N -view triangulation of scene points. The algorithm is based on applying swarm optimization inside a robustly-computed bounding box, using an angular errorbased L1 cost function which is more robust to outliers and less susceptible to local minima than cost functions such as L2 on reprojection error. Extensive testing on synthetic data with ground-truth has determined an accurate position over 99.9% of the time, on thousands of camera configurations with varying degrees of feature tracking errors. Opposed to existing polynomial methods developed for a small number of cameras, the proposed algorithm is at best linear in the number of cameras and does not suffer from inaccuracies inherent in solving high-order polynomials or Grobner bases. In the specific case of three views, there is a two to three order of magnitude performance increase with respect to such methods. Results are provided to highlight performance for arbitrary camera configurations, numbers of cameras and under noise, which has not been previously achieved in the triangulation literature. Results on real data also prove that reprojection error is improved with respect to other methods.
Analyzing sources and causes of error in multi-view scene reconstruction is difficult. In the absence of any ground truth information, reprojection error is the only valid metric to assess error. Unfortunately, inspecting reprojection error values does not allow computer vision researchers to attribute a cause to the error. A visualization technique to analyze errors in sequential multi-view reconstruction is presented. By computing feature track summaries, researchers can easily observe the progression of feature tracks through a set of frames over time. These summaries easily isolate poor feature tracks and allow the observer to infer the cause of a delinquent track. This visualization technique allows computer vision researchers to analyze errors in ways previously unachieved. It allows for a visual performance analysis and comparison between feature trackers, a previously unachieved result in the computer vision literature. This framework also provides the foundation to a number of novel error detection and correction algorithms.
We present the general idea of using common tools from the field of scientific visualization to aid in the design, implementation and testing of computer vision algorithms, as a complementary and educational component to purely mathematics-based algorithms and results. The interaction between these two broad disciplines has been basically non-existent in the literature, and through initial work we have been able to show the benefits of merging visualization techniques into vision for analyzing patterns in computed parameters. Specific examples and initial results are discussed, such as scalar field-based renderings for scene reconstruction uncertainty and sensitivity analysis as well as feature tracking summaries, followed by a discussion on proposed future work.
This paper presents a novel framework for N-view triangulation of scene points, which improves processing time and final reprojection error with respect to standard methods, such as linear triangulation. The framework introduces an angular error-based cost function, which is robust to outliers, inexpensive to compute and designed such that simple adaptive gradient descent can be applied for convergence. Our method also presents a statistical sampling component based on confidence levels, that reduces the number of rays to be used for triangulation of a given feature track. It is shown how the statistical component yields a meaningful yet much reduced set of representative rays for triangulation, and how the application of the cost function on the reduced sample can efficiently yield faster and more accurate solutions. Results are demonstrated on real and synthetic data, where it is proven to significantly increase the speed of triangulation and optimize reprojection error in most cases. This makes it especially attractive for efficient triangulation of large scenes given the speed and low mem-ory requirements.
This paper presents an interactive visualization system, based upon previous work, that allows for the analysis of scene structure uncertainty and its sensitivity to parameters in different multi-view scene reconstruction stages. Given a set of input cameras and feature tracks, the volume rendering-based approach creates a scalar field from reprojection error measurements. The obtained statistical, visual, and isosurface information provides insight into the sensitivity of scene structure at the stages leading up to structure computation, such as frame decimation, feature tracking, and self-calibration. Furthermore, user interaction allows for such an analysis in ways that have traditionally been achieved mathematically, without any visual aid. Results are shown for different types of camera configurations for real and synthetic data as well as compared to prior work.
This paper presents a novel, interactive visualization tool that allows for the analysis of scene structure uncertainty and its sensitivity to parameters in different multi-view scene reconstruction stages. Given a set of input cameras and feature tracks, the volume rendering-based approach first creates a scalar field from angular error measurements. The obtained statistical, visual, and isosurface information provides insight into the sensitivity of scene structure at the stages leading up to structure computation, such as frame decimation, feature tracking, and self-calibration. Furthermore, user interaction allows for such an analysis in ways that have traditionally been achieved mathematically, without any visual aid. Results are shown for different types of camera configurations, where it is discussed, for example, how over-decimation can be detected using the propsed technique, and how feature tracking inaccuracies have a stronger impact on scene structure than the camera's intrinsic parameters.
Despite nearly universal support for the IEEE 754 floating-point standard on modern general-purpose processors, a wide variety of more specialized processors do not provide hardware floating-point units and rely instead on integer-only pipelines. Ray tracing on these platforms thus requires an integer rendering process. Toward this end, we clarify the details of an existing fixed-point ray/triangle intersection method, provide an annotated implementation of that method in C++, introduce two refinements that lead to greater flexibility and improved accuracy, and highlight the issues necessary to implement common material models in an integer-only context. Finally, we provide the source code for a template-based integer/floating-point ray tracer to serve as a testbed for additional experimentation with integer ray tracing methods.