Three-dimensional (3-D) video applications, such as 3-D cinema, 3DTV, and Free Viewpoint Video (FVV) are attracting more attention both from the industry and in the literature. High accuracy of depth video is a fundamental prerequisite for most 3-D applications. However, accurate depth requires computationally intensive global optimization. This high computational complexity is one of the bottlenecks to applying depth generation to 3-D applications, especially for mobile networks since mobile terminals usually have limited computing ability. This paper presents a semi-global depth estimation algorithm based on temporal consistency, where the depth propagation is used to generate initial depth values for the computationally intensive global optimization. The accuracy of initial depth is improved by detecting and eliminating the depth propagation outliers before the global optimization. Integrating the initial values without outliers into the global optimization reduces the computational complexity while maintaining the depth accuracy. Tests demonstrate that the algorithm reduces the total computational time by 54%-65% while the quality of the virtual views is essentially equivalent to the benchmark.