PhD defence of Amir Abbas Haji Abolhassani – A new probabilistic model for representation of the texture-flow in natural images

Abstract

Texture-flow are locally dense parallel patterns common in natural images. The texture-flow structure of an image is the trace of intrinsic properties of the objects in a scene, and understanding it is a crucial step in the human perception of an image. High-level perceptual tasks involving recognition, detection, classification and segmentation also rely upon the notion of texture-flow. In the past decades, researchers have proposed several computer vision methods for estimating the texture-flow profile of an image. The current state-of-the-art techniques, such as tensor-voting and relaxation-labeling come short of processing sparse input images and cannot effectively manipulate scale. This thesis aims to provide a method for calculating the global texture-flow profile of sparse images and deal with the scale variation of the edge patterns in a natural image. In this thesis, firstly, the new angular orientation probability distribution function (AOPDF) is proposed for representing the texture-flow profile of a natural image. At each location in the image, AOPDF depicts the likelihood of the texture-flow and curvilinear angular-orientations defined by a new two-parametric spatial angular orientation (AO) function. Subsequently, a new numerical method is proposed for estimating the AOPDF in digital images at discrete locations (pixels) and for a discrete set of angular-orientations. It is shown that AOPDF improves the results significantly when used for solving the challenging problem of single-image super-resolution. Furthermore, the AOPDF is combined with an anti-aliasing filter and reformulated into a kernel form, the so-called angular orientation of the edges (AngOri). The multi-scale AngOri kernels are intended to initialize the convolutional layers in deep neural networks (DNN), seamlessly. In a set of experiments, the neural networks are trained for image segmentation and object detection tasks. In all cases, the trained DNNs, initialized with AngOri kernels, achieve higher validation accuracies, especially when the training set is sparse. Finally, a new image reconstruction method is proposed for very sparse images, where the texture-flow could not be estimated from the image due to severe sparsity. The missing parts of the image are constructed from a set of patterns chosen, based on the similarity of local high-order statistics, from the training set. The image reconstruction results are significantly improved compared to the results obtained from existing methods. The new AOPDF, AngOri, and high-order stochastic methods introduced in this thesis are reliable alternatives for solving challenging computer vision tasks involving either sparse input images or training data.