One-sentence Summary

The authors propose Locally Linear Image Structural Embedding (LLISE) and its kernel variant, an adaptation of Locally Linear Embedding that substitutes mean squared error with SSIM to learn an image structure manifold for capturing structural features and discriminating distortions, thereby bridging manifold learning and image fidelity assessment.

Key Contributions

• The paper introduces the image structure manifold to capture structural features and discriminate image distortions, addressing the limitations of conventional mean squared error or $\ell_2$ℓ2​ norm-based metrics in image quality assessment.
• It proposes Locally Linear Image Structural Embedding (LLISE) and a kernel variant that adapt the Locally Linear Embedding framework by replacing standard squared error loss with the Structural Similarity Index (SSIM) to preserve image fidelity during dimensionality reduction.
• This methodology bridges manifold learning and image fidelity assessment, establishing a theoretical foundation for future investigations in quality evaluation.

Introduction

Image quality assessment relies on manifold learning techniques to model high-dimensional visual data and detect structural distortions, making it essential for applications ranging from computer vision to multimedia processing. Traditional approaches, however, depend heavily on Mean Squared Error or L2 norms, which poorly align with human perception and struggle to capture meaningful image fidelity. To address this gap, the authors leverage structural similarity (SSIM) to develop Locally Linear Image Structural Embedding (LLISE) and its kernel variant. By replacing conventional distance metrics with SSIM, they construct an image structure manifold that effectively discriminates visual distortions and establishes a new foundation for perceptually aware manifold learning.

Dataset

• Dataset Composition and Sources: The authors construct the dataset from the standard Lena image, applying six distinct degradation techniques to create a controlled set of samples for manifold learning and distortion recognition.

• Subset Details: The training set contains 121 images, consisting of the original Lena image plus 120 distorted variants. Each of the six distortion types (contrast stretch, Gaussian noise, luminance enhancement, Gaussian blurring, salt & pepper impulse noise, and JPEG distortion) includes 20 images generated at incremental MSE levels ranging from 45 to 900 in steps of 45. The out-of-sample test set comprises 12 images fixed at an MSE of 500, featuring both single and combined distortion combinations.

• Data Usage and Processing: The authors split all images into $8 \times 8$8×8 blocks and embed them into a $512 \times 512$512×512 dimensional space. They apply specific regularization parameters for linear reconstruction and embedding in both LLISE and kernel LLISE. The entire training set is used to learn and evaluate the embedded manifold, where a 1-Nearest Neighbor classifier analyzes each block independently. Image-level distortion labels are then derived through a majority vote across all processed blocks.

• Metadata and Evaluation Strategy: Each block and image is assigned categorical metadata indicating its degradation type, with zero denoting the original image and one through six representing the applied distortions. The test set undergoes the same embedding and classification pipeline to assess out-of-sample generalization, with performance benchmarked against standard LLE and kernel LLE baselines.

Method

The authors propose Locally Linear Image Structural Embedding (LLISE), a manifold learning framework designed to capture image structure and discriminate between different types of distortions by leveraging the Structural Similarity Index (SSIM) instead of traditional Euclidean distance metrics. The method is inspired by Locally Linear Embedding (LLE), but adapts its core principles to better model structural relationships in images. In LLISE, each image is partitioned into $b = \lceil d/q \rceil$b=⌈d/q⌉ non-overlapping blocks, where $d$d is the total dimensionality of the image and $q$q is the block size. Each block is treated as a vector in $\mathbb{R}^q$Rq, and the goal is to learn a $p$p-dimensional image structure manifold for each block, where $p \leq q$p≤q. The framework begins by centering each block by removing its mean to ensure zero-mean data, which simplifies the SSIM computation and aligns with the assumption of structural similarity under zero-mean conditions.

Refer to the framework diagram

The core of LLISE follows a two-step process similar to LLE: local reconstruction and global embedding. First, a $k$k-Nearest Neighbor ($k$k-NN) graph is constructed for each block using SSIM-based distance, defined as $||\tilde{\boldsymbol{x}}_1 - \tilde{\boldsymbol{x}}_2||_S = 1 - \text{SSIM}(\tilde{\boldsymbol{x}}_1, \tilde{\boldsymbol{x}}_2)$∣∣x~1​−x~2​∣∣S​=1−SSIM(x~1​,x~2​), where the SSIM between two block vectors is computed based on luminance, contrast, and structural components. For each block, the reconstruction weights are determined by minimizing the SSIM-based reconstruction error subject to a unit-weight constraint. This step captures the local linear structure of the image blocks in terms of structural similarity.

Following the reconstruction phase, the blocks are embedded into a lower-dimensional space $\mathbb{R}^p$Rp while preserving the reconstruction weights. The embedding is obtained by solving a quadratic optimization problem that minimizes the reconstruction error in the embedded space, subject to constraints ensuring zero mean and unit covariance of the embedded points. This global embedding step ensures that the manifold structure is preserved in the reduced-dimensional space, enabling effective representation of image structure.

The authors also extend LLISE to handle out-of-sample data through a reconstruction-based approach. For each out-of-sample block, the $k$k-NN among the training blocks is identified, and the reconstruction weights are computed using the same SSIM-based objective as in the training phase. The embedded representation of the out-of-sample block is then obtained by linearly reconstructing its embedding from the embeddings of its $k$k-nearest neighbors in the training set. This out-of-sample extension ensures that the learned manifold can be applied to unseen data without requiring retraining.

Additionally, the framework is generalized to the kernel space through Kernel Locally Linear Image Structural Embedding (Kernel LLISE). Here, image blocks are mapped to a higher-dimensional feature space using a kernel function $\phi(\cdot)$ϕ(⋅), which allows the data to potentially lie on a simpler manifold in the feature space. The kernel matrix is constructed from pairwise kernel evaluations between blocks, normalized and double-centered to ensure zero-mean in the feature space. The $k$k-NN and reconstruction steps are then performed in this kernel-induced space, with the reconstruction weights computed using the kernel evaluations. The out-of-sample embedding follows a similar approach, where the weights are derived in the feature space and the final embedding is obtained via linear reconstruction of the kernel embeddings.

The overall architecture of LLISE, both in the original and kernel spaces, leverages SSIM as a structural distance measure, which is better suited for image quality assessment than the standard $\ell_2$ℓ2​ norm, particularly in distinguishing structural from non-structural distortions. This structural focus enables the method to capture the intrinsic structure of images and improve performance in tasks requiring image fidelity discrimination.