Non-negative matrix factorization(NMF) is a matrix decomposition method under the condition that all elements satisfy the non-negative constraint. It was first proposed by Lee and Seung in Nature magazine in 1999.

Mathematical Definition of NMF

For any given non-negative matrix V , the NMF algorithm can find a non-negative matrix W and a non-negative matrix H , so that satisfaction V = W x H , thereby decomposing a non-negative matrix into the product of two non-negative matrices.

Solution of NMF

There are many ways to find W and H, among which the doubling update method of Lee and Seung is the most common because of its simple implementation.

In addition, some algorithms are based on alternating non-negative least squares: in each step, first H is fixed and W is obtained by non-negative least squares solution, and then W is fixed and H is solved in the same way.

The methods for solving for W or H can be the same or different, as can W or H be normalized (to prevent overfitting).

Specific solution methods include: the projected gradient descent methods, the active set method and the block principal pivoting method.

NMF Advantages and Disadvantages

• advantage:

1. Processing large-scale data is faster and more convenient;
2. It achieves simplicity, interpretability of decomposition form and decomposition results, and occupies less storage space.

• shortcoming:

1. In NMF, only one layer is used to represent latent variables, which cannot handle complex learning problems;
2. NMF only constrains the non-negativity of W and H (this is the only prior that only requires this to be satisfied), but does not take into account the correlation between the internal elements of H for this prior.

Application areas of NMF:

• Image analysis
• Text clustering/data mining
• Speech Processing
• Robot control
• Biomedical Engineering
• Chemical Engineering
• Signal Processing
• Pattern Recognition
• Computer Vision