Adversarial Machine Learning (AML)

Adversarial Machine Learning is a machine learning approach that aims to fool a machine learning model by providing deceptive inputs. It therefore includes the generation and detection of adversarial examples, which are inputs created specifically to fool a classifier. Such attacks are known as adversarial machine learning and have been widely explored in areas such as image classification and spam detection.
Adversarial machine learning has been most extensively studied in the field of image recognition, where images are modified to cause a classifier to produce incorrect predictions.

The Threat of Adversarial Attacks in Machine Learning

As machine learning quickly becomes core to organizations’ value propositions, the need for organizations to protect machine learning is growing rapidly. As a result, adversarial machine learning is becoming an important area of the software industry.

How adversarial attacks on AI systems work

There are a variety of different adversarial attacks that can be used against machine learning systems. Many of these works target deep learning systems and traditional machine learning models such as support vector machines (SVMs) and linear regression. Most adversarial attacks are typically designed to degrade the performance of a classifier on a specific task, essentially trying to "trick" the machine learning algorithm. Adversarial machine learning is the field of study of attacks that are designed to degrade the performance of a classifier on a specific task. Adversarial attacks can be mainly divided into the following categories: poisoning attacks, evasion attacks, and model extraction attacks  

References

【1】https://viso.ai/deep-learning/adversarial-machine-learning/