Curse of Rarity

The Curse of Rarity is a key scientific issue in the field of autonomous driving. It refers to the fact that in real driving environments, the probability of safety-critical events (such as traffic accidents) is extremely low, which makes these events extremely sparse in driving data, making it difficult for deep learning models to learn the characteristics of these events. As the sparsity of safety-critical events increases, the estimated variance of deep learning gradients increases exponentially, which requires more data and computing power to train the model, thus seriously hindering the model's learning ability in safety-critical tasks.

The concept of sparsity disaster was first proposed by Assistant Professor Feng Shuo of Tsinghua University and Henry Liu, Director and Chair Professor of Mcity at the University of Michigan. Their research results areCurse of rarity for autonomous vehicles" was published as a commentary article in Nature Communications, a subsidiary of Nature.

This study also proposed three technical routes to solve the sparsity disaster:

1. Dense Learning using data related to safety-critical events.
2. Improve the generalization and reasoning capabilities of your model.
3. Use technologies such as vehicle-road collaboration to reduce the probability of safety risk incidents.