Cheating Is a Disaster! The First Team in the Kaggle Competition Received the Most Severe Punishment
Recently, a cheating incident was exposed in the Kaggle competition. The cheating team obtained the first place by falsifying the answers of the test data set. After being reported and verified, the team has been disqualified from the competition, and the key person who cheated has also been permanently banned.
Wherever there are exams, there is cheating, and even anti-cheating mechanisms cannot prevent it.
Recently, on a Kaggle competition discussion page, a competition organizer "PetFinder.my" announced that Bestpetting The champion team,Disqualified from the "PetFinder.my Adoption Prediction" competition for cheating.
The team also included a Kaggle Grandmaster, Pavel Pleskov, who was also permanently banned because there was evidence that he was the key figure behind the cheating activity.
After being exposed, Pavel Pleskov has publicly apologized and said he would return the $10,000 prize money as soon as possible. But the price he paid for cheating is more than that.
2,000 teams participated, and the top five teams received prize money.
This competition is a public welfare competition organized by the Malaysian pet welfare platform "PetFinder.my" with the purpose of saving animal lives. The platform has a database of more than 150,000 animals and cooperates with animal lovers, media, businesses, etc. to improve animal welfare.
The algorithms developed by the contestants will be used to predict the adoption rate of pets. Studies have shown that the speed at which pets are adopted is closely related to metadata such as photos and descriptions in their online profiles. The contestants' task is to find this correlation and help pet adoption agencies optimize the electronic profiles of pets to make them look more attractive, thereby increasing the rate of adoption and reducing the number of "euthanasias".
Game time:The registration for the competition closes on March 21, 2019, and the deadline for submission of entries is March 28. The final results will be announced on April 9, 2019.
Competition size:There were 2,000 teams participating in the competition, but only the top five received prize money. The prize amounts for first to fifth place were $10,000, $7,000, $5,000, $2,000 and $1,000 respectively.
How did the “first place winner” cheat and successfully win the prize?
The Bestpetting team, which won first place, secretly crawled a private test data set during the competition and hid the data in the submitted content. It eventually won the first prize of the competition and received a prize of US$10,000.
According to PetFinder.my, the specific cheating behaviors are as follows:
- They obtained the adoption rate answer from private test data by cheating (probably scraped from the website of the organizer PetFinder.my);
- They then encoded this data and the answers into an ID field that was disguised as part of their external dataset “cute-cats-and-dogs-from-pixabaycom”;
- When processing the data, they decode these mixed-in ID fields, and the answers can be retrieved during the prediction phase;
- Finally, they only used some coded answers to make the final score look more “real”;
- The processed code is carefully hidden and obfuscated under many nested functions and layers of code, intentionally designed to be highly unreadable and appear ordinary.
In short, the team first obtained the answers to the test set in advance through cheating, then used part of it for encoding, decoded it during the competition, and finally used this standard answer as their "result" of the competition, thus achieving "excellent results."
A report that is late but not absent
According to organizers, the cheating was discovered by Benjamin Minixhofer, the sixth place winner of the "PetFinder.my" competition.
While helping the organizer PetFinder.my to productize the first-place solution, Benjamin Micklethoff discovered signs of cheating by the first-place winner and reported the violation to Kaggle.
After an in-depth investigation, Kaggle confirmed that cheating had occurred. In addition to announcing a ban, the team was removed from the leaderboard, so all other contestants were promoted one place. This is the first time since the creation of the Kaggle platform that a contestant has been completely disqualified for cheating.
Both the organizer and the whistleblower expressed sadness about this.
"We are truly saddened that our esteemed people, including a highly respected Kaggle Grandmaster, would go to such lengths to cheat for financial gain in a competition designed to save animal lives," the competition's organizers said.
The whistleblower said: "This incident undermines the fairness of the Kaggle competition, and I spent a lot of time trying to convert their solution into a production system, but the result was cheating. Maybe Kaggle officials don't want me to publish this article that discloses a lot of private test data, but I hope that the contestants can get some inspiration from it."
Pavel Pleskov works for H2O.ai also said it had fired Pavel Pleskov and that "from now on this person has no relationship with our company."
Apologize promptly, but can he be forgiven?
Afterwards, Pavel Pleskov apologized immediately and admitted his mistake sincerely. He also said that the team would return the bonus as soon as possible. He said: "For me, this has nothing to do with money, but with Kaggle points. The excitement of constantly being the first has made me lose my judgment."
Previously, he was interviewed by Kaggle for becoming a Grandmaster, and his speech video on YouTube titled "How to Become a Kaggle Grandmaster" has been viewed more than 20,000 times. Now it seems so ironic.
He used to be a man with a halo and received much attention, but now he has lost his reputation. This is the price he paid for cheating. What do you think about cheating in the game?
-- over--