PokéStop Submission Auto-Review AI Disabled Due to Instability

Trainers, if you noticed that some of your PokéStop submissions were behaving oddly, either by being inexplicably rejected or randomly accepted, we may know the reason – it’s machine learning wonkiness!

Niantic has announced that they are temporarily disabling eMILy, the machine learning system that processes PokéStop submissions before it reaches human reviewers. Apparently, eMILy was misbehaving, and incorrectly making auto-accept and auto-reject decisions.

Niantic has shared that the auto-reject feature will remain operation, but eMILy’s auto-accept functionality is disabled until a fix is developed and deployed. Here’s the original post from the Wayfarer forums:

Hello Explorers,

We have identified an issue impacting eMiLy, our Machine Learning system, that was causing some incorrect auto-accept and auto-reject decisions. To prevent any further disruptions, we temporarily disabled both auto-accept and auto-reject features while we investigated the issue.

We have since re-enabled the auto-reject feature. However, the fix for the auto-accept issue will take some time to develop and deploy.

In the meantime, all nominations and edits that are not automatically rejected will continue to be reviewed by the community.

We appreciate your patience and understanding during this time. We want to express our sincere gratitude to the community and our Ambassadors for bringing this issue to our attention. Your valuable feedback played a crucial role in helping us identify and address this problem.

The Wayfarer Team

If you are not aware of eMILy and her existence, quite frankly, we can’t blame you for that. Machine Learning models within Niantic Wayfarer have been under development for years, and Niantic is not super transparent on their progress or updates.

We learned first about eMILy in April 2023, with another update posted a year later, on April 9, 2024:

We’re excited to announce a recent update to Wayfarer’s ML model! 

Our Machine Learning models, affectionately known as “Emily” by some of you, have been enhanced to better evaluate submissions, including wayspot submissions, images submissions and text edits. This means faster processing times and potentially more streamlined reviews for Wayfarers.

Here’s a quick rundown of what to expect:

• Auto-Accepts: The improved ML models should be better at identifying high-quality Wayspots and Edits that meet our acceptance criteria. This means more eligible submissions can potentially be auto-accepted, freeing up your valuable time to focus on the trickier reviews.
• Continued Reviewer Role: Wayfarer reviewers remain an essential part of the Wayspot review process. Even with the updated auto-accept system, your reviews are still crucial for ensuring Wayspot quality and adherence to our guidelines. The ML models are trained on and learn from the data provided by the Wayfarer community, including reviews, appeals, and takedowns.

Additionally we have increased the frequency at which you can perform wayspots appeals. You can now appeal rejected wayspots every 20 days. This is down from the previous window of 25 days from last month’s update, and a significant improvement from the original 30-day wait.

We’re confident these updates will make Wayfaring a smoother and more efficient experience for everyone. Stay tuned for further announcements as we continue to develop and improve Wayfarer!