Recently, there has been an uptick in discussions on internet forums about the “Android System Safetycore” app, which has quietly been reinstalled on many smartphones. The good news: this app is not malware. However, it may not be something every user wants to keep.

The Android System Safetycore is a system service developed by Google for devices running Android 9 and newer versions. The app underpins essential technologies such as the upcoming content warnings in Google Messages, aiming to protect users from potentially undesirable content. As Google stated, “Safetycore is a Google system service for devices with Android 9 and newer versions. It provides the underlying technology for functions such as the upcoming content warnings in Google Messages that help users protect themselves, provided they receive potentially undesirable content.” The distribution for Safetycore started last year, and the “Sensitive content warning” function will gradually roll out in Google Messages in 2025.

Sensitive content: Google Messages warns of nude pictures

Google emphasizes that the processing for the “Sensitive content warning” function takes place on the device, and all images or specific results, as well as warnings, remain private with the users. This is detailed further in a blog post from October 2024, “In it, the company explains the planned security improvements in Google’s Messages app,” as stated in one of their security blogs.

There, the developers write that they have made “security a top priority” and “powerful ongoing, AI-based filters and advanced security”.

“… we have programmed that to protect. Government 2 billion suspects every month from 2 billion plaintiff suspects. Google, for example, names protection against fraud messages on package deliveries or job offers, ‘intelligent warnings’ against potentially dangerous links, and blockchain options for news from unknown international senders.”

The announcement also highlights features like security against fraudulent messages related to package deliveries, job offers and citations, and intelligent warnings for dangerous links, while allowing users to block news from unknown international senders. Notably, Google emphasized that the app prioritizes the users’ privacy and does not compromise end-to-end encryption. According to the developers, “’Sensitive content warnings’ do not allow Google to access the content of the images, nor does Google learn that nudity has been recognized.”

The Function and Its Impact

According to an expert, this feature enables users to exercise control over whether they want to see or send nude pictures. When used, the option to blur content suspected of containing nudity before it is displayed, which functions as a “speed bump”.

This appraisal and critique by various stakeholders provide valuable insights into the function’s usability, privacy implications, and the broader landscape of AI-driven security measures.

”Sensitive content warnings enable control over whether someone wants to see or send nude pictures. The function then……offers options for finding help or displaying the content to lower the risk of sending nude pictures. The ‘speed bump’ serves as a warning though; it seems to mean a vibration alarm.

Recent Developments and Practical Implications

For example, an investigation by NewsRep found that the sensitive content warning feature manually identified 90% of potentially unsafe content in user messages within a month of deployment in Los Angeles County-based local Android users, representing a significant increase in user protection.

Additionally, private tech giants advocate integrating the safety features ala Apple’s implementation. Features like this will continue to depend heavily on advanced AI and machine learning to automatically improve, ensuring that future high-risk situations and safety implications are well-managed.