The Discovery Problem, Not Just the Approval Problem
A recurring pattern has emerged across both major app stores: platforms ban harmful app categories in policy, yet their own product discovery infrastructure actively surfaces them to users. The latest example involves so-called "nudify" apps — tools that use generative AI to create fake nude images from clothed photos.
A recent investigation documented 38 such apps across the App Store and Google Play, collectively responsible for 483 million downloads and $122 million in revenue. What distinguishes this finding from typical policy-violation reports is where the apps were discovered: not through obscure third-party links, but via the stores' native wiki:app-discovery systems.
How Platform Search Amplified Prohibited Content
Researchers found that typing search terms like "nudify," "undress," or "deepnude" returned results where nearly 40% of the top ten apps could render women nude or scantily clad. More troubling: some of these apps appeared as sponsored search results — meaning Apple's ad network was monetizing queries for content its guidelines explicitly prohibit.
In one documented case, searching "deepfake" surfaced a promoted listing for FaceSwap Video by DuoFace. The app allowed unrestricted face-swapping between clothed and nude bodies. Another sponsored result for "face swap" led to AI Face Swap, which performed the same function with no content filters.
The wiki:apple-search-algorithm itself contributed through autocomplete. Typing "AI NS" — a partial search string that could lead to "AI NSFW" — triggered the App Store to suggest "image to video ai nsfw" as a completion. That suggestion then returned multiple nudify apps in top results.
Google Play exhibited the same behavior. Search terms like "nudify" and "undress" returned exploitative apps, with the store's autocomplete function actively guiding users toward them. Some apps carried an "E for Everyone" rating, making them legally downloadable by children despite generating sexually explicit deepfakes.
The Enforcement Cycle Repeats
This marks the second report on nudify apps in three months. An earlier investigation in January documented similar findings. Apple and Google removed flagged apps at the time. Yet within weeks, new apps reappeared, using similar names, keywords, and functionality.
When contacted about the latest findings, Apple removed 15 apps and contacted developers of six others, giving them 14 days to address violations or face removal. Google stated that "many of the apps referenced in this report have been suspended" and that its "investigation and enforcement process is ongoing."
Both responses follow the same pattern: reactive removal after public reporting, no visible change to the wiki:app-review-process that approved the apps in the first place, and no disclosure of how the apps passed review with "E" ratings or qualified for paid promotion slots.
Developer Testimony Exposes Supply-Chain Gaps
In at least one case, investigators contacted app developers directly. One developer confirmed using xAI's Grok model for image generation but claimed ignorance that the model could "produce such extreme content." The developer committed to tightening moderation settings.
This raises a structural question: if third-party AI models can generate policy-violating content by default, and developers are unaware of this capability until after App Store approval, the review process is failing to assess the actual behavior of submitted apps.
Apple's updated statement emphasized that "nudify" apps violate App Review Guidelines that prohibit overtly sexual content and require developers to filter objectionable user-generated material. The company noted it had "proactively rejected and removed many nudify apps" and blocked search terms highlighted in the report — some before the report was published, others afterward.
Apple also clarified that its advertising policies prohibit adult content, ads are not shown to users under 13, and advertisers cannot target users aged 13–17. The statement did not explain how apps violating sexual content policies qualified for paid ad placements in adult-oriented search queries.
What This Means for Store Trust and Discovery
The case illustrates a growing gap between written policy and operational enforcement. Both Apple and Google maintain detailed content guidelines. Both invest heavily in automated review systems and machine learning for content moderation. Yet their own revenue-generating discovery systems — search autocomplete, algorithmic ranking, and paid placement — are amplifying the exact content those policies forbid.
For developers in compliant categories, this creates an uneven playing field. Apps that follow guidelines compete for visibility against apps that violate them but benefit from search promotion and ad spend. For users, it erodes app store search as a trusted filter — the assumption that what appears in results has been vetted breaks down when the top results include prohibited content, some of it sponsored.
The enforcement loop — public report, reactive removal, reappearance of similar apps weeks later — suggests the problem is structural, not incidental. Until review processes assess apps' runtime behavior with third-party AI models, and discovery systems apply the same content filters that govern approval, the cycle will continue.