(Can’t) Picture This: An Analysis of Image Filtering on WeChat Moments

Jeffrey Knockel, Lotus Ruan, Masashi Crete-Nishihata, and Ron Deibert:

WeChat (the most popular chat app in China) uses two different algorithms to filter images in Moments: an OCR-based one that filters images containing sensitive text and a visual-based one that filters images that are visually similar to those on an image blacklist

We discovered that the OCR-based algorithm has implementation details common to many OCR algorithms in that it converts images to grayscale and uses blob merging to consolidate characters

We found that the visual-based algorithm is not based on any machine learning approach that uses high level classification of an image to determine whether it is sensitive or not; however, we found that the algorithm does possess other surprising properties

For both the OCR- and visual-based algorithms, we uncovered multiple implementation details that informed techniques to evade the filter