I want to develop an app where users can upload an share images. I am planning to integrate a disclaimer which says that it is prohibited to upload images which contain crime, sex, insults etc. Furthermore I want to implement a “report a user” button where multiple reported users get blocked. But most of the people do not care about it. My question is: Is there an image processing algorithm or a webservice that detect such images? How do all the other social networks handle this problem? It is impossible to do this by hand.
1
As far as I know, there is not any sort of algorithm to do that. The problem comes that what is offensive here may not be offensive where you live, or vice versa. And there are certainly things that are forbidden in China that other nations wouldn’t bat an eyelash at.
You could do an image search to see if the imagine is a remastered from some blacklisted site, but that won’t catch everything. You could do a pattern match to see if it has swastikas or giant penises, but that won’t catch everything.
In the end, you need humans to handle the maybes or the misses or the false positives.
2
The only algorithm I know of that has proven to be effective is being used to detect and report the distribution of child exploitation images (Microsoft’s PhotoDNA). Once an image is identified as illegal, it’s fingerprint is stored into a database.
To build the fingerprint of an image, the image is converted to greyscale and hashes are made of various regions of that image, so that digital processing doesn’t prevent image detection (source).
This still requires that images are identified by humans before they will be blocked. But offending images won’t return.