Image Moderation

Image Moderation automatically scans every image your members upload and uses Google Cloud Vision SafeSearch to detect adult, racy, violent, medical, or spoof content. You choose whether matching images are blocked outright or held for moderator review.

Navigate to Moderation > Image Moderation

How It Works

When a member uploads an image, ChatNet sends it to the Google Cloud Vision API for SafeSearch analysis.
Vision returns a likelihood rating for each category — Very Unlikely, Unlikely, Possible, Likely, Very Likely.
If any enabled category meets or exceeds your Sensitivity Threshold, the image is flagged and your chosen action is applied.
If the Vision API is unreachable or the key is invalid, ChatNet fails open — the upload is allowed so chat is never blocked by an outage. Keep your key valid so moderation actually runs.

Step 1 — Get a Google Cloud Vision API Key

Image Moderation needs an API key from a Google Cloud project with the Cloud Vision API enabled.

Open the Google Cloud Console and sign in.
Create a project (or select an existing one) from the project dropdown at the top of the page.
Open the Cloud Vision API page and click Enable.
If prompted, set up a billing account. Vision requires billing to be enabled, but includes a free monthly tier (see Pricing below).
Go to APIs & Services > Credentials.
Click Create Credentials > API key and copy the key that is generated.
(Recommended) Click Edit API key, and under API restrictions choose Restrict key > Cloud Vision API. This limits the key so it can only be used for image moderation.

Tip: Treat the key like a password. ChatNet stores it masked, and restricting it to the Vision API (step 7) means it is useless if ever leaked.

Step 2 — Configure Image Moderation in ChatNet

Go to Moderation > Image Moderation and complete the form:

Setting	Description
Enable Image Moderation	Set to Yes to turn scanning on.
Google Cloud API Key	Paste the key from Step 1. It is stored masked — use the eye icon to reveal it.
Action for Flagged Images	Block Upload rejects the image so it is never posted. Flag for Review allows the upload but creates a report for moderators.
Categories to Check	Tick the content types to detect (see below).
Sensitivity Threshold	The minimum likelihood that triggers a flag. Lower = stricter.

Click Update to save.

Detection Categories

Category	Detects
Adult	Explicit nudity, sexual activity, pornography.
Racy	Suggestive content such as revealing clothing or provocative poses (less explicit than Adult).
Violence	Blood, gore, and violent imagery.
Medical	Medical or surgical imagery.
Spoof	Memes and altered images. Useful for reducing edited spam rather than safety.

Sensitivity Threshold

Threshold	Behavior
Possible	Strictest. Flags anything rated "Possible" or higher — catches the most content but may cause false positives.
Likely (Recommended)	Balanced. Flags "Likely" and "Very Likely" matches.
Very Likely	Loosest. Only the most obvious violations are flagged.

Reviewing Flagged Images

When the action is Flag for Review, flagged uploads create a report under Moderation > Flaged Content. See IP Access & Flagged Content for how to review and resolve reports.

Pricing

Vision SafeSearch is billed per image (a "unit"). At the time of writing the first 1,000 units per month are free, after which SafeSearch Detection is roughly $1.50 per 1,000 images. Always confirm current rates on the Vision API pricing page.

Troubleshooting

Symptom	Fix
Images are never flagged	Confirm the key is valid, the Cloud Vision API is enabled, and billing is active. ChatNet fails open, so an invalid key silently allows all images.
403 / "API not enabled"	Enable the Cloud Vision API on the same project the key belongs to.
Key rejected	If you restricted the key, make sure Cloud Vision API is in the allowed list.

AI Moderation — moderate text messages with AI.
Moderation Tools — domain filter, flood control, profanity, captcha.
IP Access & Flagged Content