Skip to main content

​
Overview

AI Moderation in the CometChat SDK helps ensure that your chat application remains safe and compliant by automatically reviewing messages for inappropriate content. This feature leverages AI to moderate messages in real-time, reducing manual intervention and improving user experience. For a broader understanding of moderation features, see the Moderation Overview.

​
What Triggers Moderation?

Moderation is triggered only for the following message types:
  • Text Messages
  • Image Messages
  • Video Messages
Other message types are not subject to AI moderation.

​
Handling Message Moderation Status

When sending text, image, or video messages, the moderation process introduces the following changes to the message flow:
  1. Initial Status:
    • On successfully sending a text, image, or video message, the response will include the moderation status as pending.
    • You can fetch this status using the moderationStatus property, which returns an Enum: CometChat.ModerationStatus.PENDING.
  2. Moderation Result Event:
    • Once the moderation service processes the message, a real-time event is triggered in your MessageListener as onMessageModerated.
    • This event provides the message object with the updated moderation status, which will be either CometChat.ModerationStatus.APPROVED or CometChat.ModerationStatus.DISAPPROVED.
Example:
  • Dart
// Check moderation status after sending a message. This check is only applicable for TextMessage & MediaMessage class. 
if(message.moderationStatus.value == ModerationStatusEnum.PENDING.value) {
	// Message is under moderation
}

// Listen for moderation result
class Class_Name with MessageListener {
	// CometChat.addMessageListener("listenerId", this);
	@override
	void onMessageModerated(BaseMessage message) {
		if(message is TextMessage) {
			if(message.moderationStatus?.value == ModerationStatusEnum.APPROVED.value) {
				// Message approved
			} else if(message.moderationStatus?.value == ModerationStatusEnum.DISAPPROVED.value) {
				// Message disapproved
			}
		} else if(message is MediaMessage) {
			if(message.moderationStatus?.value == ModerationStatusEnum.APPROVED.value) {
				// Message approved
			} else if(message.moderationStatus?.value == ModerationStatusEnum.DISAPPROVED.value) {
				// Message disapproved
			}
		}
	}
}
⌘I