What is this feature?
Safety moderation for apps allows builders, space owners, and organization admins to review when conversations with their apps contain moderated or flagged content. You can now review these conversations and ensure your apps maintain high-quality interactions.All moderated messages and manually flagged inappropriate messages are hidden from view in the conversation by default to protect users from potentially inappropriate content.
Rationale for the feature
Playlab users have been asking for improved visibility into moderated or flagged conversations with their apps. With safety moderation for apps, administrators are promptly notified when moderation occurs, enhancing transparency and enabling quick action. This feature makes it simple to identify and review potentially problematic conversations, allowing you to update your apps as needed and maintain a safe, high-quality user experience. This feature also allows users to flag conversations that they deem to be inappropriate or biased. This will then inform the creator of the app via email.How do notifications work?
User Flagged Conversation
Users can flag any AI message with the flag icon under it and choose Biased, Inaccurate, or Inappropriate. A message flagged as inappropriate is hidden. Flagged conversations appear in the Sessions tab to app creators and those who have access, empowering users to provide direct feedback on conversations they find problematic.Reviewing Flagged Conversations
The flagged conversation review interface provides a streamlined workflow for administrators to assess, address, and resolve reported issues efficiently. You can see separate icons for flagged user conversations and messages that were moderated. They appear in your app’s Sessions tab; click Needs attention to show only those sessions, and turn on Jump to messages that need attention to open each session at the flagged or moderated message. Organization Owners can also see every user-flagged session across the org in Admin → Sessions: open More filters and tick Flagged by users.For org admins: to review flagged and moderated sessions across every app in your organization, see Safety Moderation for Organizations. The Centralized org monitoring page has a video walkthrough of reviewing activity across every space.
Builder Test Conversations
Conversations you have while testing your own app in Builder preview mode also appear in the Sessions tab, alongside end-user conversations (the Builder filter is on by default). Any moderated or flagged messages from those test sessions are reviewable the same way. This means you can:- Spot moderation issues during testing, before you share the app
- Iterate on your app’s instructions when your own test prompts trigger flags
- Keep a complete audit trail of every conversation your app has participated in
Remember that moderated content may contain inappropriate material. Review with caution.
Tips for Managing Flagged Conversations
Best Practices
- Analyze patterns in flagged content: Look for trends in the types of content being flagged to identify potential improvements for your app.
- Update your app’s instructions: If you notice recurring issues, consider updating your app’s instructions to prevent similar problems.
- Follow up with users when appropriate: For identified users experiencing issues, consider reaching out to provide support or clarification.
- Document your resolution steps: Keep track of how you resolve flagged conversations to improve your response in the future.
- Consider required guardrails: For apps dealing with sensitive topics, implement clear guardrails in your instructions to prevent moderation issues.
How long are flagged conversations stored?
How long are flagged conversations stored?
Flagged conversations are stored indefinitely at this time. There is no automatic resolution system currently in place.
We Want Your Feedback!
Have you tried reviewing flagged conversations in your apps? We’d love to hear about your experience!Contact us at [email protected]
Last updated: 09-30-2026