Measure and improve

Reviewing Conversations to find knowledge gaps

Reviewing conversations is a crucial step in identifying knowledge gaps and improving the performance of your AI agent. By systematically analyzing user interactions, especially those with negative feedback or longer exchanges, you can uncover areas where the AI lacks information or provides unsatisfactory answers. This process helps you prioritize updates to your training sources and enhance the overall quality of your AI responses.

how to access and filter conversations

Start by navigating to the Conversations tab in your Unless dashboard. This section displays all conversations from the last 30 days, showing key details such as the date and time of the last message, the number of responses, user ratings, and conversation status. Use the available filters to narrow down the list based on criteria like rating, number of responses, status, audience segments, or specific user emails.

Persistent filters remain active throughout your session, allowing you to efficiently review a targeted subset of conversations without losing your place. For example, filtering for conversations with at least two responses often yields more insightful interactions to analyze.

analyzing conversation details

Click on the Details button for any conversation to view the full transcript, including all messages exchanged and any thumbs up or down ratings attached to responses. Each AI answer lists the sources it used, which you can open to verify the accuracy and relevance of the information provided.

In addition to the sources used in the final answer, the dashboard also shows related sources that the AI considered but did not include. Reviewing these can help you understand the AI’s decision-making and identify if important content is missing or outdated.

flagging and diagnosing issues

If you encounter incorrect or unsatisfactory answers, you can flag the response for internal review. When flagged, an analysis is generated immediately to help diagnose why the AI produced that response. This analysis remains accessible for future reference and can guide you in determining whether the issue stems from training sources, configuration settings, or other factors.

Flagged responses can also be reported as technical bugs to the Unless team, though this is optional. Adding notes when flagging helps communicate the problem clearly to your team.

capturing knowledge gaps with tasks and quality control

From the conversation details view, you can create knowledge suggestion tasks directly. These tasks track missing information or areas where the AI needs improvement, helping your team stay organized and focused on filling knowledge gaps.

Additionally, you can promote specific questions from conversations into your Quality control center. This allows you to refine questions and control answers, ensuring your AI training covers real user queries effectively.

monitoring negatively rated conversations

Regularly reviewing conversations with negative user ratings is essential to spot recurring problems and prioritize improvements. The AI insights dashboard provides an overview of negatively rated conversations, helping you stay on top of potential issues and maintain high-quality AI responses.

conclusion

By actively reviewing conversations, filtering for insightful interactions, analyzing sources, and flagging issues, you can systematically identify and address knowledge gaps in your AI. Using tasks and quality control tools ensures continuous improvement, leading to more accurate, helpful, and satisfying user experiences.