Analytics
After training Helper's interactions and dialogues, it's necessary to monitor your assistant's interactions with users to ensure that it's being effective in attending to the requests, ensuring its quality and assertiveness.
These interactions can be viewed in the Analytics option. It is the session of the platform responsible for viewing reports and indicators. The modules are divided into Conversation List, Rankings of Intents, Ranking of Responses, and Fallback Ranking.
Before getting started
- It is necessary to have performed at least one training;
- Have performed at least one interaction with the virtual assistant.
Procedure
- After accessing the platform, go to the menu "Analytics";
- Select one of the modules available: Conversation List, Rankings of Intents, Ranking of Responses, and Fallback Ranking.
Conversation List
In this module, it is possible to view all conversations performed by the chatbot during a selected time. The system will present information regarding the conversations held during that period.
After clicking on "List conversation", first, it is mandatory to define a period of time in the "Interval" field.
After setting a time, click on "Filter”.
The tool will present the conversation information depending on the time of your choice.
We will have the following information for analysis:
Field | Description |
|---|---|
Date and time | The date and time that the conversation between a user and the assistant was performed. |
Version | The version will vary according to the maturity of the chatbot. This means that the version of the assistant changes according to the amount of information it receives and the training it's submitted to. |
Channel | The channel that the user used for the conversation. It can be from different channels, as configured on the Helper platform. For example Facebook, Twitter, Smart Chat, and others. |
User identification | The identification of the user who chatted with the assistant. |
In addition to this information, it also presents two options:
Field | Description |
|---|---|
See | In this option, it is possible to see the details of the conversation between the user and the assistant. The blue balloon is the user's message. We can see that below it there's a percentage, which means the accuracy of the assistant and, next to it, the intent that was understood by Helper. If you want to include this user phrase to the understood intent, just click on it and you will be redirected to the dialog page. The white balloon is the message sent by the assistant. Below it there's the skill that was registered for this type of intent. If you want to include something new in this conversation for the skill, just click on it and you will also be redirected to the dialogs page. In addition, it also presents the options: "Update", in case the conversation is still happening and you want to receive the new messages; "Export" to download this conversation; and "Back" to return to the previous page. |
Download | It is possible to download the conversation in xls format, where you can see the interactions between the user and chatbot. |
Ranking of intents
In this modality, it is allowed to check-in chart format which intents are being used most by system users, that is, what users are asking and/or demanding most from the assistant.
After clicking on "Ranking of intents", first, it is mandatory to define a period of time in the "Interval" field. After setting a time, click on "Filter".
A chart will be displayed with the main intents according to the selected period. On the left side there's the chart itself, in days, and on the right side, the subtitle about what the colors represent and their particular intents.
At the end of each intent, there is a magnifying glass. When clicking on this magnifying glass, you will be taken to the conversation list page for that intent, which will show the list depending on the period reported for the chart.
In addition, when you click on any intent in the subtitle, you will be directed to the related dialog page, where you can analyze whether the phrases are correct or make any changes.
Ranking de responses
It works similarly to the ranking of intents. In this modality, it is allowed to check which skills are being used most by the users of the system, that is, what the assistant responds most during the interaction with a user.
After clicking on "Ranking of responses", first, it is mandatory to define a period of time in the "Interval" field. After setting a time, click on "Filter".
A chart will be displayed with the main skills according to the selected period. On the left side, there's the chart itself, in days, and on the right side, there's the subtitle about what the colors represent and their specific skills.
At the end of each skill, there is a magnifying glass. When clicking on this magnifying glass, you will be taken to the conversation list page for this skill, which will show the list depending on the period reported for the chart.
In addition, when clicking on a skill in the subtitle, you will be taken to the related dialog page, where you can analyze whether the phrases are correct or make any changes.
Ranking of fallback
In this modality, we can check which are the intents that the system was unable to understand, that is, which interactions with the user were unsatisfactory because the assistant was unable to understand the purpose of the message.
The fallback, for the system, is a previously registered skill that presents a message as a response to the user, created by a platform administrator in order to inform the user that the assistant does not have registered and trained knowledge sufficient to meet the request.
After clicking on "Ranking of fallback", firstly, it is mandatory to define a period of time in the "Interval" field. After setting a time, click on "Filter".
A chart will be presented with the fallback skills that were not understood by the assistant, according to the selected period. On the left side, there's the chart itself, in days, and on the right side, there's the subtitle with the fallback skill.
Clicking on the fallback subtitle will take you to the conversation list page for that skill depending on the days selected in the filter. When viewing these conversations, the analyst can verify if the assistant's lack of understanding is correct, that is, if the user is demanding something outside the scope of her creation or if it is a deficiency, in which case the administrator can improve the interaction and accuracy, building new conversation flows or refining existing ones.