You all are well aware of the chatbots and probably you all have the experience of using Chatbots for solving product or service queries. But the most important thing you all noticed that there is a pretty well chance that you are not able to get a solution of your queries by the Chatbot which further immediately solved by the human service agent. But recently in an interview, Google said that with the release of its latest Chatbot Meena there is no such issue left and all the queries are immediately solved. They also said that this Chatbot is the best in comparison to others.
Google also added that Meena is the neural conversational model with 2.6 billion parameters that can easily conduct chat with people better than any other AI generator. The conversation with the help of Meena is more sensible and better than any other Chatbots. Google’s team trained the model on 341GB of text and the text is filtered from public area social media conversations. In the last year there also a sophisticated language model was released called OpenAI GPT-2 and when it compared with Google’s latest Meena Chatbot, and then it was found that Meena was trained on 8.5x higher data and it also has 1.7 x greater model capacities.
The important part of the Meena Chatbot is seq2seq architecture which is able to understand the relationship between words in a paragraph. In the Chatbot Meena, there is a single encoder block that helps in processing the conversation in order to understand the context and there are also 13 decoder blocks that use the processed information to formulate the response.
It is hard to measure the quality of the Chatbot, so in order to reduce this issue; Google designed a new metric which is known as SSA, sensibleness and specificity average. This metric helps in measuring the sensibility and specificity of a conversation as well as response. So, it is necessary for every Chatbot to be measured on this metric. To measure this metric between 1600 and 2400 conversation are collected through about 100 conversations. Now with the use of this metric, people can easily check the specificity and sensibility of each label and response. The sensibleness of the Chatbot depends on the sensible response and specificity depends on the specific response. From these two, the average outcome is the SSA score.
From the SSA metric, it was found that the average score of Meena Chatbot was around 80%, whereas its competing bots average score was 56%, which was not good in comparison to Meena Chatbot. But Google is not releasing its latest Chatbot for external research as it is still working on its factuality and personality.