The Making Of A Conversational Platform

1. CONVERSATIONAL PLATFORMARCHITECTURE WHITEPAPER

2. TABLE OF CONTENTS 04 What is a conversational platform? 1. The difference between chatbots and virtual assistants 04 2. The shift from Graphical User Interface To Conversational Interface 05 3. High-level Architecture of a Conversational Platform 06 4. Platform Capabilities 06 5. Intelligence Insights Support desk Developer Tools Browser or plug-in requirements Multiplatform support 07 6. Voice 08 7. Yellow Messenger’s Bot Engine 09 8. Stateful Hybrid response model Multimodality 09 9. Natural language processing & Deep learning Model 09 10. Intent Matching 13 11. Database 14 12. 15 Integrations 13. Response Generation 15 14. Analytics and Supervision 15 15. Using this guidebook to your advantage 16 16. About Yellow Messenger 16 18. 02 www.yellowmessenger.com

3. Understanding the high-level architecture of Conversational AI Platform that can help leaders make a tactical decision to choose the right automation solutions from the right vendors. The rapid developments in deep learning and natural language processing in artificial intelligence led to an increase in the availability of conversational platforms during the period of 2017-2018. As of now over 1500 conversational platform vendors exist worldwide. The market is labile, to say the least at this stage. Although some solidification will begin by 2023, the conversational AI panorama remains large, huddled and comprises either early-stage startups or large enterprises and service software companies like CRM’s, etc. Many solutions provided could be premature or could lack the essentials for sophisticated implementations. For example, a bot orchestration engine which harmoniously synchronies and performs a multitude of functions amongst an army of bots (~300 chatbots). Such an implementation needs a robust learning loop for the virtual assistant - both master and child bots. This is a rare, sophisticated bot feature that is limited to a select few vendors in the world. Clients such as large conglomerates and enterprises with multiple use cases will benefit from a platform approach. Given the structure of these platforms and the services provided by them, application leaders can positively look forward to two types of conversational platforms in future - ones built around strong proprietary NLP engines, and those that are conversational intermediaries focusing on orchestration and product life cycle. The use-cases for which conversational platforms are aggressively pushing boundaries in data science and developments to customize the core functionality could create layers of complexity. It is getting increasingly difficult to compare the offerings of conversational platforms, more so, since application leaders and decision makers aren't fully aware of what goes into making and deploying an AI solution for businesses. If they don’t know what to look for, they can’t decide if it’s up to par. This conversational platform architecture is an effort to help application leaders make the right choice. Before we deep dive into the architecture, let’s look at some market definitions and insights. 03 www.yellowmessenger.com

4. WHAT IS A CONVERSATIONAL PLATFORM? A conversational AI platform comprises features that are used to build conversational user interfaces like chatbots and virtual assistants for a wide array of use cases. For example, building a chatbot that allows customers to order groceries on WhatsApp. A virtual assistant can be deployed on any interface such as messaging platform, social media channels, SMS, web, voice, etc. A conversational platform has a developer API or a software development kit (SDK) which can be used to extend the platform with your own customizations and additional capabilities. THE DIFFERENCE BETWEEN CHATBOTS AND VIRTUAL ASSISTANTS Chatbots have a limited learning loop. It can only handle queries that are already fed to it manually or can handle the queries that come from finite sources of data. It is mostly used as a FAQ answering function. They cannot process languages unless fed as an input. Here, the need for human intervention is high. Virtual assistants have a dynamic and complex learning loop. It can process multiple languages. If the virtual assistant is new to a query, it can still provide accurate answers because of its heuristic approach and give a better experience every time. They perform complicated tasks such as analytics, providing insights, studying patterns in data to predict future behaviors, and so on. MARKET DEFINITION Chatbot Virtual Assistant IMPLEMENTATION Conversational Platform ENABLER Sophistication 04 www.yellowmessenger.com

5. THE SHIFT FROM GRAPHICAL USER INTERFACE TO CONVERSATIONAL INTERFACE Application leaders everywhere need to be cognizant of the various changes in consumer behaviour, especially in these times of uncertainty. People are looking for solutions outside their realms, which brings us to the fundamental shift from a graphical user interface to the conversational interface. A GUI makes the user in charge of technology. A conversational interface allows people to instruct the virtual assistant wherein the virtual assistant determines the intent and fetches the accurate responses. Conversational platforms are predicted to be the dominant interaction interface by Gartner. Some vendors merely claim to have a conversational interface, yet all the user is doing after the initial request is clicking on pre-defined question options. Although this may suffice for certain use-cases, it is not scalable to others. In fact, this isn’t a virtual assistant at all. What's more, such implementations are merely click-bots and do not make up a conversational AI platform. Chat with us Chat with us Chat with us Lana: Lana: Lana: Welcome to our Support Chatbot. How may I help you? Please type in your product name, or choose a category below: WiFi routers & modems Lana: WiFi cameras Smart home Please write your question or choose an option to start: Business surveillance What seems to be the problem with your WF1204 Home Router? Troubleshooting User guides Type the issue or choose from the list below: New releases Updates Visitor: Connection issues Signal strength WF1204 Authentication issues Visitor: Frequent Disconnections More... I need Troubleshooting Send Send Send 05 www.yellowmessenger.com

6. HIGH-LEVEL ARCHITECTURE OF A CONVERSATIONAL PLATFORM For effective evaluation of conversational platforms, it is recommended that application leaders get a clear understanding of the high-level architecture and its capabilities as the market is growing speedily yet remains labile. This architecture for most vendors is common, hence we highlight the key capabilities and the areas of its volatility. PLATFORM CAPABILITIES BOT PLATFORM CUSTOM USER INTERFACE UI capabilities rely on modality. A question to ask - Is the conversational platform flexible enough to implement requirements as and when they arise? The conversational platform acts as a user on the communication and messaging platforms that people already used to communicate (Facebook Messenger, Kik, WeChat, KakaoTalk, Viber, WhatsApp, Slack and Telegraph, and other messaging platforms, including email and website chat). Yellow Messenger is a fully equipped platform, which can create a bot from the UI. INTELLIGENCE An administration console to track your bots' health and activities is of utmost importance. At Yellow Messenger, our admin console facilitates intents configuration, training, retraining, configuring responses and more. It allows humancuration of the bots. Channel integrations can be made from admin console. INSIGHTS Insights is the analytical tool that helps users with various aspects - a. Geographic distribution of users b. Traffic data c. Channel wise segmentation d. Topological segmentation e. Intent wise segmentation 06 www.yellowmessenger.com

7. SUPPORT DESK Support desk is the agent dashboard, where human agents can respond to the various queries. It also behaves as a conversation repository. In the AI chatbot, we can configure the bot to raise a ticket as a response for a particular intent. The bot for a specific user can be paused or resumed from the module. DEVELOPER TOOLS Developer module gives the flexibility of overriding the default bot behaviour with scripts. The scripts are injected into the library of existing functions, with higher priority. A tech savvy curator can extract a plethora of functions out of the bot using this module. BROWSER OR PLUG-IN REQUIREMENTS Chat implementations sometimes use browser capabilities, such as WebRTC, which have varying support in browsers. With this, customers can connect via voice or video call with agents to get accurate support for more complex services that perhaps required physical presence. MULTIPLATFORM SUPPORT When application leaders are comparing conversational platforms, a few major questions one should ask include - What messaging applications does the platform support? How many do they support? Will you have to deploy a chatbot for each integration? Does the vendor have a quick turnaround time? Are they modular enough to incorporate the integrations that your business desires? What is the effort required from your side as a client? Would you need to code? Every messaging platform supports different capabilities in input and output. One should understand what they support and accordingly align it with business needs. For example, some pointers for a smooth conversation between the business and it’s consumers to consider are rich input (Document sharing), rich output(images/video), request length (short requests, single intent, larger requests, complex intents), multiple participant support(broadcast), etc. 07 www.yellowmessenger.com

8. VOICE Chat is an interface whereas Voice is an input. The two need a robust natural language processing engine to function smoothly. Some players today position themselves as a voice-only or voice-first conversational platform (call center automation). Some questions you might ask - What are the languages supported? Which languages are they? How many more can be incorporated? What is the complexity to which the inputs are processed? (For example, every language could have several dialects and regional vernacular versions hard to map.) Speaker recognition. Speaker isolation or noise cancellation. Devices supported. Pattern analysis & improvements and so forth. 08 www.yellowmessenger.com

9. YELLOW MESSENGER’SBOT ENGINE Bot engine is a state of the art algorithm that facilitates the bot’s behavior. The salient characteristics of the model are: Stateful: The context from previous messages is maintained with delicacy and can be transferred to the next conversation. Hybrid response model: Our proprietary response model is hybrid and is designed to take care of responses that aren’t random. The hybrid response model is intelligent as it answers global utterances as well. MULTIMODALITY Chat & voice is the primary focus for most platforms. Capturing other forms of input has the potential to improve the accuracy and the quality of the experience. Gartner predicts this being a major differentiator over the next five years. NATURAL LANGUAGE PROCESSING & DEEP LEARNING MODEL Natural language processing is where the conversational platform picks up the input text and converts it into objects understandable by the system. This is where the intent matching occurs. This is a critical aspect and deserves detailed scrutinization. Some capabilities to consider are as follows - LANGUAGE SUPPORT AND VARIANTS How many languages are supported, and which variants (dialects, etc) does the NLP engine recognize? LANGUAGE DETECTION Eastern regions compared to the west, for example India, tend to have diverse dialects in just a small region as compared to the West. The overlapping can make it difficult to detect. Does it account for these details? 09 www.yellowmessenger.com

10. SENTIMENT ANALYSIS Understanding the opinions expressed in the conversation by the user and deciphering the attitude towards topics. SENTENCE REWRITING This is a way to handle common challenges, such as misspellings, slang, synonyms or even sentence structures (e.g., double negatives). Sentence rewriting increases accuracy in intent matching. SEMANTIC ENRICHMENT In the NLP step, text is typically enriched with semantics based on the internal knowledge base of terms and expressions (for example, tagging names, companies and actions mentioned in the text). DOMAIN SPECIFICITY & ROUTING MODELS Several layers of specialization are possible in the NLP engine. A typical general-purpose vocabulary, out of the box, is likely to be ill-suited for most implementations. But with Yellow Messenger’s NLP engine, even little to no data is sufficient for the bot to give accurate results. The various levels of specificity include industry, purpose, customization and trained specificities. Domain specificity can also qualify the routing of inputs for one or more domains. TRAINING REQUIREMENTS For achieving the results required, an implementation may or may not require additional training data, even if the platform has good performance right off the bat while in some other cases, this data may not even exist. At Yellow Messenger, we never leave the customer hanging with an unpleasant experience that stems from such shortcomings. Our NLP model achieves the highest accuracy with three major components - NLP Parser: The context from previous messages is maintained with delicacy and can be transferred to the next conversation. Heuristic mode: The noun chunks are then fed into the trained heuristic model to predict for the intents and tags. The Heuristic model is a hybrid, supervised machine learning model, which is trained manually by giving a set of utterances. As the number increases and by mapping unidentified utterances, the model’s accuracy goes up in predicting intents and classifying tags. Response Store: Response Store stores all the responses that can be chosen upon identifying a particular intent. 10 www.yellowmessenger.com

11. Some other aspects to look out for include unsupervised learning and ease of translations. *Our proprietary NLP engine is highly efficient with the amount of data that the user provides. F1-score is the measure of the accuracy of a test. We have the F1 scores for Dialog Flow by Google, Luis by Microsoft, and us. While performing the test it was observed that - The performance of all three engines was measured using the datasets of three community-driven support ecosystems; Ask ubuntu, Banking 77 and Stack Exchange. We calculated the F1 score across 10 different stints using datasets provided in 2 different academic papers. Yellow Messenger’s NLP engine has achieved better results in all stints. To go one step further we used random down-sampling to reduce the dataset by 50%. YM NLP engine still outperformed Dialogflow by 28% and Luis by 9% in this race. The highest score is 1, which hasn’t been achieved and is difficult. 11 www.yellowmessenger.com

12. Better performance than Dialogflow Better performance than LUIS With half the data 28% 9% Using 50% less data for those numbers F1-Scores 0.70 0.79 Banking 77 0.72 0.78 Ask Ubuntu (Datasets) 0.90 0.86 0.52 0.81 Web Applications 0.73 0 0.25 0.50 0.75 1 Methodology: f1-score calculated as Average of 10 runs, downsampling for 50% less data. Citations: 1. Efficient Intent Detection with Dual Sentence Encoders - Casanova et al. (2020) 2. Evaluation natural language understanding service for conversational question answering systems - Braun et al. (2017) 12 www.yellowmessenger.com

13. INTENT MATCHING Intent matching is where the processed input is matched to the appropriate handler of the request. This usually uses ML. Intent matching and natural language processing are cogs of the same process. Some of the questions to ask are - Does the architecture of the platform exude sophisticated intent matching to give appropriate contextual information? Does the platform group intents intelligently? Vendors today are increasingly moving towards intent grouping and then focus on individual intents. At Yellow Messenger, we work on pre-defined knowledge graphs that manage these intents, entities and relationships between them in a highly sophisticated, efficient way. This model can be applied to any domain and customised further to fit a specific business. Does the platform support multiple handlers? Handlers are nothing but decision trees, and the complexity in these handlers should be measured to its accuracy in providing results. Does the platform recognise multiple intents from one or more inputs? Does it then accordingly prioritize responses? Does the platform help extract multiple intents that may or may not be directly related to the query? At Yellow Messenger, we not only have predefined models and templates but also pre-defined intents that make deploying a solution for your business faster. Our predefined models can be customised to fit any business or domain. Besides, Yellow Messenger also provides developers and other automation enthusiasts to extend the capabilities of our products by choosing any intent or bot capability to work on from our Bot Store. Bot store comprises hundreds of bots for various functions and industries that you can choose from and deploy almost immediately. In relation to contextual cues, intent matching and NLP, few other aspects to note include contextual awareness, conversational history, user preferences, user content, third party data, exception handling, behaviour prediction, sentiment analysis and response generation. 13 www.yellowmessenger.com

14. DATABASE Yellow Messenger’s state-of-the-art databases ensure fast retrieval, persistence and statefulness. Persistent DB: For persistence, we use MongoDB. MongoDB is a document store, which is well suited for machine learning purposes, as there will be constant scope for improvement in the form of feature improvements and training. MongoDB has excellent support for sharding and replication. Indexed DB: In order to make the text searches fast, we use Elasticsearch’s analysed db. Every request from our bot goes through Elasticsearch and it paves way for the fastest retrieval possible. Cache: YM’s cache services powered with Redis technology. Redis is scalable and known for its robustness. Queuing: Queuing is supported by Apache Kafka, with state of the art implementation. 14 www.yellowmessenger.com

15. INTEGRATIONS Although stand-alone chatbot implementations have a purpose, many use cases require integration with existing systems. Enterprises usually suffer from what is known as a knowledge debt since their data is scattered across departments and verticals. Besides this, their transactions and interactions are still based on legacy systems. There are several ways that this integration may happen in order to solve the issues of data irregularities, isolations and outdated systems since it is imperative before a chatbot can be deployed. RESPONSE GENERATION Anything besides pre responses requires higher sophistication at a minimum natural language generation. For example, providing compound responses to one input with multiple intents, text to speech conversion, voice synthesis, personalisation and confirmational cues. ANALYTICS AND SUPERVISION Analytics is inevitably the most crucial aspect and must be an inherent capability. Furthermore, the analytics help improve the performance and training of the virtual assistant as well. Some of the questions to consider are - Considerations should be taken according to how the metrics are being used: Is the need just for reporting at regular intervals, or via real-time monitoring? At Yellow Messenger, we allow reporting as per business needs. They’re all easily customisable as per the metrics that suit the function. Can the interface map missed intents to appropriate responses thus exhibiting a supervised learning loop? Does the system pin-point and propose improvements? Does it possess the ability to ensure consistent quality, as the implementation scales? 15 www.yellowmessenger.com

16. USING THIS GUIDEBOOK TO YOUR ADVANTAGE Given our discussion on conversational platform architecture, using this knowledge effectively to your advantage is our aim. Following measures should be taken next for choosing the right platform. Talk to as many vendors as possible. Always request for a demo of their platform to understand the differences minutely. Evaluate the roadmap of these vendors with current capabilities to realise the viability of these vendors. Do you think they will scale effectively? Determine the integrations that you will possibly need and the ones that you need right off the bat. Check to see how well are these integrations made possible by each vendor. Do you have to take on much work for integrating your systems? Scope the kind of capabilities you need for your business. Ask the vendors what you think could be the solution to your underlying business problem. ABOUT YELLOW MESSENGER Our platform is an end-to-end solution to all business needs from sales, marketing, HR, to customer support and ITSM. Our virtual assistants are capable of handling millions of conversations both internally (with other hundreds of child bots) and externally (with millions of customers around the globe). Our virtual assistants come with a tremendous amount of knowledge right off the bat and provide the highest accuracy in providing responses compared to other vendors in the market. 16 www.yellowmessenger.com

17. RELATED ARTICLES Enhanced Agent Assist & Customer Experience for Sayurbox Bringing a Customer Centric approach to NBFC through automation Read more Read more Follow us @ 17 www.yellowmessenger.com

18. To understand how you can optimize your business in a unique fashion that can also generate additional streams of revenue, get a demo with us today. Request demo www.yellowmessenger.com contact@yellowmessenger.com