Transcription of: An Open-Source Chatbot Made With Rasa

so what I'm gonna do is I'm gonna make a chatbot and we're just gonna do that live coding style and what I've got here is a terminal now this is my development folder what I'm just gonna go ahead and do is I'm gonna make a new directory and the directory is gonna be called demo demo demo demo or whatever like I have many demo folders so this will be the fifth one I've made but I'm just gonna make a directory in here demo demo demo there you go and what I'm just gonna do is start a new virtual environment in here now I'm using Python three seven and what you can then do from Python three seven and I believe three six as well as you can call the vient module directly and just make a new virtual environment appear by just using this part in one line of code this will make a virtual environment and what I'm gonna do is I'm going to install Raza inside of that virtual environment the downside is this Raza has a bunch of dependencies including tensorflow so installing might take a while but I figured it would be a good idea that we have this thing installing in the background while I start explaining a little bit about conversational interfaces and what's tricky about them and why an open source solution is probably a good idea so I'm just gonna source this virtual environment activate it and let's do it the neat way with the module thing and then I'll do Python pip install Raza this should be the only thing you need but this install can take a while so just checking christian kamilla so far I'm not going too fast I hope I'm perfect so this stuff is installing and we'll get back to it in a bit but what I kind of want to talk about are like we're gonna build a chat bot from the command line it'll be the end goal of this entire little workshop but let's talk a little bit about chat bolts in general so my name is Vincent definitely ask me anything about chat bots but what I am is I'm a research advocate so it's my job to make sure that whatever research we do at Raza that people understand why we're doing that research and also I do my best to make sure that I even gel eyes open source stack so what I'm gonna do now is I'm just gonna talk to a chat bot that's supposedly state-of-the-art so what I've done here is I've opened up the conve I from hugging face and hugging face is an impressive company and they're really doing some good work but this is a state of the art chat pod in a sense that it's doing everything fully generative and I'm just gonna talk to it and we'll see where it might fail it's gonna be different failing than what we saw with Siri though so you're gonna type hello here and says hello how are you doing and let's pretend that I'm interested in buying a pizza good I want to buy a pizza and then it says that sounds good I'm a freelance accountant and the thing is like if you think about everything I was doing and the fact it has to generate all of this text I mean there's definitely something state-of-the-art happening here generating this grammatically correct sentence is on its own certainly not unimpressive but if I now say Pizza hawai please don't let's start saying that it is a free accountant and it has OCD which is an original response but this is not a system you want to put in production that that's kind of the thing it's it's a really cool demo but I really want to have a chat bot that's kind of constrained that says hey for this one use case you're allowed to have responses because in this case you know it's talking about OCD and they're really a fan of that but there's also a risk that maybe the chat bot has learned from a large corpus of reddit text and it starts swearing which is also something that you don't want to have in production so even though this is still definitely like a state-of-the-art result this is not something that at some point you're gonna go and put in production and razza we kind of care more about you know the theory and articles are great but we want to have chat bots that actually go ahead and help people so that means that we typically design our assistants with a little bit of a constraint but one thing I should point out first and foremost and that's I'm assuming that a lot of you are data scientists and a lot of you are familiar with this system called scikit-learn that's pretty much the the get-go these days and in a you know scikit-learn pipeline life is relatively simple we've got this X that we put into our system and then at the end we get this prediction that comes out and we can have all sorts of like pre-processing steps in here and machine learning models in here and that's fine but one thing goes in and one thing goes out we're making a chatbot it's a little bit different because we still have let's say one thing going in we have a user here that says I want to have pizza but then the assistant in this case needs to detect multiple things for starters we need to say hey this uh pterence that indicates a certain intent namely this user wants to buy something but not only does that user want to buy something the thing that it wants to buy is a pizza so we have to sometimes may be able to find entities in the text that are relevant for processing the message and then we have to give it back an action and this action sometimes has to be driven by custom Python code so we are definitely going to do machine learning and there's definitely neural networks that are sort of in the background of this but I do hope that at the moment already you recognize that we're really in a different domain we are not in this simple pipeline one thing goes in one thing goes out but what Raza offers is essentially a system where you can still put in your own custom pipelines and components here but Raza also offers the framework to deal with all of this around it and to give you a bit of an example of that this is actually somewhat involved I've got this one example of a person trying to order a pizza here so you know we've we've got the user here and then we have the digital assistant over here and a theoretical conversation could be that you say hello and then the first action that this this assistant has to do is it has to reply with hello back after that the assistant has to listen we have to not send anything back but the action that we should do is we should listen to the user to figure out what to do next then the user in this case could say hey I'd like to buy a pizza and then the digital assistant says oh what kind of pizza what you like and what the user that might do instead of actually answering this question the user might actually throw some sort of an interruption that says oh yeah by the way are you a human or are you a digital assistant and then it will be nice if the chat bot was able to say well first I'm going to answer your question then I'm going to pick up the conversation where we left it and then I'm gonna go ahead and listen again to what the user has to say and this whole system all the machine learning you need for this razza is a nice open-source tool to handle this situation but that's sort of what we offer but what I hope that you recognize is that we don't have the same pipeline anymore and also we're training the system for a sequence of tasks it's not just text goes in and a single action comes out as you see in this example the sequence of actions that have happened so far they really matter so that so I hope you also agree that we're not gonna start this research by loading in a CSV file I mean that's not gonna work for our use case here we need to have some sort of a data format that's a little bit more than just see his feet so we're gonna start a new Rossum project in a moment but there's a couple of files that Arase projects should have the data for this project is typically stored in markdown files and it might be a little bit counterintuitive to what you're used to because it's not really common to do machine learning or markdown files but the idea is that markdown gives us sort of a domain-specific language to declare conversations and that's typically quite useful so in this example I have my annal unit MD file here and NLU stands for natural language understanding and the idea is that we can specify a certain intent so right now I've got the intent for a greeting and then under that I have a list of examples that fit that specific intent and what's nice is that this file will eventually get into git and it will be easy for people to add examples to certain intense or add an intent but we basically say here intent examples intent examples that that's the way that we're gonna define the understanding but having just an intent is typically not enough because we saw the example before that we have this order that we have to keep in mind and you can definitely assume that you know saying goodbye usually happens after saying hello and not vice versa so yes we have this part of the data but we also have are these stories and the idea is that if this intent is like a Lego brick then a story is where you can click these Lego bricks together to fill sort of two constructs a conversation that could have happened in the past so for example we have this happy path over here where somebody says hey hello the robot says back hello otter greet and then the user says I'm happy and then the robot says okay I hear that you're happy there's a example of a story that's a conversation that could have happened so if I can ask greed and know that's a really good question by the way mood Greed's is the intent after happy is the action from the assistant so we have some definition of an action that should happen over here and the idea is that the training data that we're generating to some extent there's a machine learning model that will detect ah given this intent this is probably the best action that we're interested in but there's also a different model that will sort of handle the fact that we're moving a long time here as well I'm not going to go too much in detail because this is meant as a more of an introduction thing but mapping this text to the correct intent we typically refer to that as a classifier but given a sequence of intents and responses what's the next best action now we call that a policy and as a different machine learning model so to say but for the question is to detail but an intent if it happens later in the story can trigger a different response technically yes so you're right there technically you are completely correct so one example I have I have one example of this is something I made a demo for today actually it was part of my job so let's say I have a chat bot that has a countdown so the user says I want you to count and then the first response is 10 and then the user says ok and then the next response should be a 9 and then the user again says okay and the next response should be an 8 right that's something that could happen but now if the user were to say are you a bot then the correct response here would be to say yes and then a 7 and if I were to ask are you a both here then I should get yes and then a 6 so this context is handled by the policy system so to say but definitely I hope you also agree that depending on what happened in the beginning of the conversation at the end of the conversation I might need to give a different answer as well but the trouser does have mechanisms in place to deal with this a good question by the way um another thing that's useful to mention I suppose is we see a nice example of that here we it might be the case that we have a certain intent and then we have to say oh here's two actions in sequential that have to follow and that's also a put an option here it doesn't just have to be we say one thing it can also be here's like two actions and sequential and also after after we've just done this action we should always listen until the user says something back so listening is also an active actions that I say anyway back to the main topic we have these intents and we see that these intents are used as stories but that means that we have one last Lego brick to deal with and that is what about these actions we have to define those as well and typically those are stored in this domain but Yemma file and that's just a configuration file where you can say hey here's all of my intents and you're also able to say and here's all these actions and responses typically a lot of these responses can be simple for example just say a little bit of text if you're making a frequently asked questions chatbot then usually just sending some text back can be fine but we are also able to add actions in here that are just Python code so then you can customize whatever response you would like an action could be some Python code that fetches some information from a database etc but we have to declare our intents and our responses and also some entities in this domain at yellow file that's what that config file is for there's one final configuration file and that is aptly named config tamil and the idea behind that file is that that is where you define your machine learning pipeline now we have a yellow file here because we think that's convenient but in the back all the code that you see here is just Python that's running you know some tensorflow code and in the case of this count vectorizer that Sai could learn but you can also add your own custom Python code in here if you want to experiment with your own methods that's completely possible but these are typically - for files that you have to specify in order to get a chatbot running and I hope that given the examples that I've shown you that it's plausible that we need more than there's a CSV file we need these settings files in order to get started a benefit of this is that we don't necessarily have to write a whole bunch of custom Python code to get started because these four files on their own will be able to cover a lot of ground for us so just a reminder we have four files that are disposal that we have to use the config that llamó file machine learning stuff domain that yem oh that's the overview of all the intents and the responses and custom actions the stories are examples of LEGO bricks click together that represent conversations that could have happened in the past and the day of the NLU file that has all sorts of examples of intents and entities that are meant to sort of give training data towards a intent you so just to quickly check Christiaan any questions on this anything unclear and I was wondering if you um intent detection is a classification problem you teach by example in that case you need to provide enough examples to detect this particular intent is that right so you're completely right there the question is I it is hard for me to know when I've given you enough data and in general at least in practice I can make this argument let's say that this is all the data that we train on and it means that those are all the examples we've given to our system and let's say that these are all the conversations that users actually end up having with our chat BOTS and there's the the main problem I think in production is that you usually come up with paths and stories that you think users are gonna have then when you put it into production you realize actual users actually behave a little bit differently but you are completely correct in the sense that it helps if you have this panel you file that you definitely supply with more examples note though that we do have word embeddings at our disposal as well and the way the way to improve is provide examples there's nothing about syntax or keywords or you just well I mean there's also this notion of conversational design right so you as a programmer get to decide when am I gonna use entities when are intense enough and how our user supposed to interact so I don't want to discount the fact that conversational design definitely matters here but in general you're certainly correct the more valid examples that you have of training data the better the only thing and we'll see that in a moment is that it is a good idea to come up with training data that covers a lot of ground I can come up with 10 examples which are all variations of spelling for the word hello but it would also be good to say greetings and hat you know come up with a little bit of ground covering data set so to say so the ideal case would be your examples are everything that could be said everything every way that this could be for me in terms of machine learning training performance I would argue yes but it's not gonna happen in reality and also I mean we should also be aware that the more data we give this thing training times are gonna increase and we will get a limit there at some point as well and we can use GPUs of Rous I mean all that stuff works but in practice my advice is to give this to users as quickly as possible because in my experience so far no chatbot goes as planned in the sense that you cannot predict what people give to these chatbots some pure gibberish in practice how long do these files need to be how many examples do you need until you see some good results what what might be better in that sense is how about we just make a chatbot right now well just start locally and my goal will be to fool the chat BOTS we're gonna see if we can break it and we'll find out it's relatively easy but but it will but one thing I have to warn you for it is a bit of a cat-and-mouse game and that's mainly because I don't know exactly how my users are gonna end up using my app and if the distribution of stuff I get in production is different than what I have in training and you're gonna be in a tough spot and I would argue that that's probably the main concern there is a question here in the chat how does any human human of conversations as reinforcement modify any config or domain or override it this is a question that's in the chat as well so can i where's that button it's normally on your thought oh wait I think yes I'm using two screens here and I think I can click more and then chat and okay but it's fun it's grand how does actual human human conversation has been forcefully modify and configure the man so we don't actually the the idea is because we have this idea of a story we do constrain the algorithm a little bit because we can say these are all the intents that you're able to detect and also these are the only responses you're able to give back so we're not using actual human to human conversations here now there are systems that do that and the hugging face system that I showed you earlier something in that realm but for now I think the state yard is not in the position where I would comfortably put something like that in production the main reason being if your corpus has swear words in it or chatbots kind of swearing and typically when you put something in production you constrain the artificial stupidity more than you want immediately optimize all the artificial intelligence you've learned from Microsoft a that that is a industry reference yes there's another example which I thought was interesting you can google the photo it's pretty funny so I think this was a photo taker and from the earliest chat BOTS at PayPal and then a chat bot says hey I'm still learning but you can ask me anything and I'll try to help you and then the person replies by saying oh I got scammed and then the above reply great which you know is some ideal response but that totally there's a photo of that mmm but again let's very quickly just go into this demo because I think it's gonna give a couple of nice examples - so we've just done rasa installed rasa and one nice thing is that that gives us the rasa Camaro blank app now this one outside currently and we're trying to fix this if you try to get like the help from rasa here it's gonna be a little slow because there's all sorts of tensorflow libraries that have to run in the backend of it so instead of doing the help of showing you everything command line can do how about I just run the init command and what this command is gonna do is it's gonna initialize the project so that means that we're just gonna get a clean starter kit to play with now again this is gonna be a little bit slow as still tension flow has to be imported and loaded and all of that stuff but I think in a couple of seconds we should see something of a prompt that says oh you want to start a project select the folder - you're interested in yep there we go let's make this just a little bit bigger depends on what like I think most people should see at least something like this just checking people are noting but I think it's fine so you have to give her the path where the project will be created and I think the current directory is fine because again I'm doing this in the demo demo demo folder it's asking it's not empty is it fine I think it is so I'll just say yes and the next thing that it's asking me is if I want to train an initial model and I do so what's happened now is a bunch of files appeared and we're now also going to say use those files to train an actual chat bot it's gonna be super basic but we're just gonna go ahead and train this and what you should see now is you should if you if you used to care us you should see some things that you might be familiar with like this epoch thing you might have seen that before if you're using Karis but essentially here you're seeing that we're training a bunch of systems okay so some more stuff is training does a GPU come in handy so I the thing is probably it's just like enough I've never really needed the GPU myself and I will say that that's a benefit but like my I have colleagues in the research team who do use their GPU the thing is that we are currently trying to figure out what's the best way to benchmark all with this because there are parts because there's bottlenecks etc it's a bit of a detail but do you there are speed ups that you can achieve using GPU most of the stuff that we're using is based off of transformers so we're not using LS CMS as much anymore in our stack so that seems that the improvement that you should be able to get our should be reasonable anyway you should now see something like this where hey yeah we've trained an assistant on the command line so we could go ahead and sort of start talking to it so I'm gonna start talking to it and everyone's allowed to sort of do the same thing I'll zoom in just a little bit okay so this is the demo of a very very basic chat bot but what I am able to do now is I'm able to say hello it's asking me hey how are you I can say I am sad and then it's telling me that there's something to cheer me up so let's let's put that mysterious link in a browser and see what happens well it's a cute picture of a cat so that's always nice did that help you I can say no and then the chat bot says goodbye again I'm what's I'm not suggesting that is the best assistant ever but this is the basic demo now yeah so I think we have a little bit we have enough time for that so this is the basic thing it just got running so what I'm just gonna go ahead and do now is I'm gonna ctrl C out of this and I'm going to just open this folder up in visual coach studio and there is this nifty little shorthand command that you can type called code which will open up the app but you can also just point to the folder and I'm just gonna work from here for now because from visual studio code I also have this terminal my disposal now you can just source the virtual environment here and then this will be a good place for me to play around and then demo some stuff I suppose so what are we gonna go ahead and do is I'm gonna go to this data folder and I'm gonna open the NLU 2 MB file and you should recognize a few things like we have this intent greet which is detecting the greeting for example and we also have this intent goodbye as well as a firm of an eye a mood great and the mood unhappy and these are all in here and there's also an intent for challenging whether or not you are a bot which is a really good intent to actually implement but but these are sort of the intense and I hope we recognize that from the slides I showed you earlier we can also see that there are some stories and again we see that there's the greeting which again is the fine over here and then we have a greeting we have a response we have the mood great we have the response and there's a couple of paths here now there's also this domain a yellow file let's just quickly have a look at that and we see a list of all the intents but the main thing at least to us is we also see these responses so utter greets for example that's actually listed in here and there's the utter cheer up where we get some text as well as an image and so I hope we recognize some of the separate parts that we've talked about earlier and what we can also see here is that we have this config dot yellow file and again these are all these settings for the machine learning stuff that we're running we're not gonna really change this as much but I figured it'd be good to point out that these are here and you can also point to your own custom Python classes from here that's also totally possible what I'm not gonna go ahead and do is I am just gonna go ahead and type rasa shell what rasa shelled am I going too fast by the way just because we have these files but now I want to show you hey let's see the robot fail and then we'll see if he can fix it so in particular what I would like to do now is I'll type the rasa shell and rasa shell is just gonna give us the interface that we had before so nothing new per se takes a while let's get started but once it's there yes so I can type hello here and we get the hey how are you and I call it a type hi and I can also say yo and I'm gonna I would argue that this is a very clear failure alright so I type yo and it thinks that I have to say goodbye and the question kind of is well why is that all for starters yo is not listed here as one of the possible intents so there you kind of go it's probably this count factor that hasn't recognized and also what we're doing here is we are so far only using if you look at the config file the only real features that we're using here are count vectors so we're not using word embeddings or anything like that yo probably would have been picked up correctly if we're using word embeddings but we're not just one question at this point yeah does it always output an intent or is there a kind of a default or failure mode if it doesn't clearly detect there are settings that you can tweak for this and all um so there's two ways of dealing with this we give you the option of saying hey well first of all let's have an intense for out of scope that's one thing that we can do and it's actually not the worst idea so suppose you have a chat bot for pizzas if someone starts talking about fire hydrants that would be an excellent opportunity to say hey let's fall back so you can have the fall back intent that's one thing that you can do the other thing that you can do is we have this idea like all these models they have some sort of probability that they output some confidence level it always sums up to one and we have this fallback policy where we can say if the difference between the top two if the difference between that probability is really slim okay we need to fall back and ask the user to sort of repeats themselves because that way we might get a sentence that's easier to detect so then I was wondering where the buy comes from when you type in you like does it automatically go to the most probable class or does it go to the goodbye intent automatically so let's have a look at that because what I can do is I can also type rasa shell and all you and that's gonna give us a little bit more insight and what the model output is and we're just we're just gonna investigate this for now because I I agree with your curiosity now one thing about this mlu is that this is going to completely forgo the order in which the conversation happens these are just the classifiers for the intents and the entities that we're going to be looking at but for debugging purposes we are going to get more information so let's start by saying hello again and now we get this big Jason thing we can stare into and what you can see here is that yeah definitely greet that's that's the most the highest one but now let's say yo didn't just see what happens yeah so it's definitely confused here and if I if I were able to hypothesize what is going wrong here we are using count vectorizer x' and we're looking at character engrams as well and if I think of the word yo right this is a y in there and then always well so I can imagine it eventually probablistic ly that's the reason why we're seeing this so what's the most probable class oh yeah that's that's certainly true and the one thing that would be nice now though isn't and that's something you know if you see this in production you would label this and what we can do now is we can just add the word yo here and the nice thing about this is this gave us an opportunity to call rasa train once again and and at this point you know this is a really minor change but one thing that I do think is kind of nice about this tax is know towered sort of looking at this file as if it's a config file and if you think about machine learning pipelines I didn't think there's something elegant about your data actually being a config file as something that you actively maintain as such so in that sense it's it might be uncommon to write your data in this format but I hope that we also recognize that Mary love it now one thing that just happens because we just retrained the new model if you pop open this models folder you will notice that there's a new file here with a date/time stamp in the name and that's because we just trained the new model and every time you call rasa shell well we are doing internally is just grabbing the most recent one but you can also select a flag that says open up another one but note that this tar over here you can upload this and set that into production this is just everything that we've trained and zipped but now if we go to the router shell and we do the whole yo thing again then we should see an improvement okay so I'll do yo now and it's now being detected correctly and again this is a little bit cheating of course because the example is literally in the training data at this point in time but I hope that people recognize like oh yeah we just trained a new model of the new model appeared in this models folder and that's picked up by Raza failure so what I'm hoping is that at this phase people recognize that if they want to add an intent or more examples that they have a feeling on how to do that I mean sure we haven't shown you two steps yet but if you want to add examples you just add another row here in this and all you dub MD file if you want to add an intense we can just copy this and give it a new name here and fill in these examples well we don't have to do is we have to add a story here and then we have to make sure it's in this domain at Gemmell and what I could do now I could do it a demo where I take all of those steps and I showed them to you but I think that you know that these are not the hardest things to do and I imagine and at least I imagined that were a little bit more interested in seeing how entities work and how we can get our custom Python code running as well so what I would propose we do now is just half the little break like a couple of minutes and then what I'll do is I'll share a git repository with you that contains a pokeymon chat bot that I am building that is meant as an open source example but that has a custom Python component in it and it also has examples of entities that we can go ahead and play with as well very good do we need to know anything about Pokemon to follow or just barely if for all intents and purposes we all agree it's some Japanese kids show that's five years like I'm not the biggest Pokemon expert I get corrected all the time on Twitter like all the time but first I just want to make sure that the people's questions are answered at this phase I think this is a nice little milestone in that sense so give me all your questions and if you want to grab a glass of water go ahead and someone says pika pika and that's okay okay and pokemons go that's good to know so there was a question by prom night is there a way to programmatically copulate in tents there yes so definitely then there's even packages that do this I I I do want to mention that you might want to be just a little bit careful there so the idea is that you might be able to say well it's full on I'm gonna come up with all sorts of questions about Pokemon so you can imagine I have what pokemon has this type and then the type is something I can loop over maybe I can replace the word wats with which and I can generate all sorts of training examples there the analogy here is that this is reasonably similar to deep learning and images where I take an image of someone's face and you rotate that the idea behind it is okay but if it's one person's image and just rotating it 10,000 times we shouldn't think that that's 10,000 data points and and with that in mind it might actually be a better idea to just sit down with a bunch of people and actually grab a thesaurus to make sure you're covering a lot of ground instead of just repeating the same data over and over again that's a risk that we have seen as clients one thing that's interesting about this particular problem is that one of our researchers actively working on this we are trying to figure out if we can come up with generative algorithms where you give us five sentences and we'll generate twenty that are similar you can select three did you like those are now all in the generative parts and then we generate more of them and we think that that might actually be a pretty valid use case for generative algorithms because we're using that to generate training data that you yourself still label this is an active research project but if people are interested we do have collab notebooks where people can try this out themselves as you might have noticed we're a pretty open company when it comes to research many more questions in a chart please Camila this is there was just one question now when talking about entities is it mandatory to be taxed or it can be numbers for example I create story that asked how many tickets and user answers integral three instead words how has a solve this problem it's from Murali yeah now so we have tools for that so a couple of ways we have this I think it's called duckling if I'm not mistaking so we have this duckling entity extractor and duckling is just there's a bunch of heuristics in there and they're really good for getting like numbers and monetary amounts and then that sort of thing so if you want to fetch all the dollars out that's why I'm way of doing it and you can also select entities by giving us a red X that's also something that we can use that it's X enemies and recently this is a feature that we released I think three weeks ago maybe two we are now also making roles and groups for entities so you can say hey this is the first dollar amount in this utterance and this is the second dollar amount that I've seen the idea here being if suppose you're making a chaplet for an airline then you have two cities but one of them is a destination and the other one is where your is the origin so the entities that we train for can also have that context attached as well this is a new feature I should add to that but definitely if you're more interested in just getting numbers out we have some you have some components that are specialized towards doing just that so what I'm not gonna do is go to github and I think github slash rasa HQ is what you're gonna want to have and then CR opens projects here but there's like rasa rasa del Rosso SDK there's a bunch of them what you cannot type in here though is pokedex and I think if you type in pokedex - demo browser into Google you should also get here then but just to reiterate get up calm slash rasa HQ and then there is this pokedex demo that you can find and well I'm just gonna go ahead and do is I'm gonna clone this a cheesy HTTP but for my co-pilot you probably want this with SSH instead because it will be nice you can make a merge request in a bit and what I'm just gonna go ahead and do what's used those to do cut for that go back here I'm just gonna go ahead and do is I'm just gonna do the quick and dirty thing I was gonna make you're gonna clone rectly in this folder here and it might be better to move that to a separate repo that's that's definitely valid but I'm just gonna make the Pokedex demo pair in the folder I felt over here and any reason for is I then have access to the virtual environments but you can definitely put this in a Perl or fuller as well nice and what I'm then gonna go ahead and do gonna open the Pokedex demo in a new window for Visual Studio code not save anything they've done here and what you should see is some it looks really similar to the rasa init program it's just that here we have different examples in our NLU dat MD file and there's also this Pokemon db2 JSON file over here but this is in all intents and purposes still more similar to Raza and NIT than than anything else but one of the things that you're going to notice here is that there's a couple of different intents in particular there's one here called demand joke so one of the features that I've made is I've made this intent where the user says that they want to learn hey they want to hear a Pokemon based joke so that's an intent that I have added and you'll notice that I've actually come up with a bunch of ways to ask for a joke here another thing that I've done below here is I've got this intent but there's also some entities here so for example I could have an intense where I'm asking the chat BOTS could you confirm if this pokemons name exists that's technically exactly what the Pokedex is supposed to do so you can say hey is Bulbasaur a Pokemon and in this case with this syntax we're saying bill bazaar this string that is an entity of type Pokemon name but the name of the Pokemon so to say and in this particular case because there's lots and lots of these Pokemon I'm probably not gonna have an example for each and every single one of them so we have these utilities like a lookup table so in this particular case I can say hey here's all the pokemons names and then Raza is able to figure out alright so then I can replace whatever pokemons name is listed in here and I can put it there generate some data and that way I don't have to come up with a every single Pokemon that I have and and the reason why I like showing this is Raza has a couple of these utilities so to say they kind of make it easier for you to declare a whole bunch of these entities so you can also come up with things like synonyms so you can say NYC instead of New York City and that's also something that you can declare this DLS dsl star the same to occur it still needs to train with all of these generated examples it's just a way of automating the creation of training data like yeah that's it's it's it's just so that you as a developer have less cognitive load on this yes and also the way that we generate these training examples is we look at this story then we say oh we had a greeting here okay we're gonna grab a greeting okay this one and then we're had a we had an action and then we're gonna take another example of this and that's how this training data is generated on your behalf but we automate that such that you don't really have to bother with that that much but you can see here that definitely I have some more intense and I even have an entity here at the bottom so that that's that's something that that's relatively exciting so let's have a so we've had a look at this an alert and B file so the next thing that you might want to have a peek at are these stories but the stories files not particularly interesting the main thing that is interesting is that we are using our new demand joke over here and we're saying hey let's uh turret joke and then we say over here that we have a confirm exists intent that is followed by an action check existence and one thing that is interesting is that action here it starts with the word action instead of the word utter so it isn't a different type of action in that sense it has a different name and there's a reason for it if you want to discover the reason why part of that can be found in this domain that yellow file yeah we've all of our actions and again we see that one action sort of stands out let's check existence one we also see that the intents are definitely in here we see that the entities is also listed here so we've got this entity that the main files now also aware of but then if we look at the responses greeting cheering-up we have a lots of jokes and our disposal here but the one thing that's not in here is this action check existence that's the one thing that we don't have and that is because this is a custom action we need custom Python code to be able to handle that and that also means that we cannot just have some text that we utter we actually have to write that down in a file somewhere and if you were now to you know check that later but if you were now to go to actions that PI you will see a Python file that has a class implemented and this this class is called action check existence and it inherits from the action class that comes from the Raza software development kit and in particular this class has a method called name and the name of this action is action check existence so this is how you make the link to the domain file that this is indeed a custom Python code that's supposed to run and there's one method in particular that where all the logic essentially gets handled and it's we might get into the details of this at the end but the main thing that's important is that we can we can and we have access to this tracker and this tracker has all every everything that happened in the conversation so far and in particular it has this one object called latest message which is a dictionary that has all sorts of information and in particular it can grab the entities out for us and what I can then do is I can say if that entity type is pokemons name what I can then do is I can check hey have I seen that name before in this pokemons the txt file if I have well then I can say yes it's a Pokemon and if I haven't and I can say I don't recognize the name and in this case I'm just reading it in from a file but in practice you can imagine you would read in from a database here so it's quite straightforward code right it doesn't check and all puts the message so I definitely have a world yeah do you do complex stuff in these actions like update some state so the thing is you could and one and there's one use case for example so and suppose I have the pizza-delivery chat bot thing so we have this notion of entities right those are sort of things that we can detect but maybe some of these entities are long-lived like your address for example is that's I don't want to get that from the last message that's something that's to be long-lived so you can imagine that if currently I don't have your address yet I might first check the database if I have it and if I have it there then I can fill in this thing called a slot and the slot in Raza is like an entity but it's long lived so so that could be a use case so to say but the goal of this is in this custom code so that means that you have the freedom to do whatever you think you might need want to scrape a website you can so what you don't do is write your own state machine maintain everything but you use the tools that rather provides like I mean there's custom Python code so if you want to go ahead and do that I'm not going to be able to stop you but I will gently urge you that there is a better way of going with so so state machines I get where the idea came from right but it's eventually it's harder to maintain that's sort of the idea behind this and the nice thing about these stories is that you know you can strain the chap out a nice way you can actually say oh this is a story that I assume happens and yeah I guess you can see some things like a state machine in here but I have found it to be an unusual reference there is a lot of literature on it though so I get the reference but it's I have found it it gets in the way mentally the better way to think about is hey a user has this intent now given this intent what's the next best action that's a good yeah it's it's kind of a UI thing I would argue almost for developers but but really good question well I hope people recognize from this though unless there's other questions something about the clarity of grammar and it feels a little bit off topic so I'll ignore that one for now but but what I think might be interesting for us to do is let's just run this custom action with a chat bot so far I think it might be a cool thing to do so well quickly now is all we'll need to start up two things actually so I'll get the terminal here and what I'm gonna do is I'm gonna start another terminal on the side as well so I've got these two terminals at my disposal now I will source the virtual environment first and foremost oops source them in oh wait those in the folder right this is kind of annoying because the virtual environment is in the folder in the parent folder okay I've got I've got my virtual environments here so that's good and what I'm gonna do first is just call Raza train from here because I need that model well there's a second thing that I need to do as well and that has to do with how Raza works so you can imagine that I have my natural language understanding part of Raza right so that's but something of Raza offers and then text goes in and then we we try to give you the right action to follow but then if we have our own custom actions right the question then is in production do we want to have those actions inside of this NLU or should we maybe consider that as a separate service now if I'm gonna run this in a docker container should these two live in the same container and in production we found out that just there's a lot of engineering reasons why I want to separate the two so that means that you have typically this NLU service running that will handle the you know the state of the user as well as what's the current action etc but then if a custom action needs to be called then we have this action service that we have to call and that has benefits because we can look at these logs separately and kind of in the micro service style of thinking we do want to have a separate box for this guy and that means that if I want to now you know play with these actions I should run my own action server to keep all that stuff simple though what you can just go ahead and do is you can type browser run actions and it will just pick up this action file and start a server locally for you if you want to do is more formally we could package it up as a docker container and open up the right port but for now this will just be easier so what I'm gonna do is in this terminal I'm gonna run browser run actions and that's gonna start a servers a service and I'm gonna type as a shell in this one and that's going to give us the old shell that we're used to and it should be able to pick up though that we have an action server running there's a config file that I've set in this project for that this takes a while to start up but the bots should now be loaded what I'll do is I'll just grab this and all you file so we have this example here and what I should be able to do now is I could say hey just hi how are you say give me a joke why did it squirrel across the road to get to the other tide well not the good another super good joke but okay the feature works but now what I can do is I can say is though thus our pokemons that's a question I can ask and we see that something over here happened and it's saying here yes Bulbasaur is a Pokemon so we can see that we have the natural language understanding saying hey a certain action needs to be triggered that message is passed to the action service and then we get a response back here and we see that we're also printing some extra information here any extra information that we're printing here by the way and that's information you have access to is all the confidences for all these different intents they're passed with the message that you get into the action server as well so this is a lot to suddenly see but I hope that we from a know if we take a step back the main thing that's just happened is we have this custom action over here that is able to trigger Python code that's basically what's happened and we give this message back by accessing this dispatcher and we're saying hey just send text back we can also send images I mean that's also something that we can send this way we depending on what platform we are launching this on we can also send buttons and then all sorts of forms etc but we do have custom Python code handling some logic for us and I can do something like is boo Babar a Pokemon and it's still not perfect because in this case it's unfortunately detected that we want to have a joke right so we still need to add examples this this Chapel is not perfect but I think if I have the example is nine tails right then we trigger the action server again because this is misspelled they show again the command that we need to run the actions yes sorry if I'm sorry I went too fast razza run actions and that command will assume that your custom Python code is in an actions the PI file so that that's the one assumption that that thing makes but this should start the action server locally so those that are interested there is this extra yellow file that I've not really discussed but it's called endpoints that yamo and you can specify hey the action endpoint that I have for this that's over here so that's because and that's how it's able to find the action server it's basically something that you can configure I'm currently running some experiments to see if we can also run the action server in a cloud function that's something that you could potentially also do here but in production this would basically points to a docker container somewhere it's definitely not perfect but but I do hope that we see like how the separate parts work together any quick questions on this also from the chat Mary guys are you still with us okay I just yeah we have some yes yeah yes we have some years on the chat people listen maybe you can explain to me what's happening so I typed in the only thing I know the answer is Picasso a Pokemon and the answer was I do not recognize Pikachu are you sure this correctly I'm not sure Pikachu is spelled with a k did you spelled this way yeah this is how you spelled it or you spelled differently okay so you type is Pikachu a Pokemon like this with a question mark and pika yeah so I think it's the capitalization of Pikachu because it's really good but if the action function that's doing a bit of a mess up I think one way to fix that I think is if I look at the entity yeah so I should called up lower there I think that's the the fix or maybe put it into pre-processing could your car what do you mean by that exactly I mean if you want to ignore capitalization in general could you define it somewhere the pre process is everything in this case I would argue all users are able to capitalize stuff to you right so that's something that you should be a little bit mindful of technically what you can definitely do is write your own tokenizer that makes everything lowercase actually come to think of it I think that's actually a setting of the white cat white space tokenizer I think you're able to say hey I want you all the tokens to be lowercase I would maybe not do that because you do lose information that way but you can if you want to okay I think this is still a pretty good practice though we've got some questions in the chat mm-hmm is Rosa a Pokemon why there's no trigger so let's there's a wall so there's a couple of things happening here in the background so what I'm going to do now is I'm going to hypothesize what is happening here but it's there's because there's multiple parts I cannot suggest that I'm 100% accurate here so it's saying here I absolutely love Pokemon so then the question is what action was triggered and if I go to my domain but Yama file I think this is a joke in fact I'm sure it's a joke it's listed here and the reason why it's a joke is because absol is a Pokemon this this joke in particular was a PR from New Zealand I can can I remember that so the thing that happened here was other joke was sort of mentioned here so then the question is okay how can that be well I'm using the word Pokemon here and I can imagine that the word Pokemon also appears if I go to the NLU file over here so the word Pokemon does make a couple of appearances in this intent over here and you know I have three examples over here with Demand joke and I only have one example here and so that already is kind of a thing that can go wrong especially if you further consider that it's not just the word that I'm detecting it's not just a count vector for that word it's also for all the little combinations of characters inside of it as well and I'm also generating lots and lots and lots of features so my guess is is that that is where something is going wrong and the easiest way to fix that is just make sure that I've got more examples over here this feels like a data problem I would say now one thing I suppose it is interesting I what I can do here is I can say is Raza Kokomo named a Pokemon name that exists now I can I'd this example and in this particular case what is interesting is that Raza does not exist as a Pokemon name but in the context of this sentence I might be able to argue that this token over here needs to be picked up and checked by our action as if it is a pokemons name so in that sense I still want this to be detected as the entity even though it doesn't exist as an actual Pokemon name that's also something for this particular use case I suppose to consider incredible hypothesis thank you so at this point in time there's a couple of things that I think might be interesting or fun to do so one thing that we could that the one thing that is valid I think at this point is let's add some data and it can be like a nice PR that's something that we can do but if we're gonna add more data I think I want to show you a good habit you got a definitely we can add like yo here right I mean that's something that we could - but when I think about Raza and ways that me and the research team can help some of you folks sure we make algorithms but some of the things that we're also now open sourcing are data sets and I just want to highlight that because I do think that's probably the most valid way to get started so if you go to Raza HQ and I think if you do like NLU yes so if you if you in github type Enel you in the repository search you will find this NLU - training data repository and the idea behind this repository is is that we want to crowdsource some datasets such that you have an easier time coming up with good models so we have some common things that people in banking like to use we have some custom scripts that will remove entities and that's something you're worried about but in our particular case we also have this small talk file and I'll just open up the raw file and we just have a couple of examples listed here that you can just go ahead and copy and paste and this is something that's also a community project this is something that we're actively building but we do think and we seriously think that having a good data set to start with is probably your primary concern in the beginning you know we can have very fancy transformer architectures etc but if the data set is not representative of something that users are actually going to type then you know you're gonna be in a bad spot by your way so if you're gonna get started with the chatbots remember that we have these repositories with datasets you can copy and paste from so that's the first sort of hint and tippet I might want to give you and there's actually a second hint I think I should be giving as well and to do that what I will need to do is share this browser tab because so far I think I've given like the hello world like I've shown you the features of razza etc but I haven't really talked about yet is how do you actually do this stuff in practice because you know there's this deployment story but there's also this data quality issue that I think is at the center of all of this and for that I just want to show you this other products that we've made called rasa X and the idea behind Rose X it's free so you can go ahead and download it and use it but it's not open source because we don't want Amazon to host this on behalf of themselves we are more than happy that you host this for your chat box but we're less happy if you're gonna host this product and charge other people money for it but if you're using your own chat pod you can go ahead and stalls on the VM and what this essentially is is a user interface on top of all the files that I have shown you and we have you know some sentence that was spoken and then we can say hey is this the prayer crew I predicted correctly and I can say yes or no and this is sort of a labeling tool that we can use for that there's a nice little train button here so I can retrain all of this stuff and in particular the coolest feature and I think that's also something I would like to remind everyone this is why you want to install razza X is I have this button over here that allows me to share this chat box with other people so I can say something like demo people please play around and this will then generate a link for me and I apologize it's an HTTP link North HTTP but this allows me to give to you a link to a chat bot made live rasa everyone in the meeting and you should be able to interact with that so if I go to this other browser window here assuming nothing is going wrong because I am using a experimental version right so I should be able to type hello in here and then we have a very very bare-bones chat bot and the main thing that this chat bot does is it asks you the user what features you would like to see in this chat bot and the reason why I think that's an interesting way of talking about it is because I don't know what Pokemon people like like literally no idea I played the game when I was 10 but what people can now do is they can say well I want to learn the Team Rocket theme song or something like that and then says okay is there anything else I can say yes simulations will also be kind of nice no and then I get a couple of these responses but the nice thing is is that while that's happening yep here's the conversation I just had so this allows me to do some user tests actually get this out there and have people interact and play with it and the nice thing about this rasa X instance is that everything that I label there's a nice little git link so I can click a button that will update the get surfer on github and and I also can train from here and we like to think that this is sort of from user interface and maintainability perspective probably something that you want to get started with after you've actually done everything on your laptop if you want to get anywhere near production there's the first thing that you should probably install so that's the whole demo that I had in mind at least so I hope people sort of look at browser X and go alright if I if I want to do something in production this rasa X is something I should keep in mind because I do think this labeling and keeping the data quality high that's super important but until then I hope that I have given people a nice introduction on what we think are good building blocks for making conversational instances and in sort of apps what what I think is nice about this approach is that we have this nice little middle road between high tech algorithms but still some constraints such that the algorithm doesn't derail into something that we don't want it to so that is something that I think is going to be really valuable if you want to do this stuff actually in production and then I people are interested definitely go to the get up thing and add like some some intents maybe or like maybe some extra examples it should be relatively easy for you to do so but I think watching other people merge that p.r might be a little bit boring so probably now is a good time to do questions instead thank you so much listen I have a question right away speaking about building blocks yes we've seen it's a bit of a process to get the right behavior into your chatbot you need to do a bit of engineering but hopefully everyone will now use Raza and start building better and better models and there will be a model that's pretty good as a small talk and chitchat and if I want that behavior plus some other behavior plus some custom behavior that I built is there an easy way to like combine behaviors abilities from chat pods so in an extreme case you could say oh we have one action server and that decides whether or not like extreme scenario let's say you have multi language we're German in English let's say those are the two things we need to have for this banking app you could take this super extreme route and say oh you know what I'm gonna have one action server that's going to determine from there on which of these two rouser instances you're going to talk to and that's something that we can build out that way like the the the thing here is is because it's all Python we part of that responsibility lies with you like you have to seriously think how am I gonna maintain this the main thing that we try to do is come up with stable building blocks such that a lot of use cases are properly covered so I'm just gonna keep on with the multi-language use case but I think it's interesting to most people so one of the things that we allow is we allow you to use multi-language multilingual word embeddings like Bertie have a couple of multi-language versions of that but but there is whole notion of conversational design that I do want to admit is still an unsolved problem this is all relatively new stuff so we are trying our best to learn it quickly as possible and come up to like good practices but I have to be honest here like we also learn a lot from our clients like they have datasets that we don't have access to so the lessons that they learn we know we have to get it in and then that sense we're still learning as we're going along we'd like to think we've got a couple of good habits but for example multi-language chat bolts that definitely still work progress we're not done there and the way I'm not yet at the point where I can say okay give me this small talk ability and then give me the Pokemon ability and I just plug into the other into and it's still my job to integrate everything to some extent we we like the thing that that's where the data sets will help you because again we the models that we make are this finds to be really general and he gets yes pretty state-of-the-art results and that's once we think that for small talk datasets are probably going to be more effective than anything else just like to thank you again if we're lucky it's a matter of merging the datasets [Music] pretty much yeah well so you still need to map that to actions and intense and whatnot but let me the example FAQ but that's a relatively standard thing because it's it's pretty much in action tent action there's no other sequence of actions that you have to follow strictly so we have a couple of these demo boss said you can't go ahead and just put if copy and then just your own data set and for chitchat in particular you have a little bit for data sets you can also just go ahead and copy but I do fundamentally think that a large chunk of making general that bots is just having the mineral data sets oh definitely a ball on our scientist to have like a standardized data set for chitchat and German chitchat in Dutch it in whatever and then by giving you the data set you should already cover the ground a bit they're questions that they want to take them directly from the chat why should I read them to you this I think I think I could take this one could you show a bit more about Raza X how would you take a demo conversation I used for training so super quick demo and I should I should mention here this is not the most recent version a more recent version of Raja X also has like a keyboard shortcuts that I can't demo right now but if I were to go to L your training LS where is Pikachu and if that's that someone said that as a the things that will suggest feature intents and I want to learn a cats and ng simulations and I want to learn a team rocket's theme song these are some of my responses but I think other people saw mentioned that and the models is hey that's a suggest feature and I can say yeah that's that correct and then someone here says please tell me a joke and the chat BOTS thought it was goodbye I can say nope that's bad and I can say that should have been a joke yeah mark is correct now okay so you do this you you label a couple of things and every time you do this there's gonna be a new line appear in that Enel you end the file that's something that's happening under the hood well while you're doing this and if I were then to click this button a new model gets trained and then I have this models tab where I can select which of the models is active so that's something you can all do from here essentially all the commands that we've done from the command line we have also made a user interface for that with the idea that you know it we can extend this where maybe you have multiple people that can do labeling and maybe we can say things like only when two people agree do we actually put the label on disk and these are features that we are considering putting in this and we think that having a tool like this is going to make it easier if you to maintain and update and keep your model in the loop so to say definitely a lot of work in progress if you get the newest version you should get the keyboard shortcuts but that's the flow typically that you would get another way of saying this is some of you might have heard this tool called prodigy the the Spacey people are the explosion AI there they have this tool where you can also do some labeling and this is in a way like prodigy but we like the thing that this is specialized towards the digital assistant chatbot use case but that's why we see this going that for chat bosses is something that's going to be super specialized and to that