Today, we continue refactoring the chatbot project!
Hello everyone! In the previous post, we spent a lot of time refactoring our third-party integration libraries. Feel free to check it out if you missed it! Today, we continue refactoring the chatbot project, but this time we focus on the web application structure as well as the main logic of the bot that involves all the reactions to the user input. In the end, I present the new goals for this project and give you a small teaser about what's to follow.
Restructuring the web application
Before we start, in case you want to follow along and have the whole picture in place:
If you read the previous post, you might remember how the index.js looked like, nonetheless here it is. Quite messy!
We split the index.js into two files. First, app.js is responsible for packaging all elements of the application and exposing a factory function. Also, main.js is creating the application object, bootstrapping the Telegram webhook, and starting the server.
You would think that main.js should be straightforward, right? Which is true, so let's start there.
The main.js file is responsible for three things:
First, it creates the Telegram webhook, this call is idempotent, so in case the webhook already exists, it won't have any effect; if it doesn't exist, it will create the webhook. We keep the bot token in an environmental variable so we can have different bots for the same codebase, and because we should not expose this information in our code. The same applies to the webhook URL since it depends on where we deploy our bot.
Second, it imports the application factory function and creates an application object. In this place, we could also set different environmental variables or pass a configuration in the application factory.
Finally, it starts the express server.
Now let's move on to the app.js, which is also simple, but we have some things to discuss.
We still have all the Express application logic in one file, routes, controllers, middleware, and bootstrapping. Currently, I think this is ok since the web app is trivial. However, I will change it in the upcoming posts when I add more functionality and new endpoints. Usually, I would split the logic into modules where each module has its middleware, controller, and service files; or if we have a smaller application, we could go for a structure where we have separate folders for controllers, middleware, services. However, I find the second approach very confusing especially, when a project starts to grow because you cannot differentiate the logic of different modules easily.
One more thing I want to bring up is the separation of the bot's core business logic and the Express server, which is the way we expose the bot logic. However, we could use a different API to the bot logic, for example, a CLI. So in the next step, I remove the reaction.js from the app/ folder, and I rename the app/ folder to rest-api/. Moreover, I create a new bot-core folder where I put all the core logic of the bot, which is only the reactions.js file, for now. So the new project structure looks like this:
Restructuring the bot core
Currently, all the core logic of the bot lies in the reactions.js file. The logic is very primitive. It's the function reactToUserMessage that takes as input the user text and metadata and then reacts to this input. You can find the file below.
What a monstrosity, right? Just a bunch of if statements, and that's it—a ton of duplication and not scalable at all. Imagine having a thousand reactions. You might also have noticed, in the first if condition, where I do an extra check for the user name. Imagine the mess we will have in our hands with keywords triggering different reactions for different users or different chats. We would have a ton of nested ifs on top of this. So how do we fix this mess?
First of all, we need to approach the problem from the definition of a reaction. What is a reaction, and how could we model it? My first approach to model it looked like this:
We have a keyword that is what we are looking for in the user's text; then, we have a match property to define if we want the incoming message to match precisely the keyword, or we just want to include it. Next, we have the action property that should be a function name we want to execute along with the arguments to pass in the payload sub-property. Finally, I added a users and chats arrays to be able to allow some reactions only for specific users and chats.
With this in mind, I do the following refactoring in the reactions.js file; I also rename the file to bot.js:
If you read my previous post, you might remember that I removed a lot of logic from the libraries and said that I would move them in the core logic of the bot. You can see this now; all the formulation of data we get from third parties and then sending the data through a Telegram message happens in dedicated functions. I omitted the Imgur-related functionality to keep it short. Finally, I wrapped all the core bot logic in a Bot class.
Moreover, I created a factory function that setups the bot in bot-core/index:
The web application will use this function to bootstrap the bot without the need to care about what to inject in the Bot constructor.
Even though things look much better now, I spot one more problem. The reactionsRepository in the reactToUserMessage function is supposed to be able to find a reaction by the provided user input text. Such complex logic should not happen in the repository of an entity. Thinking about it a bit more, I conclude that the Reaction is trying to model too many things. We try to model the intention of the user's incoming text and, on top of that, the action that the bot should take in response to this input. Moreover, what happens if the bot starts being proactive? For example, imagine we have a task scheduler, and then the bot acts on its own will and not as a response to user input. In this case, we have to remodel the actions the bot takes in a different context. Then the two models will overlap; this is bad. So I think it is better if we split the data model into two entities the Action, which models the possible actions the bot can take, and the Intension which models the intension of the user. Then we derive the intention from the user input, and we map each Intention to a specific Action; these models look like this:
So first of all, we rename reactions to actions. Then we need to create an IntentionService that we inject in the bot. This service encapsulates all the logic that is related to deciding what the intention of the user is and what is the respective action the bot will take. At this point, I also think that it might be a good idea to encapsulate the actions logic in one service. But one step at a time. Below is the new IntentionService, and the mock IntentionRepository. I also created an index.js file in the intentions module that exposes a factory function. This way the consumer of this module does not need to inject all dependencies.
Next, we remove all the actions from the bot and move them to a separate actions module. We do this to keep the same level of abstraction in the Bot class. Same as above, we create an ActionService, ActionRepository, and an index.js file for the service creation.
Below is how the Bot class looks after extraction:
And the bot factory:
At last, this is the final structure of the project:
You can find the final version of the project at this link.
In this post, we finalized the project refactoring. We restructured the web application and decoupled it from the core business logic of the bot. Afterward, we decomposed the core logic of the chatbot into its essential parts, created dedicated modules for these parts, and re-assembled it. However, we can still do many things to improve it. In the upcoming posts, I will aim to do some of the following:
Migrate the project to TypeScript
Add proper logging functionality
Write unit tests for the most critical parts of the project
Create end-to-end tests and setup Continuous Integration
Create a data model and add a database
Create scheduled tasks, so the bot executes actions based on a schedule you provide.
Create REST endpoints that will enable CRUD operations that will allow an Admin to create and modify bot actions and intentions.
Create a simple Single Page Application for an Admin dashboard.
I hope you found this post informative! Stay tuned!
Subscribe to Backend Definite
Don’t miss out on the latest issues. Sign up now to get access to the library of members-only issues.