Natural Language Understanding

Leverage NLU Technologies

When it comes to building a bot for open questions, e.g., FAQ of customer support, or an in-bot search engine, you may think of making a bot with the NLU engine to analyze the user's input. The primary mission of NLU is to analyze the user's intent and entities to respond with the right answer.

A well-trained NLU engine usually has a 90%+ accurate rate in terms of telling the right intent or find the user input is beyond the current knowledge base. Sometimes, it takes time and a few iterations to find the best intent structure and train the NLU engine smart.

In the following sections, you can see how to integrate Bottender with various modern NLU services:

• QnA Maker
• Dialogflow
• LUIS
• Rasa

Building with QnA Maker

The reason that we choose QnA Maker in the first place is because of the friendly building process. Unlike other NLU service requires a certain amount of time to build the intent and write the training phrases. In QnA Maker, you just need to copy & paster your FAQ, and the NLU engine is ready to use.

Note: Tweet us if you would like to share your QnA Maker engine performance with us :)

Step 1: QnA Maker Setup

To build a bot integrated with QnA Maker, you have to create the QnA Maker knowledge base and publish it following the Official Guide.

After you publish your knowledge base, you will get RESOURCE_NAME, KNOWLEDGE_BASE_ID, and ENDPOINT_KEY (See Here for detailed guide). Make sure you copy them into the .env file.

# .env

RESOURCE_NAME=
KNOWLEDGE_BASE_ID=
ENDPOINT_KEY=

Step 2: Connect Bottender with QnA Maker by bottender/qna-maker

To make the bot development enjoyable, we made a bottender/qna-maker package. You can install the package with npm or yarn.

With npm:

npm install @bottender/qna-maker

Or with yarn:

yarn add @bottender/qna-maker

In the following sample code, you can see how elegant it is to integrate Bottender with QnA Maker. All you need to do is to fill in your environment variables, and score thread, then Bottender uses answers from QnA Maker as the response.

const qnaMaker = require('@bottender/qna-maker');

async function Unknown(context) {
  await context.sendText('Sorry, I don’t know what you say.');
}

const QnaMaker = qnaMaker({
  resourceName: process.env.RESOURCE_NAME,
  knowledgeBaseId: process.env.KNOWLEDGE_BASE_ID,
  endpointKey: process.env.ENDPOINT_KEY,
  scoreThreshold: 70,
});

module.exports = async function App() {
  return chain([
    QnaMaker, //
    Unknown,
  ]);
};

Note:

• Fore the full example code, please refer to Bottender example, With QnA Maker.

Building with Dialogflow

Google creates Dialogflow. Since 2019 Google announced its NLU pre-training BERT, we are confident in Google's NLU solution; at least we can think it might be the state-of-the-art.

Step 1: Dialogflow Setup

To build a bot integrated with Dialogflow, you have to set up Dialogflow following the Dialogflow doc, Quickstart: Setup and fill in the two values into the .env file:

• GOOGLE_APPLICATION_CREDENTIALS, which is the file path of the JSON file that contains your service account key
• GOOGLE_APPLICATION_PROJECT_ID, which stands for the GCP project ID

# .env

GOOGLE_APPLICATION_CREDENTIALS=
GOOGLE_APPLICATION_PROJECT_ID=

Step 2: Create a Dialogflow Agent

Next, you can build a Dialogflow agent following the Dialogflow doc, Quickstart: Build an Agent. In this example, we make an agent and create an intent with the display name greeting. You can set your training phrases on the Dialogflow console for this intent.

After you finish the settings of the agent, you can call Dialogflow's API to analyze the intent of the message the bot receives.

Step 3: Connect Bottender with Dialogflow by bottender/dialogflow

To make the bot development enjoyable, we made a bottender/dialogflow package. You can install the package with npm or yarn.

With npm:

npm install @bottender/dialogflow

or with yarn:

yarn add @bottender/dialogflow

In the following sample code, you can see how elegant it is to integrate Bottender with Dialogflow. All you need to do is to fill in your environment variables, write a map between intents (e.g., greeting) and corresponding functions (e.g., SayHello).

const dialogflow = require('@bottender/dialogflow');

async function SayHello(context) {
  await context.sendText('Hello!');
}

async function Unknown(context) {
  await context.sendText('Sorry, I don’t know what you say.');
}

const Dialogflow = dialogflow({
  projectId: process.env.GOOGLE_APPLICATION_PROJECT_ID,
  actions: {
    greeting: SayHello,
  },
});

module.exports = async function App() {
  return chain([
    Dialogflow, //
    Unknown,
  ]);
};

Note:

• Dialogflow offers intent name and intent display name. The value of the former one is fixed once it is created; the value of the latter one can be change at any time. While writing the map between intents and functions at bottender/dialogflow, you can use any of the two to represent a single intent.
• Fore the full example code, please refer to Bottender example, With Dialogflow

Building with LUIS

Step 1: LUIS Setup

To build a bot integrated with LUIS (Language Understanding Intelligent Service), you have to create a new app in the LUIS portal following the Official Setup Guide and fill the three values: LUIS_APP_ID, LUIS_APP_KEY, and LUIS_APP_ENDPOINT into the .env file.

# .env

LUIS_APP_ID=
LUIS_APP_KEY=
LUIS_APP_ENDPOINT=

Step 2: Train and Publish Your LUIS Project

In this example, we create an intent with the intent name greeting. You can set your training phrases on the LUIS console for this intent. And then you have to train the model and publish it.

Step 3: Connect Bottender with LUIS by bottender/luis

To make the bot development enjoyable, we made a bottender/luis package. You can install the package with npm or yarn.

With npm:

npm install @bottender/luis

Or with yarn:

yarn add @bottender/luis

In the following sample code, you can see how elegant it is to integrate Bottender with LUIS. All you need to do is to fill in your environment variables, and score threshold, then write a map between intents (e.g., greeting) and corresponding functions (e.g., SayHello).

const luis = require('@bottender/luis');

async function SayHello(context) {
  await context.sendText('Hello!');
}

async function Unknown(context) {
  await context.sendText('Sorry, I don’t know what you say.');
}

const Luis = luis({
  appId: process.env.LUIS_APP_ID,
  appKey: process.env.LUIS_APP_KEY,
  endpoint: 'https://westus.api.cognitive.microsoft.com',
  actions: {
    greeting: SayHello,
  },
  scoreThreshold: 0.7,
});

module.exports = async function App() {
  return chain([
    Luis, //
    Unknown,
  ]);
};

Note:

• Fore the full example code, please refer to Bottender example, With LUIS.ai.

Building with Rasa NLU

You may choose Rasa NLU if you're finding an on-premises NLU solution.

Step 1: Rasa NLU Setup

To build a bot integrated with Rasa NLU, you have to install Rasa first following the Official Installation Guide. Next, you can train your NLU model by running:

rasa train nlu

This command will look for the training data files in the data/ directory and saves the model in the models/ directory. For information about how to generate training data, you can see Rasa's document, Training Data Format.

After you get your NLU model ready, you can run the following command:

rasa run --enable-api -m models/nlu-your-model-id.tar.gz

This command starts a server with your NLU model locally on port 5005. Next, you can request predictions from your model by calling the /model/parse endpoint. You can see here for the document of this API.

Step 2: Connect Bottender with Rasa by bottender/rasa

To make the bot development enjoyable, we made a bottender/rasa package. You can install the package with npm or yarn.

With npm:

npm install @bottender/rasa

Or with yarn:

yarn add @bottender/rasa

In the following sample code, you can see how elegant it is to integrate Bottender with Rasa. All you need to do is to set up the origin URL, and confidence threshold, then write a map between intents (e.g., greeting) and corresponding functions (e.g., SayHello).

const rasa = require('@bottender/rasa');

async function SayHello(context) {
  await context.sendText('Hello!');
}

async function Unknown(context) {
  await context.sendText('Sorry, I don’t know what you say.');
}

const Rasa = rasa({
  origin: 'http://localhost:5005',
  actions: {
    greeting: SayHello,
  },
  confidenceThreshold: 0.7,
});

module.exports = async function App() {
  return chain([
    Rasa, //
    Unknown,
  ]);
};

Note:

• Fore the full example code, please refer to Bottender example, With Rasa.