DF Designer: Design Discourse-Driven Open-Domain Chatbots with DeepPavlov Conversational AI technologies

This extension (previously known as DD-IDDE) enables developers of the open-domain chatbots with an option to design them within VS Code using customized Draw.io UI and Discourse Moves Recommendations to make dialog more smooth and natural.

This extension uses DeepPavlov's Dialog Flow Framework as the runtime environment for the open-domain chatbots. It can be used to build simple chatbots using DF SDK or to build complex multi-skill AI Assistants using our DeepPavlov Dream platform.

Discourse Moves Recommendation System has been built based on the Discourse & Speech Functions theory originally created by M.A.K. Halliday and further developed by Eggins & Slade in their book "Analysing Casual Conversation".

This extension is built on top of the unnoficial integration of the Draw.io (also known as diagrams.net) into VS Code made by Henning Dieterichs, hediet on Github (Main Contributor / Author).

Features

• Open .py files representing dialog flows of the open-domain chatbots using the Draw.io editor.
  • To create a new dialog flow, simply copy an example file from examples folder, right click on the file, choose "Open With", and select "DF Designer".
• Uses an offline version of Draw.io by default.

Tutorials

English

Stay tuned for a demo!

Russian

Here's a recording of introduction to DFF & DF Designer we've made back in the end of December.

The image link below leads directly to the introduction of the DF Designer itself:

Prerequisites

Environment

Python Modules

Build Simple Chatbots with DF SDK

DF SDK Installation

1. Clone SDK: git clone https://github.com/deepmipt/dialog_flow_sdk
2. Change to its directory: cd dialog_flow_sdk
3. Install requirements: pip3 install -r requirements.txt

DF SDK Runtime

1. Run DF SDK Runtime with Docker-Compose: docker-compose up --build

Design & Run Your Open-Domain/Scenario-Driven Chatbot in DF Designer

NOTE: By default, the extension uses an SFC predictor running in the cloud, so you do not need to have the SDK running locally for predictions to work. You can still use a local predictor by changing the sfc-predictor-url in VS Code settings.

Start With The Built-In Example

1. Open VS Code in your dialog_flow_sdk folder by running code ..
2. Go to examples folder.
3. Open food.py.
4. Right click on food.py, choose "Open with...", and in the dialog box choose "DD-IDDE Dialog Designer".

Use Discourse Moves Recommendation System

1. In Draw.io designer tab in VS Code, double click on the node, e.g., start_node, then choose a speech function from the list and click on save. Now you can click on the node again and click on Show Suggestions menu item. If nothing shows up click again.
2. Pick the suggestion based on the Speech Function you want to add support for.
3. Double click on that suggestion. You can specify the speech function of your target response if you like, or you can do that later, either in code or from the Draw.io Dialog Designer.
4. To add the text, you need to find the node of your target response in code and write the text in the RESPONSE field.

Use MIDAS Recommendation System

If you want to work not with speech functions, but with dialog acts:

1. In extension settings find Dff > Vscode-drawio: Selected-predictor and change sfc to midas
2. In Draw.io designer tab in VS Code, double click on the node, e.g., start_node, then choose a dialog act (MIDAS) from the list and click on save. Now you can click on the node again and click on Show Suggestions menu item. If nothing shows up click again.
3. Pick the suggestion based on the dialog act you want to add support for.
4. Double click on that suggestion. You can specify the dialog act of your target response if you like, or you can do that later, either in code or from the Draw.io Dialog Designer.
5. To add the text, you need to find the node of your target response in code and write the text in the RESPONSE field.
6. You can see the list of dialog acts and examples here.

Run

Once you've designed your Discourse-Driven open-domain chatbot, you can run it:

1. Open Terminal in the dialog_flow_sdk folder.
2. Run python3 examples/food.py.
3. If you didn't edit the file, you can type “Hi” to begin the dialog. The system then will ask you a question. Give any answer you want. If DF SDK Runtime is running (in Docker), you should see debug output from the system that says how your utterance was classified by the Speech Function classifier, and the system will provide the response based on the transition conditioned by the Speech Function of your response from the start_node to the corresponding node in the food.py file.

Build Complex Multi-Skill AI Assistants with DeepPavlov Dream

DeepPavlov Dream Installation

1. Clone Dream: git clone https://github.com/deepmipt/dream
2. Change to its directory: cd dream
3. Until Feb 21 2022, switch to feat/speech-function-dist-book-skill

NOTE: By default, the extension uses an SFC predictor running in the cloud, so you do not need to have the SDK running locally for predictions to work. You can still use a local predictor by changing the sfc-predictor-url in VS Code settings.

Prepare Dream to Run Built-In Example Locally

1. Create local.yml:

python3 utils/create_local_yml.py -p -d assistant_dists/dream_sfc/ -s dff-book-sfc-skill

Design & Run Your Open-Domain/Scenario-Driven Skill in DF Designer

Start With The Built-In Example

1. Open VS Code in your /skills/dff_book_sfc_skill/ folder by running code ..
2. Go to scenario folder.
3. Open main.py.
4. Right click on main.py, choose "Open with...", and in the dialog box choose "DF Dialog Designer".

Use Discourse Moves Recommendation System

Pre-Requisites (needed in your custom skill, e.g., dff_template_skill)

To use Discourse Moves Recommendation System using Speech Functions you need to add integration with Speech Functions classifier:

1. Copy /skills/dff_book_sfc_skill/scenario/sf_conditions.py next to main.py in the scenario folder of your dff_template_skill.
2. Add line from df_engine.core.keywords import MISC and line import scenario.sf_conditions as dm_cnd to your main.py file after line 14.

Using Recommendations in Dialogue Design

1. In Draw.io designer tab in VS Code, double click on the node, e.g., start, then choose a speech function from the list and click on save. Now you can click on the node again and click on Show Suggestions menu item. If nothing shows up click again.
2. Pick the suggestion based on the Speech Function you want to add support for.
3. Double click on that suggestion. You can specify the speech function of your target response if you like, or you can do that later, either in code or from the Draw.io Dialog Designer.
4. To add the text, you need to find the node of your target response in code and write the text in the RESPONSE field.

Run

Once you've designed your Discourse-Driven open-domain chatbot, you can run it:

1. Open Terminal in the dream folder.
2. Run docker-compose -f docker-compose.yml -f assistant_dists/dream_sfc/docker-compose.override.yml -f assistant_dists/dream_sfc/dev.yml -f assistant_dists/dream_sfc/local.yml up --build
3. In a separate Terminal tab run:

docker-compose -f docker-compose.yml -f assistant_dists/dream_sfc/docker-compose.override.yml -f assistant_dists/dream_sfc/dev.yml -f assistant_dists/dream_sfc/local.yml exec agent python -m deeppavlov_agent.run -pl assistant_dists/dream_sfc/pipeline_conf.json

Type your response. If you didn't edit the file of the demo dff_book_sfc_skill, you can type "Hi". After the response was returned by the bot, type "I love reading". As you custom Dream distribution is running (in Docker), you should see debug output from the system. 4. Alternatively, can talk directly via REST API. Go to localhost:4242 and send POST requests like this (where user_id should be different for each session):

{
	"user_id": "MyDearFriend",
	"payload": "Hi!"
}

then followed by:

{
	"user_id": "MyDearFriend",
	"payload": "I love reading"
}

By this moment, we've arrived at ```book_start`` node. Now, you can see how Speech Function Classifier works in this situation. To trigger it, we want to say something that it would classify as "React.Rejoinder.Support.Track.Clarify". Here's one example:

{
	"user_id": "MyDearFriend",
	"payload": "Why of course I do of course I do. Why do you ask?"
}

And at this very moment, you'll get the response that is conditioned by this very Speech Function:

Bot:  I think that reading is cool and all people should read books

More examples of SF-based conditions

For a detailed description of each speech function and more examples of working with speech functions when building skills, see SF-augmented version of Book Skill: https://github.com/deepmipt/dream/tree/feat/sf-bookskill-new/skills/dff_book_sfc_skill

Use MIDAS Recommendation System

If you want to use dialog acts (from MIDAS) instead of Speech Functions:

1. Clone Dream: git clone https://github.com/deepmipt/dream
2. Change to its directory: cd dream
3. Switch to feat/is_midas

NOTE: By default, the extension uses an SFC predictor running in the cloud, so you do not need to have the SDK running locally for predictions to work. You can still use a local predictor by changing the sfc-predictor-url in VS Code settings.

Prepare Dream to Run Built-In Example Locally

1. Create local.yml:

python3 utils/create_local_yml.py -p -d assistant_dists/dream_sfc/ -s dff-book-sfc-skill

Design & Run Your Open-Domain/Scenario-Driven Skill in DF Designer

Start With The Built-In Example

1. Open VS Code in your /skills/dff_book_sfc_skill/ folder by running code ..
2. Go to scenario folder.
3. Open main.py.
4. Right click on main.py, choose "Open with...", and in the dialog box choose "DF Dialog Designer".

Using Recommendations in Dialogue Design

1. In extension settings find Dff > Vscode-drawio: Selected-predictor and change sfc to midas
2. In Draw.io designer tab in VS Code, double click on the node, e.g., start_node, then choose a dialog act (MIDAS) from the list and click on save. Now you can click on the node again and click on Show Suggestions menu item. If nothing shows up click again.
3. Pick the suggestion based on the dialog act you want to add support for.
4. Double click on that suggestion. You can specify the dialog act of your target response if you like, or you can do that later, either in code or from the Draw.io Dialog Designer.
5. To add the text, you need to find the node of your target response in code and write the text in the RESPONSE field.
6. You can see the list of dialog acts and examples here.

Run

Once you've designed your Discourse-Driven open-domain chatbot, you can run it:

1. Open Terminal in the dream folder.
2. Run docker-compose -f docker-compose.yml -f assistant_dists/dream_sfc/docker-compose.override.yml -f assistant_dists/dream_sfc/dev.yml -f assistant_dists/dream_sfc/local.yml up --build
3. In a separate Terminal tab run:

docker-compose -f docker-compose.yml -f assistant_dists/dream_sfc/docker-compose.override.yml -f assistant_dists/dream_sfc/dev.yml -f assistant_dists/dream_sfc/local.yml exec agent python -m deeppavlov_agent.run -pl assistant_dists/dream_sfc/pipeline_conf.json

Provide any user id and type your response.

Editing the Dialog Designer and its DFF Python Side by Side

You can open the same *.py file with the Draw.io Dialog Designer and as .py file. They are synchronized, so you can switch between them as you like it. This is super pratical if you want to use find/replace to rename text or other features of VS Code to speed up your diagram creation/edit process. Use the View: Reopen Editor With... command to toggle between the text or the Draw.io-based DF Designer editor. You can open multiple editors for the same file.

DF Designer Contributors

• Oleg Serikov, oserikov on Github (Original Author)
• Bálint Magyar, mablin7 on Github (Frontend Software Engineer)
• Dmitry Babadeev, prog420 on Github (Main Contributor in August-October 2021)
• Denis Kuznetsov, kudep on Github (Author of DD-IDDE SDK & Dialog Flow Framework/DFF)
• Lida Ostyakova, lnpetrova on Github (Author of Discourse Moves Recommendation System)
• Dmitry Evseev, dmitrijeuseew on Github (Author of Wiki-based entity detection extensions for DFF)
• Kseniya Petukhova, Kpetyxova on GitHub (Developer & Tester of SFC-enabled skills)
• Veronika Smilga, NikaSmilga on GitHub (Developer & Tester of SFC-enabled skills)
• Maria Molchanova, mary-silence on Github (PM, Dev Tools)
• Maxim Talimanchuk, mtalimanchuk on Github (DevOps)
• Daniel Kornev, DanielKornev on Github (CPO)

Special thanks to Yuri Kuratov, yurakuratov on Github (Senior Researcher at DeepPavlov.ai)

Original Draw.io Extension Contributors

• Henning Dieterichs, hediet on Github (Main Contributor / Author)
• Vincent Rouillé, Speedy37 on Github

See Also / Similar Extensions

• Draw.io - This extension relies on the giant work of Draw.io. Their embedding feature enables this extension! This extension bundles a recent version of Draw.io.
• vscode-drawio by eightHundreds.

Research Paper

• Discourse-Driven Integrated Dialogue Development Environment for Open-Domain Dialogue Systems

Other Cool Conversational AI Tech by DeepPavlov

If you like this extension, you might like our other Conversational AI tech too:

• DeepPavlov Library: An open-source Conversational AI library (Python, TF 1.x, PyTorch).
• DeepPavlov Dream: An open-source Multiskill AI Assistant Platform.
• DeepPavlov Agent: An open-source Conversational AI orchestrator (it used by DeepPavlov Dream).