DF Designer: Design Discourse-Driven Open-Domain Chatbots with DeepPavlov Conversational AI technologies
This extension (previously known as DD-IDDE) enables developers of the open-domain chatbots with an option to design them within VS Code using customized Draw.io UI and Discourse Moves Recommendations to make dialog more smooth and natural.
This extension uses DeepPavlov's Dialog Flow Framework as the runtime environment for the open-domain chatbots. It can be used to build simple chatbots using DF SDK or to build complex multi-skill AI Assistants using our DeepPavlov Dream platform.
Discourse Moves Recommendation System has been built based on the Discourse & Speech Functions theory originally created by M.A.K. Halliday and further developed by Eggins & Slade in their book "Analysing Casual Conversation".
This extension is built on top of the unnoficial integration of the Draw.io (also known as diagrams.net) into VS Code made by Henning Dieterichs, hediet on Github (Main Contributor / Author).
Features
- Open
.py
files representing dialog flows of the open-domain chatbots using the Draw.io editor.
- To create a new dialog flow, simply copy an example file from examples folder, right click on the file, choose "Open With", and select "DF Designer".
- Uses an offline version of Draw.io by default.
Tutorials
English
Stay tuned for a demo!
Russian
Here's a recording of introduction to DFF & DF Designer we've made back in the end of December.
The image link below leads directly to the introduction of the DF Designer itself:
Prerequisites
Environment
Item |
Requirements |
Comments |
OS |
Debian-based distribution, e.g., Ubuntu or Windows 10 |
This version was tested on Ubuntu 18.04 under WSL2 on Windows 11 and Windows 10. |
Python |
v3.9+ |
This version was tested on OS with Python 3.9. |
Docker |
v20+ |
This version was tested with Docker v20.10.7 (64-bit). |
Docker-Compose |
v1.29.2 |
This version was tested with Docker-Compose v1.29.2. |
Python Modules
Item |
Requirements |
Comments |
lxml |
v4.6.3 |
This version was tested with lxml v4.6.3. |
Build Simple Chatbots with DF SDK
DF SDK Installation
- Clone SDK:
git clone https://github.com/deepmipt/dialog_flow_sdk
- Change to its directory:
cd dialog_flow_sdk
- Install requirements:
pip3 install -r requirements.txt
DF SDK Runtime
- Run DF SDK Runtime with Docker-Compose:
docker-compose up --build
Design & Run Your Open-Domain/Scenario-Driven Chatbot in DF Designer
NOTE: By default, the extension uses an SFC predictor running in the cloud, so you do not need to have the SDK running locally for predictions to work. You can still use a local predictor by changing the sfc-predictor-url
in VS Code settings.
Start With The Built-In Example
- Open VS Code in your
dialog_flow_sdk
folder by running code .
.
- Go to
examples
folder.
- Open
food.py
.
- Right click on
food.py
, choose "Open with...", and in the dialog box choose "DD-IDDE Dialog Designer".
Use Discourse Moves Recommendation System
- In Draw.io designer tab in VS Code, double click on the node, e.g.,
start_node
, then choose a speech function from the list and click on save
. Now you can click on the node again and click on Show Suggestions
menu item. If nothing shows up click again.
- Pick the suggestion based on the Speech Function you want to add support for.
- Double click on that suggestion. You can specify the speech function of your target response if you like, or you can do that later, either in code or from the Draw.io Dialog Designer.
- To add the text, you need to find the node of your target response in code and write the text in the
RESPONSE
field.
Use MIDAS Recommendation System
If you want to work not with speech functions, but with dialog acts:
- In extension settings find
Dff > Vscode-drawio: Selected-predictor
and change sfc
to midas
- In Draw.io designer tab in VS Code, double click on the node, e.g.,
start_node
, then choose a dialog act (MIDAS) from the list and click on save
. Now you can click on the node again and click on Show Suggestions
menu item. If nothing shows up click again.
- Pick the suggestion based on the dialog act you want to add support for.
- Double click on that suggestion. You can specify the dialog act of your target response if you like, or you can do that later, either in code or from the Draw.io Dialog Designer.
- To add the text, you need to find the node of your target response in code and write the text in the
RESPONSE
field.
- You can see the list of dialog acts and examples here.
Run
Once you've designed your Discourse-Driven open-domain chatbot, you can run it:
- Open Terminal in the
dialog_flow_sdk
folder.
- Run
python3 examples/food.py
.
- If you didn't edit the file, you can type “Hi” to begin the dialog. The system then will ask you a question. Give any answer you want. If DF SDK Runtime is running (in Docker), you should see debug output from the system that says how your utterance was classified by the Speech Function classifier, and the system will provide the response based on the transition conditioned by the Speech Function of your response from the start_node to the corresponding node in the food.py file.
Build Complex Multi-Skill AI Assistants with DeepPavlov Dream
DeepPavlov Dream Installation
- Clone Dream:
git clone https://github.com/deepmipt/dream
- Change to its directory:
cd dream
- Until Feb 21 2022, switch to
feat/speech-function-dist-book-skill
NOTE: By default, the extension uses an SFC predictor running in the cloud, so you do not need to have the SDK running locally for predictions to work. You can still use a local predictor by changing the sfc-predictor-url
in VS Code settings.
Prepare Dream to Run Built-In Example Locally
- Create
local.yml
:
python3 utils/create_local_yml.py -p -d assistant_dists/dream_sfc/ -s dff-book-sfc-skill
Design & Run Your Open-Domain/Scenario-Driven Skill in DF Designer
Start With The Built-In Example
- Open VS Code in your
/skills/dff_book_sfc_skill/
folder by running code .
.
- Go to
scenario
folder.
- Open
main.py
.
- Right click on
main.py
, choose "Open with...", and in the dialog box choose "DF Dialog Designer".
Use Discourse Moves Recommendation System
Pre-Requisites (needed in your custom skill, e.g., dff_template_skill)
To use Discourse Moves Recommendation System using Speech Functions you need to add integration with Speech Functions classifier:
- Copy
/skills/dff_book_sfc_skill/scenario/sf_conditions.py
next to main.py
in the scenario
folder of your dff_template_skill
.
- Add line
from df_engine.core.keywords import MISC
and line import scenario.sf_conditions as dm_cnd
to your main.py file after line 14
.
Using Recommendations in Dialogue Design
- In Draw.io designer tab in VS Code, double click on the node, e.g.,
start
, then choose a speech function from the list and click on save
. Now you can click on the node again and click on Show Suggestions
menu item. If nothing shows up click again.
- Pick the suggestion based on the Speech Function you want to add support for.
- Double click on that suggestion. You can specify the speech function of your target response if you like, or you can do that later, either in code or from the Draw.io Dialog Designer.
- To add the text, you need to find the node of your target response in code and write the text in the
RESPONSE
field.
Run
Once you've designed your Discourse-Driven open-domain chatbot, you can run it:
- Open Terminal in the
dream
folder.
- Run
docker-compose -f docker-compose.yml -f assistant_dists/dream_sfc/docker-compose.override.yml -f assistant_dists/dream_sfc/dev.yml -f assistant_dists/dream_sfc/local.yml up --build
- In a separate Terminal tab run:
docker-compose -f docker-compose.yml -f assistant_dists/dream_sfc/docker-compose.override.yml -f assistant_dists/dream_sfc/dev.yml -f assistant_dists/dream_sfc/local.yml exec agent python -m deeppavlov_agent.run -pl assistant_dists/dream_sfc/pipeline_conf.json
Type your response. If you didn't edit the file of the demo dff_book_sfc_skill
, you can type "Hi". After the response was returned by the bot, type "I love reading". As you custom Dream distribution is running (in Docker), you should see debug output from the system.
4. Alternatively, can talk directly via REST API. Go to localhost:4242
and send POST requests like this (where user_id
should be different for each session):
{
"user_id": "MyDearFriend",
"payload": "Hi!"
}
then followed by:
{
"user_id": "MyDearFriend",
"payload": "I love reading"
}
By this moment, we've arrived at ```book_start`` node.
Now, you can see how Speech Function Classifier works in this situation. To trigger it, we want to say something that it would classify as "React.Rejoinder.Support.Track.Clarify". Here's one example:
{
"user_id": "MyDearFriend",
"payload": "Why of course I do of course I do. Why do you ask?"
}
And at this very moment, you'll get the response that is conditioned by this very Speech Function:
Bot: I think that reading is cool and all people should read books
More examples of SF-based conditions
For a detailed description of each speech function and more examples of working with speech functions when building skills, see SF-augmented version of Book Skill: https://github.com/deepmipt/dream/tree/feat/sf-bookskill-new/skills/dff_book_sfc_skill
Use MIDAS Recommendation System
If you want to use dialog acts (from MIDAS) instead of Speech Functions:
- Clone Dream:
git clone https://github.com/deepmipt/dream
- Change to its directory:
cd dream
- Switch to
feat/is_midas
NOTE: By default, the extension uses an SFC predictor running in the cloud, so you do not need to have the SDK running locally for predictions to work. You can still use a local predictor by changing the sfc-predictor-url
in VS Code settings.
Prepare Dream to Run Built-In Example Locally
- Create
local.yml
:
python3 utils/create_local_yml.py -p -d assistant_dists/dream_sfc/ -s dff-book-sfc-skill
Design & Run Your Open-Domain/Scenario-Driven Skill in DF Designer
Start With The Built-In Example
- Open VS Code in your
/skills/dff_book_sfc_skill/
folder by running code .
.
- Go to
scenario
folder.
- Open
main.py
.
- Right click on
main.py
, choose "Open with...", and in the dialog box choose "DF Dialog Designer".
Using Recommendations in Dialogue Design
- In extension settings find
Dff > Vscode-drawio: Selected-predictor
and change sfc
to midas
- In Draw.io designer tab in VS Code, double click on the node, e.g.,
start_node
, then choose a dialog act (MIDAS) from the list and click on save
. Now you can click on the node again and click on Show Suggestions
menu item. If nothing shows up click again.
- Pick the suggestion based on the dialog act you want to add support for.
- Double click on that suggestion. You can specify the dialog act of your target response if you like, or you can do that later, either in code or from the Draw.io Dialog Designer.
- To add the text, you need to find the node of your target response in code and write the text in the
RESPONSE
field.
- You can see the list of dialog acts and examples here.
Run
Once you've designed your Discourse-Driven open-domain chatbot, you can run it:
- Open Terminal in the
dream
folder.
- Run
docker-compose -f docker-compose.yml -f assistant_dists/dream_sfc/docker-compose.override.yml -f assistant_dists/dream_sfc/dev.yml -f assistant_dists/dream_sfc/local.yml up --build
- In a separate Terminal tab run:
docker-compose -f docker-compose.yml -f assistant_dists/dream_sfc/docker-compose.override.yml -f assistant_dists/dream_sfc/dev.yml -f assistant_dists/dream_sfc/local.yml exec agent python -m deeppavlov_agent.run -pl assistant_dists/dream_sfc/pipeline_conf.json
Provide any user id and type your response.
Editing the Dialog Designer and its DFF Python Side by Side
You can open the same *.py
file with the Draw.io Dialog Designer and as .py
file.
They are synchronized, so you can switch between them as you like it.
This is super pratical if you want to use find/replace to rename text or other features of VS Code to speed up your diagram creation/edit process.
Use the View: Reopen Editor With...
command to toggle between the text or the Draw.io-based DF Designer editor. You can open multiple editors for the same file.
DF Designer Contributors
- Oleg Serikov, oserikov on Github (Original Author)
- Bálint Magyar, mablin7 on Github (Frontend Software Engineer)
- Dmitry Babadeev, prog420 on Github (Main Contributor in August-October 2021)
- Denis Kuznetsov, kudep on Github (Author of DD-IDDE SDK & Dialog Flow Framework/DFF)
- Lida Ostyakova, lnpetrova on Github (Author of Discourse Moves Recommendation System)
- Dmitry Evseev, dmitrijeuseew on Github (Author of Wiki-based entity detection extensions for DFF)
- Kseniya Petukhova, Kpetyxova on GitHub (Developer & Tester of SFC-enabled skills)
- Veronika Smilga, NikaSmilga on GitHub (Developer & Tester of SFC-enabled skills)
- Maria Molchanova, mary-silence on Github (PM, Dev Tools)
- Maxim Talimanchuk, mtalimanchuk on Github (DevOps)
- Daniel Kornev, DanielKornev on Github (CPO)
Special thanks to Yuri Kuratov, yurakuratov on Github (Senior Researcher at DeepPavlov.ai)
Original Draw.io Extension Contributors
- Henning Dieterichs, hediet on Github (Main Contributor / Author)
- Vincent Rouillé, Speedy37 on Github
See Also / Similar Extensions
- Draw.io - This extension relies on the giant work of Draw.io. Their embedding feature enables this extension! This extension bundles a recent version of Draw.io.
- vscode-drawio by eightHundreds.
Research Paper
Other Cool Conversational AI Tech by DeepPavlov
If you like this extension, you might like our other Conversational AI tech too: