This extension provides tasks for training, testing, and deleting NLU models from LUIS, Lex, and Dialogflow. It wraps the NLU.DevOps CLI tool to reduce the number of steps needed in a build pipeline to set up CI/CD for an NLU model.
Training an NLU model
To train an NLU model, add the following configuration to your pipeline:
- task: NLUTrain@0
You can also train a model from the generic utterances model using the
Check out the docs for LUIS and Lex for more information on pipeline variables you will need to set up access tokens and toggle NLU provider-specific features.
For more information about this task, see the documentation for NLUTrain.
Testing an NLU model
To test an NLU model, add the following configuration to your pipeline:
- task: NLUTest@0
utterances input should be a path to a JSON file with labeled generic utterances. This format is similar to the LUIS batch test format, except that entities are configured with
matchIndex instead of
To test with speech WAV files, set the
speech input to
true and set the base directory for the speech files with the
speechDirectory input. For more information about running NLU tests from speech, take a look at the docs on NLU.DevOps.
true will run your test utterances against the NLU provider and compare the results against the details in the
utterances input. It outputs failing tests for any false positive or false negative intents or entities. It outputs passing tests for any true positive or true negative intents or entities.
For reference, here is an example run of NLU test results.
For more information about this task, see the documentation for NLUTest.
Deleting an NLU model
For CI/CD, you often want to import, test, and then delete an NLU model. To delete the NLU model you set up for testing in a CI environment:
- task: NLUClean@0
For more information about this task, see the documentation for NLUClean.