This extension provides tasks for training, testing, and deleting NLU models from LUIS, Lex, and Dialogflow. It wraps the NLU.DevOps CLI tool to reduce the number of steps needed in a build pipeline to set up CI/CD for an NLU model.

Training an NLU model

To train an NLU model, add the following configuration to your pipeline:

- task: NLUTrain@0
  inputs:
    service: luis
    modelSettings: path/to/luis.json

You can also train a model from the generic utterances model using the utterances input.

Check out the docs for LUIS and Lex for more information on pipeline variables you will need to set up access tokens and toggle NLU provider-specific features.

For more information about this task, see the documentation for NLUTrain.

Testing an NLU model

To test an NLU model, add the following configuration to your pipeline:

- task: NLUTest@0
  inputs:
    service: luis
    utterances: path/to/tests.json
    publishTestResults: true

The utterances input should be a path to a JSON file with labeled generic utterances. This format is similar to the LUIS batch test format, except that entities are configured with matchText and matchIndex instead of startPos and endPos.

To test with speech WAV files, set the speech input to true and set the base directory for the speech files with the speechDirectory input. For more information about running NLU tests from speech, take a look at the docs on NLU.DevOps.

Setting publishTestResults to true will run your test utterances against the NLU provider and compare the results against the details in the utterances input. It outputs failing tests for any false positive or false negative intents or entities. It outputs passing tests for any true positive or true negative intents or entities.

For reference, here is an example run of NLU test results.

For more information about this task, see the documentation for NLUTest.

Deleting an NLU model

For CI/CD, you often want to import, test, and then delete an NLU model. To delete the NLU model you set up for testing in a CI environment:

- task: NLUClean@0
  inputs:
    service: luis

For more information about this task, see the documentation for NLUClean.