Let's harness the power of voice to collaborate and interact with your code and the people you work with
🔥 What it can do? (everything with just voice)
▶ Trigger your Azure DevOps pipeline build
📞 Call a person on Microsoft Teams
🗣 Open Microsoft Teams Chat of a person
📩 Open Outlook of a person
🔁 Get total pipeline runs
😅 Crack a programmer joke
📝 Tell a funny programmer story
🤖 Chat with Sandy (Powered By OpenAI's GPT-3 model) - Optional
👾 Generate code (Powered by OpenAI's Codex model) - Optional
🔒 Lock Screen
More Coming soon!
⚙ Setup the environment - Windows
Requires - Python 3.7 and above
Requires - node ^12
Once you install the extension, hit Ctrl+r to open Run and type %USERPROFILE%\.vscode\extensions
Search for sandy-codes-py voice collab extension
Open a Powershell/command prompt inside the extension folder and execute npm install - this will install all the node modules required
Inside the same folder create a python virtual environment py -m venv venv
Activate the environment and install the requirements venv/Scripts/pip install -r requirements.txt
The Microsoft Team call,chat and outlook email option will use your default browser, change this from the system settings if you want to open up in another browser
Create .env file inside python_scripts folder with the following variables to make the API's work
PERSONAL_ACCESS_TOKEN = "" # Azure DevOps token
OPENAI_API_KEY = "" # Optional
Copy the voice-collab-deepgram-templates folder inside python_scripts to C:\Users\<User>\AppData\Local\Programs\Microsoft VS Code - replace User with your User name
Restart VS Code, you should see a 🔇stopped icon in the status bar and a success notification if the setup was successful ✅
Read the FAQ below for possible issues you might face and how to add your own custom commands
Once you've set all the above them your good to go 🏁
🤔 How to Use
Once the setup is done, open up the developer tools if you want to see the actual working of the extension (Help> Toggle Developer Tools)
Open the Command Palette (Ctrl+Shift+P), type Voice Collab :
You should see the list of commands you can use, select Start Recognizer server, this will start the Python server which will accept the WebSocket connection on local host port 8002
Open the Command Palette (Ctrl+Shift+P), type voice collab: connect to server, this establishes a WebSocket connection from the extension to the local Python server
Finally, Open up localhost:8002 on your browser
Now start speaking out the below commands to trigger the respective actions 🔊
🦾 Commands to invoke
Description
Trigger phrases
Triggering Azure DevOps pipeline
say - start build or trigger pipeline build
Create the requirements.txt file
say - create requirements.txt or create requirements file
Get all Azure DevOps pipeline runs count
say - get total pipeline runs
Call someone on Microsoft Teams
say - call megan (this name could any name from MSFT_ACCOUNT_NAME_LIST list in _constants.py file)
Email someone on Outlook
say - email megan
Open up someone's Microsoft Teams Chat
say - open megan's chat
Speak out 5 commands you can use
say - show help or help or what are the commands I can use
Speak out all the commands you can use
say - help more or list all the commands I can use
Push the code to remote
say - git push or push code (still in progress)
Commit code and lock the screen
say - taking a break or break time
Lock the user screen
say - lock screen
Crack a random programmer joke
say - crack a joke or joke about programmers or make me feel better
Narrate a random programmer story
say - tell a story or story time
Chat with Sandy
say - hey sandy or sandy and then say the phrase you wanna ask Example: Am I the best programmer in the world?
Generate code
say - hey codex or codex and then say what you want Example: Create a Python class with name Employee, Create a Python dictionary with country and it's capitals
💻 List of commands available in the Command Palette (Ctrl+Shift+P)
Commands
Description
Voice Collab: Notification
This will send in a test notification to check if the extension runs without errors
Voice Collab: Start Recognizer Server
Start the Python server that will accept WebSocket connections on port 8002
Voice Collab: Connect to server
Establish a WebSocket connection with the locally hosted Python server on port 8002
Voice Collab: Stop Voice Recognizer
Terminate the Python server
Voice Collab: Disconnect from server
Close the WebSocket connection with the Python server
Voice Collab: Setup virtual Environment
Create the virtual environment for the PIP packages for the server to run
Voice Collab: Install requirements
Install all the PIP packages needed for the Python server
📌FAQ
How to install PyAudio?
Install PyAudio as pipwin install pyaudio, but first install pipwin as pip install pipwin - This is not needed but just in case
How to add more people into the system?
You can add more people from your org into the MSFT_ACCOUNT_NAME_LIST in _constants.py file
How to add your own commands?
Add a new item to COMMAND_DETAILS dict in _command_mapping.py file with the following details
method_name - Name of the method that you'll have to create in _helper.py file,
description - A description of the command (this will be used when you ask for help)
success_message - This will be spoken out if the method returns true
failure_message - This wil be spoken out if the method returns false
add_args - This is used if you want to pass any args to the method
args - This is passed all the time to the method
kargs - This is passed all the time to the method
speak_args - If set to true then it speaks out the return string from the method
Add the number os phrases with which this method should be triggered in COMMAND_MAPPINGS dict in _command_mapping.py file, the key should match the key in the COMMAND_DETAILS dict
Open developer tools from Help> Toggle Developer Tools in VS Code to see the logs
The speech recognition is done with Deepgram's Deep Learning model
Search for the PID netstat -ano | findstr 8002 the last column is the the Process ID - Powershell
Kill it with taskkill /F /PID <PID> - Powershell
Raise an issue if you're unable to resolve it yourself
