Voice Setup in CodeRush for Visual Studio

Voice Features are invoked by holding down one of the two Ctrl keys while speaking.

Also available:

Setup involves three steps:

1. Entering your API keys
2. Enabling Voice Features
3. Setting Microphone Level

API Keys

CodeRush Voice Features requires an Azure Speech Services API key. At the time of this post, you can create a free Azure account that includes 5 audio hours of speech to text processing services every month. 

To set up a new OpenAI account (and get your OpenAI API key), click here.

Once you have set up your Azure account, create a new Speech Service if necessary, and copy one of the keys shown in the Keys and endpoint section:

Also note the Location/Region setting on this page.

Inside the CodeRush options dialog, navigate to the IDE\Cognitive\API Keys options page, and enter your Azure Speech Service API key (copied from the previous step) and the Azure Location/Region (which you can also copy from Azure):

CodeRush stores these API Keys as environment variables on your machine.

If you want to use the "Allow semantic equivalents" feature in Voice Commands, you'll need to also add an Open AI API key.

Once your API keys have been entered, click on the IDE\Cognitive\General options page.

Enabling Voice Features

On the IDE\Cognitive\General options page, check the  "Enable Voice Features" checkbox. Optionally you can also enable the "Enable Cognitive Features" checkbox to turn on the "Allow semantic equivalents" feature in Voice Commands. This option will also impact future cognitive features in CodeRush.

Setting Microphone Level

Below the Enable Voice Features checkbox, there is a slider allowing you to set the volume threshold that must be exceeded for CodeRush to start listening (when the Ctrl key is down).

The easiest way to set this for your location is to remain quiet for a moment, and then click the Auto Set button. This will measure the ambient volume in the room and set the threshold slightly above that level. If you regularly listen to music on speakers or work in a location with more ambient noise, make sure that your location's noise source is available at its normal levels when you click Auto Set. 

After setting the threshold, speak normally into your microphone. The volume meter should turn light blue when you are speaking (your volume exceeds the threshold), and be light orange when you are quiet (when the volume falls below the threshold).

Quick Test

Try these two tests after setting the API keys and enabling Voice Features:

Comment Dictation

1. Open or create a new C# file.
2. Go to an empty line in the file and enter "//" to start a comment.
3. Hold down the right Ctrl key and say "Hello Visual Studio."

Your spoken words should appear inside the comment:

Voice Commands

1. Open a Visual Studio solution.
2. Open any file from that solution holding down the left Ctrl key and saying "Open filename" (where filename is the name of the file you want to open). The file extension is not required if it's a *.cs file inside a C# project.

The file you specified should be opened immediately.

If either of these tests fail to work as expected, see the Troubleshooting section below.

Voice Feedback UI

When using Voice Features in Visual Studio, the Voice Feedback UI will appear near the caret in the code. The Voice Feedback UI shows live microphone volume levels when speaking (while one of the Ctrl keys is held down), and provides a color-coded animation and label corresponding to the kind of Voice Feature you are using.

For example, when naming a new identifier, you'll see feedback that might look something like this while you're speaking:

Once the Ctrl key is held down and from the moment you start talking, CodeRush immediately starts an audio streaming session with Azure Cognitive Services and the feedback UI starts to fade in. You can abort the session by pressing any non-modifier key, by moving the mouse a short distance, or by clicking the mouse. You can stop the streaming session and use the intended Voice Feature by simply releasing the Ctrl key when you are done speaking.

The Voice Feedback UI has other appearances:

When entering expressions in code (or inside interpolated strings), the animation is orange:

When dictating comments, strings, and some TextFields and TextBoxes (any place where human-readable display text might be expected) the feedback UI animation is green and the label describes what CodeRush is listening for: 

If the Alt key is held down while speaking (useful for referencing code inside comment or string dictations), the animation goes to orange (to indicate code expressions can be spoken) and an indicator to the right shows the Alt key is down:

When referencing a code symbol from inside an XML Doc Comment (by holding down the Alt key), the animation turns purple and there's an indicator to the right:

As shown earlier, new identifiers are pink:

Type references are yellow:

And when listening for voice commands, the animation is blue:

Each of these different modes represents different voice processing engines working in the background to turn your spoken words into something sensible.

Troubleshooting

Nothing happens when I hold the Ctrl key and speak.

Bring up the CodeRush IDE\Cognitive\General options page. Make sure you can see the volume indicator change as you speak. Also check the IDE\Cognitive\General options page and verify your Azure Speech Service API key is entered correctly, and the Azure Speech Region matches the region setting established for your key on the Azure portal (both of these Azure fields need valid values). Bring up Visual Studio's Output tool window, in the Show output from combobox switch to the "Voice" output page, and look for error messages here.

I see a "No audio detected!" message on the IDE\Cognitive\General options page.

Check your microphone to make sure it is not muted. Check your Windows sound settings (right-click the speaker icon in the system tray and choose "Open sound settings").
 
When you press the Ctrl key, CodeRush listens to the default sound input device (microphone) selected on this Windows settings page. This page includes a  "Test your microphone" UI where you can verify your microphone is sending audio to Windows.

Voice Feedback UI hides unexpectedly while talking.

Voice Features are designed to work when the Ctrl key is held down and no other keyboard or mouse input actions are performed. So make sure the mouse isn't moving significantly or being clicked while you speak.

Voice Recognition Seems to Have Difficulty Getting My Words or Does Nothing When I Speak

Some microphones reset their volume to a lower level after a reboot or when the machine awakes from a sleep state. If the volume indicator on the Voice Feedback window is low, try turning up the sensitivity on your microphone.

Generated code/dictation sometimes misses the last spoken word.

Generated code isn't quite right.

• Try a second time (make sure you're holding the Ctrl key down for the entire time you are speaking).
• If there is phonetic ambiguity, try disambiguating by saying the words you might normally omit (e.g., "dot" to separate qualifiers, "with" to enter parens and specify arguments).
• Bring up the Output tool window and switch to the "Voice" page, and review the data there to see if the words heard (listed in a column with timing data) match what was said.
• If the words in the Voice Output are correct but CodeRush still fails to generate the expected code, bind a shortcut to the GetVoiceTestCaseData command on the CodeRush IDE\Shortcuts options page, and try the Voice to Code again.

If you can duplicate the issue, now press the shortcut bound to the GetVoiceTestCaseData command, then paste the clipboard text into an email and review it (it will contain some code from where you tried the voice command and the data we need to reproduce). Send it to support@devexpress.com (feel free to edit the included code to remove anything sensitive, or better yet, try to duplicate in a method that contains code you don't mind sharing).

Voice Features do not work in certain CodeRush text boxes.

Voice Features do not work when IntelliRush is up.

Voice Features sometimes do not work when a tool window has focus.

Voice features will not work when some of the older Visual Studio tool windows (built under WinForms technology) are active.

The work around is to give the editor focus first by pressing Escape (or clicking on the editor).

Some of the older Visual Studio tool windows that block Voice Features include:

• Breakpoints
• Properties (for winforms projects)
• Resources
• ​Toolbox
  

Voice to Code seems to have problems when there are long pauses in the middle of an expression.

I see an error message - "Cognitive features enabled but OpenAI Key is missing"

We recommend adding a valid OpenAI Key on the IDE\Cognitive\API Keys options page, or clearing the "Enable Cognitive Features" checkbox on the IDE\Cognitive\General options page. 

I see an error message - "OpenAI rejected your authorization, most likely due to an invalid API Key"

We recommend double-checking that your OpenAI Key has been correctly entered (on the IDE\Cognitive\API Keys options page), or clearing the "Enable Cognitive Features" checkbox on the IDE\Cognitive\General options page. 

I see an error message - "Error at completions..."

This error may indicate your OpenAI model name is incorrect or has been deprecated by OpenAI ("NotFound" appears at the end of the message). This error message can also happen when the OpenAI model name has not been specified ("BadRequest" appears at the end).

We recommand bringing up the IDE\Cognitive\General options page and verifying the text in the OpenAI Model Name text box represents a valid OpenAI model name:

Voice to Code takes longer than it should to generate code.

Voice Feedback UI has an unexpected color or label (which may lead to unexpected results).

Let us know how to reproduce. We are still working on integrating voice into all the CodeRush refactorings and templates. If you find one we've missed or got wrong, please let us know.

And if you're still having trouble, please contact our Support Center (support@devexpress.com).