When we started working on CodeRush for Roslyn, we realized we needed a convenient way to deliver updates to our customers.
We knew that Visual Studio already provided a great tool for extension developers - the Extensions Gallery on the Visual Studio Marketplace. This public repository makes it easy to deliver polished updates to the world, but we also wanted internal developers using CodeRush for Roslyn to have the same experience in their daily work. We wanted to make it as easy for them to participate in beta testing and getting new beta updates as it is for customers to get new releases of CodeRush: Automatically.
Fortunately the Visual Studio Extensions Gallery lets us add a private source for extensions, which can be used to deliver updates for a selected group of beta testers.
I’d like to share our experiences with the Private Gallery and show how we found it to be an effective tool in creating better products for customers.
Daily Builds
When you are working in an agile environment, it is crucial to have a rapid update cadence and a stable channel to reliably deliver those updates to testers, internal users, and customers.
In building our products at DevExpress, we use an internal build farm that executes the entirety of the test and build process automatically, running on dedicated hardware. This allows developers to remotely generate new builds in a matter of minutes. Once a build is ready, we can distribute that build using the Private Gallery. This combination of dedicated build farm plus effortless & precisely-targeted distribution allows us to easily provide updates for our beta testers daily, or more frequently if needed.
Quality Assurance
This workflow is indeed fast with barely a hit on resources, but what about quality? Since our developers are constantly pushing code out to our development branch through version control, we must take preventative steps to ensure that new code doesn’t break existing features before giving it to our testers. And since CodeRush is an extension to Visual Studio, our beta testers are also using it to create production code. So every beta we ship has to be solid. For us, we found having a comprehensive suite of unit and integration test cases goes a long way in ensuring solid builds. If one test case fails, the build farm doesn’t allow an upload to the Private Gallery.
So what does a comprehensive testing suite look like? For us, that means about 36,000 unit tests, which thoroughly covers about 75% of our code. We also have nearly 500 full-blown integration tests, which start instances of Visual Studio and ensure CodeRush for Roslyn properly loads, MEF composition binds as expected, and integrated features are operating correctly.
Having a comprehensive suite of tests is essential to being able to deliver high-quality daily builds to your testers. It is so important, that it has changed how we work. For example, developers on our team are not allowed to check in any new features without also checking in supporting test cases to prove the code works as expected (and that it doesn’t work in unexpected ways). And developers are not allowed to submit bug fixes without also submitting new test cases proving that the fix works.
So we definitely consider our test suite to be a valuable company asset, one that every developer continues to invest in as we move forward.
Setting up the Private Gallery
The first step in setting up a Private Gallery is to specify which URLs Visual Studio will use to download details about any updated extensions.
To do this, open the Options window using the Tools | Options menu, then navigate to Environment | Extensions and Updates options page. To the right of the Additional Extension Galleries list box, click the Add button, then specify name and URL for the Private Gallery:
In the example above, we’ve named our gallery “Custom Gallery” and specified http://localhost:5000/custom.xml as the URL.
Click OK to close the Options dialog.
Now, if you bring up the Extensions and Updates dialog, and open the Online category, Visual Studio will show our new Private Gallery:
Unfortunately, if you try to use it right now, Visual Studio will likely complain about its inability to connect to the remote server. Don’t worry. This is expected at this point because we also need to setup a server to supply an Atom Feed file for the extensions we want to distribute.
The Atom Feed File
The Atom Feed file is an XML file containing a list of all the extensions available in the gallery. Each entry in the list includes essential information, such as extension ID, version details, author, etc. Official documentation on Atom Feed files already exists in the Atom Feed for a Private Gallery article, so we won’t dive into the details and structure here. However, we will show you how we dealt with the hardest part: generating and updating this file.
Check out the code in this project which updates the Atom Feed file each time you build and publish your extension. It is a console application expecting two command line parameters: the VSIX file path and the destination folder to be served by an HTTP(s) server. For example, you might add a call like this to the UpdateAtomFeed application inside your build script:
UpdateAtomFeed.exe "C:\Builds\TestExtension.vsix" "C:\GalleryServer\wwwroot"
The UpdateAtomFeed application copies the VSIX file to the specified server folder, then extracts the VSIX information and creates a custom.xml Atom Feed file (or updates an existing Atom Feed file if found).
A Simple .NET Core Atom Feed Server
Both the Atom Feed file and your VSIX files need to be hosted on a server available to your network, so let’s create a small .NET Core server application that does just that.
Start by creating a new .NET Core console application and name it “GalleryServer”. Then add the following NuGet dependencies:
- Microsoft.AspNetCore
- Microsoft.AspNetCore.Hosting
- Microsoft.AspNetCore.StaticFiles
Next, add the following code which creates and configures our static file server:
using System.IO;
using Microsoft.AspNetCore.Hosting;
using Microsoft.AspNetCore.Builder;
using Microsoft.AspNetCore.StaticFiles;namespace GalleryServer {
public class Startup {
public void Configure(IApplicationBuilder app) {
var provider = new FileExtensionContentTypeProvider();
provider.Mappings[".vsix"] = "application/vsix";
app.UseStaticFiles(new StaticFileOptions() {
ContentTypeProvider = provider
});
}
public static void Main(string[] args) {
string path = Directory.GetCurrentDirectory();
string wwwRootPath = Path.Combine(path, "wwwroot");
newWebHostBuilder()
.UseKestrel()
.UseWebRoot(wwwRootPath)
.UseContentRoot(wwwRootPath)
.UseStartup<Startup>()
.Build()
.Run();
}
}
}
This console app listens to the http://localhost:5000 address and starts serving static files from wwwroot folder:
Note that in the code we have also specified a FileExtensionContentTypeProvider and registered the .vsix extension mapping as an application/vsix MIME type. This is necessary to serve VSIX files, because VSIX is not one of the standard file types.
Now when you open Tools and Extensions, Visual Studio should reveal our new test extension:
Nice. And now we can now use our Private Gallery to easily install & update new builds of this extension.
Of course, we also need to allow connections from outside our local host, so the gallery becomes available to users outside our network. On Windows we can use IIS to accomplish this.
Wrapping Up
For us, the benefits of agile development are amplified when the edit/compile/test/build/distribute/feedback cycles are short. Thanks to Visual Studio’s Private Gallery options, along with our build farm and comprehensive test cases to ensure product quality, we were able to shrink that typical feedback loop (which can lumber on for days, distracting valuable resources) down to a cycle measured in just a few hours and with very little impact on existing resources.
Full source code and samples are here.
Written by Alex Zakharov & Mark Miller