Auto-Deploy SSIS packages with VSTS and Octopus Deploy

For a bit now, I have been describing to teams how they can deploy their SSIS packages to their SQL instances automatically, but I never “really” had to do it myself. Until now. The client that I am working with currently just went through a very large migration of on-premise TFS  to VSTS and with that came the need to change how their reporting structure was both generated and consumed. Before the migration, we were generating ALM reports through direct SQL queries against the on-premise TFS Warehouse. During the migration and onward we had to generate an SSIS package that would pull information directly from the VSTS Rest API and populate a local data warehouse that could help us accomplish nearly the same thing. Since I wanted to keep my solution and project clean – no OctoPack (which isn’t supported, although you can hack your project to include it), and no nuspec file or extraneous other items in my solution like a .nuget folder – I opted to see if I could perform an auto-deploy just using VSTS and Octopus Deploy. Here is how I did it, while keeping my SSIS solution pristine and leveraging the available marketplace tasks and Octopus Deploy step templates.


For this blog post I am using VSTS and Octopus Deploy version 3.13.10. Caution – your mileage may vary depending on whether you use TFS 2017 or a lower version of Octopus Deploy, but the technique is sound.


  • You are using the internal nuget repo of Octopus Deploy. I can’t claim this process will work with external repos, but that is a challenge to others who read this.
  • You have a rudimentary understanding of VSTS and Octopus Deploy and how they each work independently.

Here is what you will need:

  • VSTS Account where your SSIS solution is located – this example is using Git, TFVC will work just as well
  • SSIS Marketplace Task – installed on your VSTS or TFS instance
  • Octopus Deploy Integration Tasks – installed on your VSTS or TFS instance
  • Octopus Deploy Project with servers already added to your environments
  • Octopus Deploy API key pushing packages from VSTS to Octopus Deploy built in repository (How to generate api key)
  • The Octopus Tentacles running on your destination servers should be running as a service account that has dbo access to your DB and local admin rights on the server to execute Remote PowerShell.

VSTS Component

Building out the CI build is straight forward, except that with SSIS, SSRS projects, the standard MSBuild and .NET Desktop build templates do not work as required. As Abel SquidHead points out you can perform an SSIS Build using commandline tasks within a build definition, but wouldn’t it be a more elegant to have a Build Task that did all the work for you save for a couple of parameters?

So to start, we need to add a Octopus Deploy Service Endpoint connection.


Click Ok and you should see something like this:


Now click on the Build and Release tab and create a new Empty Build definition. image

Then add the SSIS Build Task, Copy Files, Publish Artifact,the Octopus Deploy Pack, the Octopus Deploy Push Packages.

Note: You add tasks by clicking on the “+” symbol in the Phase 1 section.


Edit the Build SSIS task to point to your solution and project if necessary.


Next edit the Copy files Task so that it looks like so:


Next Edit publish artifact task so that it looks similar to this. You want to make sure that pushing your artifacts to a drop location for the next step to pick it up without all of the other bits and pieces that comprise your source tree.


Next Edit your Package Step. There is a lot to do here, but I will try to explain it a little by the numbers. I won’t describe the Advanced areas, but will leave that to the reader to experiment with. What this step does is simulate the same effort that a nuspec file performs when added to your project.


  1. Represents the name of the package you want. This name should be a constant for any package you wish. If you need to put other information, revision numbers, etc you can use the Nuget or Advanced options to provide the proper metadata for your package
  2. Represent the package format, until recently Octopus Deploy’s internal nuget repository only handled nupkg files, now it allows you generate either nupkg or zip files for you usage in your Octopus Deployment process
  3. This is the version number that will be appended to the package ID described above. This should be an auto-incrementing number so that each time you build you get a new version. Important: This version number does not typically represent the internal versioning of your package’s content. This is used as a metadata field to ensure that you are going to deploy the latest from your build output.
  4. Represents the location where your output resides, in this case it will be in the bin directory of the solution.
  5. Represents where you want to put your zipped or nupkged file contents, in this case the “artifactstagingdirectory\drop” folder

Now we move onto the last task in our process. Edit the Push Packages to Octopus. This pretty straight forward. Just select your Octopus Deploy Connection that you created earlier in this blog and the location of where you put your nupkg or zip file. (In this example I am using the nupkg format).


Now click the Save and Queue drop down and select Save. Give a reason if you wish and then we select the Triggers menu item for our process. Select CI, use the master branch and then click Save again.


You are now done!

It wise for you to test out your build first. One piece of advice is to disable the Package and Push Octopus steps first so that your build actually does what it is supposed to do and then enable each step after each successful build.

Octopus Deploy Component

There are a couple of things here that need to be accomplished for this section of the blog.

First login into your Octopus Deploy instance and then navigate to your project. You should see something like this:


You first want to select the Process to create how your deployment process is going to work. The following Options will be dependent upon you creating the process flow.

In the beginning your project and process will be blank and it will be helpful in getting your process started.  There are a lot of options to choose from, but for this exercise we are going to choose Deploy Package and Deploy ISPAC SSIS project from a package step found under the SQL Server grouping.  Once you do that your process should look something like this:


I will not dig too deep into the mechanics, but I will let you know what you need to change in order to make this deployment process clean and smooth.

First, lets edit the Deploy Package step. You will need to  scroll down to the bottom of the page. Select the Configure Features link and uncheck everything except for “Custom Installation Directory”. Click Apply and now your Deploy Package step should look similar to this:


Select Package name you chose from setting up VSTS. Simply typing a few letters will give you an autocomplete for the package you are looking for. In the custom installation directory, select a directory that you wish to put the package in and have it unpack itself. For more control you can use Variables to change this location based on server or environment.  Click the save button at the bottom of the page and now we go on to the next step.

Edit the Deploy IPSAC SSIS project step. This step is an incorporation of a community step that I have used before in other SSIS implementations. Since we are going with the basics here. You should see something similar after you have filled out the appropriate fields.


Once we have filled in the appropriate values and have selected the step that gives us the location where the ipsac file is going to be located at. Now we will finish this up with either one of the following options.

Note: The fields can be edited depending on what your particular requirements are. Coordinate with your DBAs to better understand how these different fields work within your particular implementation.

Option 1 (Automatic Release Creation):

Option 1 revolves around Automatic Release Creation. Octopus Deploy has an observer mechanism that looks at changes to packages that are being inserted into the Octopus Deploy internal repository.

First select Triggers. This is going to get us to the point where when a new build from VSTS has successfully completed we can observe a new package coming in and then have it automatically deploy to our initial environment.


This section was moved from a previous version of Octopus Deploy, but the mechanism. First check the Create a release check box and then select the package step that will be executed when a new a package is pushed to the internal repository. Click Save and you are almost ready to go.

Warning: Automatic Release Creation will not be available if you do not have a Package Deploy step template in your deployment process. In our particular case this doesn’t matter, but it is worth to note.

Option 2 (Lifecycles):

Lifecycles are yet another way to for you to

  1. Define Quality gates for your deployments
  2. Give greater control on how to keep your deployments from reaching Production pre-maturely

By default, when a project is a created the Default Lifecycle is pretty wide open. So what we are going to do is give ourselves a gated way to ensure that we automatically deploy to our lowest environment and then require manual or other types of approval for higher environments.

First select Library and then the Lifecycles left hand menu selection. You should see something like this:


Select the Default Lifecycle so that we can edit it. The purpose here is to put the gateways in place for our different environments.  The default before we edit will not have an element selected called Phases. Phases allows for deployers to select different environments to start and end a particular process. Since we know that we have just one project and it will have to start in our dev and then move upwards from there after various checks.

Note: Once you have selected a Lifecycle for your project, each new release and deployment will use that Lifecycle. You cannot change your Lifecycle once you have created a release and deployed that release.

First we are going to add our 1st phase, this will help us establish where are starting point in the deployment process is going to be. We click Add Phase and then we see something like this:


We select our initial environment, select the automatically deploy radio button, click add, then give our Phase a meaningful name and then click Save. We can then go and other phases for this lifecycle, but for the sake of brevity this work is for getting things done.

I won’t go into any in depth discussions here, but there are a lot of options and other items you can use within these different phases to help promote your code from one environment to another.

Now that you have saved everything you are ready to deploy your SSIS code from source control out to servers all the way out to production.  You may find that you have do some tweaking with your servers or with the variables that you need to use for your deployment process. By practicing your deployments in a lower throwaway environment, you have a chance of greater success to apply the same process for all of your downstream environments.

Note: If this doesn’t work right out of the gate, don’t get frustrated! The VSTS build logs and the Octopus Deploy Task logs provide a wealth of troubleshooting information your ultimate success.


This exercise has shown that you can get your SSIS package built, packaged, pushed, and deployed to your lowest environment for immediate testing. The steps shown here are pretty straightforward and can be adjusted to suit your needs. To be honest it took me longer to write and screen capture this post than it took me to build out this pipeline. Smile 

Happy SSIS Deployments!


How to Update Octopus Deploy Tentacle to Restart Automatically

Seen this too many times before, where the users (Developers were unaware of a server reboot that caused their Tentacle to stop. Here is a defensive way to make sure your Tentacles start consistently.

Here is how to update the Octopus Tentacle in the case that the server is scheduled for downtime or just in general to avoid Tentacles being or considered offline within the Octopus Deploy server UI. Typically during installation of a tentacle on a destination server you don’t have the ability to recover gracefully if something goes wrong. This can be troublesome during patching or long server reboot times. Hmmm. For some automation here.

Open PowerShell as Administrator:


Open services.msc


Scroll to OctopusDeploy Tentacle service:

Right click and select properties:


Change Startup type to: Automatic (Delayed Start)

Switch to the Recovery tab:


Change the 1st, 2nd, Subsequent failures to Restart the service


Click apply or OK and you will now have a tentacle service that will be more stable after your machine boots from either a scheduled or maintenance reboot.

Benefits of using Octopus Deploy Integration Tasks in vNext Builds

If you are like me and you use Octopus Deploy for deploying your projects; it can be a challenge to keep your OctoPack version updated. Restoring the OctoPack NuGet package each time you build with VSTS or TFS can be a challenge because if you perform a TFS XAML Build the build will fail because it cannot find the associated OctoPack targets and associated dlls. A workaround is to include the packages folder that contains the OctoPack targets file and associated dlls with your checked-in codebase, but that can be messy and lead to artifacts being left over in the case you wish to upgrade OctoPack to a newer version.

Another detractor to leveraging OctoPack in your solution sometime around version 3.4 a number of breaking change were introduced that caused nuget push issues. The teams that I work with on a daily basis are still on an older version of Octopus and when they installed the latest version of the Tentacle issues started to crop up along with failed builds and pushes to the internal NuGet repository.

So what are the benefits? According to the marketplace documentation, you can still use your OctoPack MSBuild arguments, but it doesn’t really apply to your older XAML builds.

Benefits of Octopus Deploy Integration

Some of the larger benefits when using the Octopus Deploy integration steps are:

  • You are always up-to-date
  • You have a clean project (no more packages to put with your codebase)
  • You have more Octopus Steps to play with (OctoPack can do them, but again it means more MSBuild parameters)
  • Troubleshooting is easier (Build shows all of the output in the console)

Benefits of vNext Builds with Octopus Deploy Integration

There are others who have blogged about the benefits of moving to the next version of Build. So I won’t go into the particulars. Suffice to say that replicating your XAML build in the new Build system is extremely beneficial and coupled with the Octopus Deploy Integration extension it can be even more powerful (

  • Control – You own it, you build it
  • TaskGroups (combined step tasks for Build templates)
  • Build Templates (cloneability, reusability)
  • Cleaner Visual Studio Solution
  • Centralized build/package/deploy processes
  • Decoupling of dependencies to installed packages

Even if you still want to use OctoPack, you can, you just have to take your old MSBuild arguments and paste them into the Visual Studio Build Step MSBuild Arguments parameter. Under the covers, it still does a lot of the same work, but the added benefit of with using the Octopus Deploy Package, Push steps allows for cleaner output logs during a build, package and push. One other benefit not mentioned previously is that with the new Build System you don’t have to check-in your packages folder that contains the OctoPack information (the NuGet Restore step takes care of that).


There probably some especially around references, but I can’t think of any that would hinder the overall usage.


The better approach of keeping a clean project/solution and letting TFS/VSTS do all of the work is just makes sense. Cluttering your project with excess complexity can make sustainable, reusable codebases hard to achieve. Coupled with that is the fact that over complicating your solution can also cause other developers within your team to have trouble trying to build the solution locally.

CI/CD with TFS/VSTS and Octopus Deploy

Building off my previous posts here and here about building multiple projects within a solution and troubleshooting packaging of your projects. We will now delve deeper into the Continuous Integration and Continuous Deployment pipeline that can ease the tensions of Deployment Aversion. Using two very powerful tools in your arsenal, you can streamline your deployment process and ensure that you are getting the biggest bang for your buck. Up until now we have identified our pipeline components and we have successfully, albeit manually, deployed our sample codebase to our destination servers. The question now is: can I streamline my pipeline using good ALM practices?


  1. You have your source code checked in
  2. Your build definition is in place
  3. You either have a partial Octopus Deploy project or similar

Setting up for success

Lets double check our Octopus Deploy Project. We will need to ensure that we have certain steps, configuration values, and settings.

Lets start out with cloning our already working deployment project.


Save our newly cloned project. We will get back this Octopus Project in a moment, but first we will need to do somethings with our TFS/VSTS build definition. In this example, we will expand upon our previous build definition and clone it. Why? Well first we know that it works and second it is easier to take away unneeded steps instead of trying to add and configure steps from memory. Cloning a Build Definition is very simple and straight forward. The below gif shows you how its done.


Since our build definition is only queued manually we will need to go and make some changes to the Triggers for the build to allow it to perform the CI for the beginning of our CI/CD pipeline. We will edit our newly cloned build definition and choose the Triggers tab and then select.  For now we will just check the Continuous Integration and leave the defaults.


Now each time there is a change to the codebase and change is checked in, this build definition will start and finish with our packages being pushed to Octopus Deploy. One of the many benefits here with the use of Triggers is that you can have multiple build definitions that can focus on specific branches and with CI/CD working in your favor your can get your code to testers or others for faster consumption and approval.


CI/CD Option 1

You are probably asking yourself why there is an Option 1 and 2. Well the fact of the matter is that with Octopus Deploy Extension you basically  perform can perform all of the steps of packaging, creating a new release, and finally queuing up a deploy.  Essentially, TFS/VSTS is now a one stop shop for building and orchestrating  your build and deployment pipeline.

How to perform Option 1

Going back to our CI/CD build definition we need to add a couple of more steps. Click add build step and select and add Create Octopus Release and Deploy Octopus Release.


Now we see that for our entire build definition we can build, test, package, publish, create and finally deploy a release. If you noticed that there was Promote Octopus Release, that task can be used in other build/deployment scenarios. For now, though, we are just going to use the create and deploy release.


Editing our create Octopus release step, we see a lot of options that we will need to fill out. We fill out the Octopus Deploy Server connection endpoint, we select the project name. For now we are going to use the Octopus defined version number, but later we can change this to be what ever SemVer compatible version number we want to display. Finally in the Deployment section we select our initial environment we would like to deploy to. After that because I like verbose logging for troubleshooting, select the Show Deployment Progress. This will ensure that we have a complete set of logs for our End-to-End test.


One thing to note here is that you add a lot of data for your release notes. This great for troubleshooting or even attempting to rollback to an earlier series of changesets.

We are going to kick off a manual build/release creation first, but to make sure that we are putting everything in the correct order.


and the end result without doing anything special with Octopus Deploy.


and now our Web application is still functioning.


Let’s go and make a change now in our source code, check it in and then see what happens.

Made some changes to our index page.


checked it in


Synced with the master (because this is a Git repo)


Build kicked off


Build finished and deployed


Now to validate against the working web application.

It worked. I made the change and the change was “automatically” reflected on the site. Now keep in mind this is simple way to perform this function. If you needed to perform a rolling deployment, or some other type, it would make sense t to adjust for Octopus Deploy process to accommodate those requirements.


CI/CD Option 2

Option 2 involves some manipulation of Octopus Deploy and not so much TFS/VSTS. If we leave what we were currently performing in TFS/VSTS from my previous post we will need to work and manipulate Octopus Deploy a little more.  So following the same example as above we will clone both the TFS/VSTS build and the Octopus Deploy project/build definitions, to ensure that we have a clean break from what worked  to what could work.

How to perform Option 2

There are a couple of items that we need to accomplish here. Because we have already cloned both the Basic Build definition and the Octopus Project, we will work with those cloned items in this mini exercise. For now we are not going to touch the clone of the build definition because it is generic and just performs the build and then packages and publishes the package( s ) to our Octopus Deploy internal NuGet repository.

First we will need to change how our Releases get created. After selecting your Process, find the Automatic Release Creation in the left hand side of the page. Select Project Triggers, this has moved from previous versions of Octopus Deploy and then select the create a release when a package is pushed.




Then we need to change the behavior of our Lifecycle (more information is here). The important fact here is that if we wish to stay away from modifying our build definition to automatically deploy we can do it within our Lifecycle. One key note here is that the default lifecycle allows for deployments to created environments. While this is not ideal, there is a way for us to create and manipulate a new Lifecycle to benefit our needs. For our cloned project we will need to “Choose a different lifecycle”.


Special note here is that the lifecycle once applied, will only be used by your deployment project going forward. Any previous releases that were created will use the previously assigned lifecycle. This can be confusing, but it makes sense because Octopus Deploy takes a snapshot of all of the variables and state of your project for each deployment.

Let’s go ahead and create a new lifecycle:

Click Library->Lifecycles-Add lifecycle


Then we need to give the new lifecycle a Name and add a Phase.  This is important because the phase helps us describe what we wish to do in each step of our deployment.


Adding a Phase means that you are going to be describing what happens to your deployment. After you click the Add Phase you will get this:


Give it a name and then click Add environment


Initially you will see and empty drop down box and a couple of radio buttons. Select your dev or other environment that you wish to deploy to first and then select Deploy automatically to this environment as soon as the release enters this phase. Then click Add.


Now it should look like this:


Click Save and we are almost there.

You can can multiple phases and multiple environments to each phase depending on your need. For this exercise, we are putting out the CI/CD basics that you can build upon for your future development and deployment efforts.

Now that I have created that lifecycle, I need to perform a couple more things to get everything wired up and in sync.

First, I need to go back to my project and change the lifecycle from Default to CI-CD Option2.


The next thing we have to do is change the Automatic Release Creation under the Project Triggers.  Warning here is that if you have an external NuGet or other repo that you are dealing with this option will most likely not work for you.  There may be an option 3 workaround, but I haven’t gotten to that state yet.

Click on Project Triggers and then


image check the “Create a release when a package is pushed to the built-in…” make sure you select the deployment step that contains your package, then click Save


Let’s test this out.


Our build was successful.


And it just works now.



From Option 1 it is clear that you can just use the Create Release step and not be dependent on both the Octopus Create Release and Octopus Create Deployment. Either option is viable and can be used interchangeably depending on your needs. You can have build definitions that work on specific branches of code for CI/CD and then use the promote package to your higher environments. The above technique while focused on a simple web application can be used for more complex deployment scenarios and applications.

Troubleshooting _PublishedWebsites Errors

Leading up to my next post about performing a full CI/CD pipeline with TFS/VSTS and Octopus Deploy, I came across an interesting “glitch” with the prescribed use of the _PublishedWebsites folder from my earlier post. Something was missing or misplaced during either the build or the packaging of my example Visual Studio solution. Using my basic troubleshooting skills, I found that not only were things missing from my packages, but there were items that were definitely misplaced.

(This post is looking at TFS2017/VSTS/Visual Studio 2017 and an OOB MVC Web Application built from a .NET Framework template of 4.5.2)


After performing a test deployment of my created packages namely the ones that are related to WebApplication and WebApi’s (in extension to those that use the Web.Application.targets for MSBuild), I did a simple test of trying to view the web site.  Keep in mind that viewing the website via F5 within Visual Studio will be perfect, but when you are attempting to utilize the “single source” of truth (TFS/VSTS) and building and moving your code through a CI/CD pipeline with MSBuild challenges can arise.

If you followed my previous example and created a release/deployment in an Octopus Deploy project, you probably would have found this:


If you are scratching your head, so was I! _PublishedWebsites should have had everything I needed to run a web application, correct?  What is missing that I need to include possibly?

Digging back into my TFS instance, I slowly came to the realization that the _PublishedWebsites didn’t really contain all of the items required to run a WebApplication or other related project. Case in point is that in my previous post about performing builds and packaging on multiple projects, I found that the “roslyn” folder  was nowhere near _PublishedWebsites. So how could roslyn folder be in an incorrect.  If I removed _PublishedWebsites, the pages, css and other items would be gone and I would still have an issue.

Web Application or Web Application-like build output


vs Windows Service build output


But then the problem starts to compound itself with:

  • How can I keep my package small and minimalize duplication of files in unwanted directories? –According to Parker’s Packaging Paradigm
  • Why does this seem to just work when I publish a WebApplication directly from Visual Studio?

The first question is more about what I want to discuss and the second question is a philosophical one that expounds upon the larger differences between small and larger teams that approach development/deployment from different angles. When you are a large organization it makes sense from an auditability standpoint to have a central way for you check-in, build, and deploy your codebase.


Scratching my head some more I went to Bing/Google to see if someone else had:

  1. Experienced what I was experiencing.
  2. Provided an easy repeatable fix.

Apparently not, I did this search and I found a lot of different ways to remove the issue. “Hacking” the config files in the project, “Hacking” the project file, uninstalling packages, etc.

All of these were seemingly kludgy. Couldn’t there be another approach to this? I had to try.  The KISS method was beating down my door.

When I tried to perform the “fixes” described in my searching and then performed a deployment and subsequent refresh of the web application I would still get an error. Missing .NET assemblies, complier errors. Something wasn’t right WebApp-ville.


So what could I do? I had to prove that I could simply keep the VS project clean and still be able to push my prescribed changes through to my Octopus Deploy instance and finally view my results.

Resolution (?)

Warning, this may work for some, but not all projects.

Going back to my original build definition, I had to come up with a strategy that could work for the issues that I was facing. How do I keep the project clean? Meaning that I do not add or take away from any of the .csproj files or .sln file. Finally how can I minimize my project package footprint without losing integrity? Meaning what would I need to do keep a small enough package foot print with all of the correct dlls and other appropriate files.

The partial answers to the questions were: I would have to move the roslyn folder and then I would have to clean up the extraneous files that were copied into the bin directory by the MSBuild

 Creating directory “E:\TFSBuildAgent\vsts-agent-onprem\_work\3\a\DemoLabs.WebApi\roslyn”.
Copying file from “E:\TFSBuildAgent\vsts-agent-onprem\_work\3\s\packages\Microsoft.Net.Compilers.1.3.2\tools\csc.exe” to “E:\TFSBuildAgent\vsts-agent-onprem\_work\3\a\DemoLabs.WebApi\roslyn\csc.exe”.

How it’s done

Opening up my build definition, I added two new PowerShell steps and placed them after my Publish Artifacts task and before my Octopus Package tasks. The first step is to move the roslyn folder to its intended location and the second task was for cleaning up the csc/roslyn related files that were copied into the bin directory during the build process.


The first step of moving the roslyn folder. Here we edit the PowerShell step to be an Inline Script. This could be done another way with a PowerShell script referenced within the codebase, but it was simpler to leverage the power of the build/orchestration steps.image

Here is the code:

 Write-Host "Moving the roslyn directory"  
 $folders=gci $env:BUILD_ARTIFACTSTAGINGDIRECTORY -Recurse -Include "roslyn","_PublishedWebsites"  
 for($d=0;$d -lt $folders.Length;$d+=2)  
   if($folders[$d].Parent.Name -eq $folders[$d].Parent.Name)   
     if($folders[$d].Name -eq "roslyn" -and $folders[$d+1].Name -eq "_PublishedWebsites")  
       move-item -Path $folders[$d].FullName -Destination "$($folders[$d+1].FullName)\$($folders[$d+1].Parent.Name)\bin\"  

Now to get rid of the duplicate files. Because the build actions copy roslyn folder contents directly over to the bin directory the web application complains that it cannot find roslyn folder so if we move the roslyn folder then we will end up with duplicate files one set in the roslyn folder and the other one level up in the bin folder.


 Write-Host 'Removing duplicates from bin directory'  
 $folders=gci -Path $env:BUILD_ARTIFACTSTAGINGDIRECTORY -include "bin" -recurse  
 foreach($folder in $folders)
 if(Test-Path "$($folder.fullname)\roslyn")  
   $Ref = Get-ChildItem -Path $folder.FullName  
   $Diff = Get-ChildItem -Path "$($folder.FullName)\roslyn"  
   $dupes = Compare-Object -ReferenceObject $Ref -DifferenceObject $Diff -Property Name,Length -PassThru -ExcludeDifferent -IncludeEqual   
   foreach($dupe in $dupes)  
     del $dupe.fullname  

Next save the steps in your build definition and queue up a new build.


Now that we are good to go, we can check our build artifact location so that we can view the contents of our NuGet package. Once that is done we can see that the library for our NuGet packages have been updated too.

Downloading the package from Octopus we can just change the file extension to .zip and open up the file to see the contents as shown here.


Now to confirm that everything is working the way it should I created a new release in Octopus Deploy and Deployed it.




Now that we have a successful deployment, lets look if all that work was worth it.


At the end of the day, without editing the Visual Studio project and leveraging the various utility steps within TFS2017/VSTS, you can “fix”/ “clean” your project for a better, smaller, cleaner pipeline solution.

Next time I’ll start the process of getting your web application and other projects to the next step towards Continuous Integration and Continuous Deployment.

Building Multiple Projects and Pushing to Octopus Deploy with Build vNext

In some cases there is need to have multiple projects within a solution that each require being packaged and then published or pushed to your repository of choice for deployment.  Typically from a Web Application perspective you could have Web Services, Web Api, and finally the Web Application itself.

What I am about to detail for you is performing these actions with success using TFS 2017 and Octopus Deploy.  These same actions can be applied to Visual Studio Team Services (VSTS). Be warned that this technique will not apply to older XAML build definitions.


We are going to assume that you have basic solution with .Net MVC Web Application, MVC WebApi, and Windows Services projects. You can more than this, but for the purposes of this demonstration this is what we have. Something like this:


And we have the solution checked into Source Control like this:image

You will also want to make sure that you have Octopus Deploy extension installed or applied to your account or TFS instance.

Where to Start

Well the most common thing here is that you will need to create is a basic Visual Studio vNext Build Definition. Log into your Project and Navigate to Build & Release and then click New in the right hand corner.


Now we are going to Scroll down and choose a Visual Studio build template. I won’t go into the details of all the different templates are, but for this exercise we will choose Visual Studio because it is a very quick way for you to get started.

image image

Now that we have our basic build definition in place it should look something like this.


Now save what we have so that we can start with the meat of this exercise. The next little preparation that you have to do make sure that you create a Service Endpoint for your Octopus Deploy input. I will be using Built-in Nuget Repository, but can also use and external NuGet repository of your choice. The fields that you will need to fill out are listed at points 3, 4, 5. You need to give the connection a meaningful name that you can remember. You must provide a url to the octopus instance that you intend to publish to and finally you will need to have an API key with the minimum permissions to publish to the Octopus internal NuGet Repository. Rule of thumb here is to create or have a Octopus Service Account that has the appropriate permissions.


How its done

We now have the basics set up. So lets get into the requirements for getting multiple projects to be compiled, packaged, and pushed to Octopus Deploy. Previously you would have to have a lot of pre-work done to your actual project/solution, e.g., adding OctoPack, creating a nuspec file, etc. With the new build system and its extensibility, it makes your projects/solutions much cleaner and much simpler to use especially if you add more projects to your solution that require packaging and publishing later on.

Edit your saved build definition and select Build Solution step. We need to make an adjustment here to ensure that

  1. We get a separate folder for each project
  2. We ensure our Output Directory for the solution is pointing to our ArtifactsStagingDirectory

Like so.


What this will do will ensure that the build output from the step will go to the build artifacts directory on your build agent, similar to this:image

The “a” in this case means Artifact.

We will not worry about the Test Assemblies step right now and we can also leave the Publish Symbols and Copy Files Step and the Publish Artifact alone for now. The point of this post is to get your projects built, packaged, and published for deployment. Next what we need to do is add another few build steps to our Build definition.

Click on Add Build Step


And then we are going to Add the Octopus Package Application step, one for each of our projects that we wish to publish. Then we are going to add the Octopus Push Package(s) to Octopus step to finish up our Build.


Now for each of the projects you will need to edit the Package Id, Version, but more importantly you need to know what and where your source and destination path is going to be. If you don’t these two items correct, your build will fail.


We do know that our output is going to the artifacts directory and we know that each project is going to have its directory. But a critical component here is what the Octopus Step is going to do. It is important to note that we want, must, need to do is

  • Keep your package small
  • Take only the most relevant codebase

I have seen so many teams in the past and currently take everything from their source tree and package it into a 100MB+ file and attempt to push it to Octopus Deploy then deploy all of source tree to production. It is really hard to contemplate the security risks of putting your entire codebase for someone to steal and compromise.

For web applications and web api projects there is typically a _PublishedWebsites folder, this folder contains a reduced build output with only the most relevant dlls, config files, and other content for your web application or web api to run. Everything else is irrelevant and if you choose the folder at the higher level you will essentially get two copies of the same code base in your package. Citing the two points above, keep the package small and only package the most relevant code! This doesn’t mean only the files that have changed! This means your most minimal amount required run your application.

Now that we have edited the Package steps, we can move on the other important final element to this post, which is the Push Packages to Octopus.

Select the Push Packages step and you will notice that you need to edit the Octopus Deploy Server (this is the Service Endpoint that created earlier) and the most important item is the the location of the packages.  Earlier in this post I told you that you didn’t really need to worry about the Publish Artifact step. The drop directory is where we are going to put our package after the build and packaging steps and that same directory is going be where we find those packages and publish them to our Octopus Deploy NuGet feed.


Now we are done! Save all the work that we have done and then queue a build. Don’t get frustrated if you your build doesn’t register success. Take a look at the logs for each build you attempt and step through to ensure that your folder paths are correct. Ensure that the API Key has correct permissions to publish to the Octopus NuGet Repository.

If you do have a successful build it tell you that it performed the compilation, packaging and publishing of the packages. Similar to this.




Checking each step to ensure that the logs are telling me what is happening.




and finally verified it has been pushed to our Octopus Deploy instance.


Next Steps

Now that I showed you how get your build to package and publish to Octopus. You may wonder what about CI/CD? or what can I do now?

To answer the second question, you can actually go ahead and make this a task group and save it for other teams to utilize in the future. You can also use this technique for other project types as well. Once you understand how you can create a package with the Octopus Deploy Extension, you can change the build definition to a CI/Timed Build and get those packages published for the next phase.

To answer the first question: That is for another post.