Ok, we have several different scenarios and let’s say you’re the boss. In each of these scenarios we can assume that CI & CD are going to be mandatory to achieve the team’s greatest potential to deliver. So the question I’ll be attacking is, do we roll our own with Drone.io, TeamCity, continuous integration and delivery or do we get more advantage from utilizing a managed solution like CircleCI, Codeship, or related offerings. The following are the scenarios in relation to team size and project that you’re heading up:
Scenarios
Scenario 1: You have team of 4 people. All developers by trade and experienced. The team may, about 20% chance in the coming 3-6 months, grow to ~8 people, with 1-2 of those being automation experts that could prospectively manage CI/CD systems. Currently the project is just starting, and the decision on what language and tech stack has been determined and initial coding will be starting in the coming week. Currently, there is no CI/CD setup for this project.
Scenario 2: Another team has 33 developers, 4 are site reliability with 2 more working with them frequently on systems automation. No real growth for the team in the next 2-3 years. The application these developers work on has new feature development and maintenance, lot’s of ongoing maintenance. Currently there is also a fair amount of technical debt associated with the code base that needs paid off for each new feature implemented, or any significant changes during maintenance of the software. This team currently has no CI or CD of the application setup.
Scenario 3: The third team is 4 people, pairing, and they have the highest output and quality of the rest of the teams. Largely because they work well together but also they have minimal technical debt and can move quickly. The application has been under development for about 3 months and they currently have a CI process setup with Drone.io. No deployment is setup yet nor is any infrastructure or other tooling setup beyond what they’ve already setup with the Drone.io solution.
Scenario 4: Fourth scenario is a large team of 232 people. There are 9 organizations with multiple people in each of those organizations, split out to developers, project management, product development leads, and a core architecture team. Among the 232 there are also 17 people dedicated to automation and infrastructure management. This team manages one huge monolithic application that currently is built with manual build processes that are partially scripted. Of these build processes there are 11 of them, which then must be combined via some more scripts and more manual processes that are then pushed to a test development environment, where a subsequent 4 people work full time testing the test development environment.
Each of these scenarios has a clear and present need for continuous integration and delivery setup. Just for context, when I write continuous integration or continuous delivery I define these like this:
Continuous Integration - Wikipedia - Thoughtworks
…is the process of bringing together the assets of an application or service project to compile that code into a binary. In the case of JavaScript or other non-compiled language this process would include any transpiling or gathering of assets that are needed in order to execute the code via the integration process. In most cases I include unit testing, and sometimes certain types of or styles of behavior tests in a project to be included under the process of continuous integration.
Continuous Delivery - Wikipedia - Thoughtworks
…is the process of events that must be completed, in an automated fashion, after the continuous integration process to take code or a binary from the transpiled or binary state into a production environment in which it will be used or executed. This is fairly easy to outline with a services or web application, since there is an obvious way in which this code needs to be called (via http/https in a browser) but some other applications, like a CLI or desktop application are slightly more diverse in the way they may be delivered. These later applications we can safely assume have a continuous delivery process that would vary widely but be something like being packaged into an installer (in the case of OS-X a .pkg file or in the case of Windows into a .exe or .msi file.)
Also an excellent resource is available here.
Solutions & Discussion for Scenarios
Scenario 1: Considering there are 4 people, prospectively 8 at the most over the next 6 months, the team really should not spend time on putting together, automating, and building out systems they don’t absolutely need to. The emphasis with a small team needs to be lean, mean, and focused on the mission at hand. Continuous integration and delivery will help delivery that, but putting it together isn’t directly related to business value.
What to do? Well, the quickest and simplest way to get started and enable a core focus on the mission is to use a managed continuous integration and delivery provider. This is a great case to use Codeship, CircleCI, AppVeyor, or a host of providers.
Scenario 2: This is an interesting scenario in that there are people tasked with automation and what would be the effort of continuous integration and delivery. However, there are questions to ask. There are three questions that immediately come to mind. These would likely have the most weight on a managed or self-hosted CI & CD solution would be:
Would running our own CI and CD save the company money?
Does the team need specific features or capabilities inherent in self-hosted CI and CD solutions?
Would the self-hosted capabilities versus the managed enable the team to deliver more effectively?
Let’s dig through each question.
The first, is the cost question. If the team had enough demand for builds or other integration and delivery automation it is feasible that costs could be fairly huge. I myself have seen price tags in the $40-50k a month range, which more than validates self-hosting build and delivery solutions to more finely manage or even decrease costs. However, I’ll add that $40-50k does only pay for work equating to a part time effort on this work. That doesn’t go that far either. By proxy, the cost is a valid question, but it isn’t all that it’s cracked up to be. I tend to lean heavily toward using a hosted solution even if it costs 10-40% more as that cost is usually worth it to refocus efforts on business value instead of CI and CD management. Let’s say, for this 33+ person team in scenario 2, that CI and CD costs will be around $40k per month via hosted offerings and about $25k if it’s self-hosted.
The second is likely the most important that would be asked of the three. Case in point the market has provided very few .NET build solutions that are managed. This has limited the ability to gain CI and CD advantages for startups that choose .NET. This has likely been one of many reasons .NET has taken a massive nose dive in popularity over the last decade among startups. Even among the more popular build service one can use, there are certain limitations that can stop a project cold. In these cases, a team can be forced into a self-hosted option. Let’s say, for scenario 2 however there isn’t a legitimate need for self-hosted control, but some of the team think it would be a nice to have.
The third question, is more inclusive of the aforementioned other two questions plus other criteria. But just as I mentioned the team thinks it would be a nice to have that begs another question. If the team had the extra control would that give them a boost or assistance with certain CI or CD automation, or will they get into situations where someone will end up with managing or spending significant time toying about with capabilities? Would that time be worth it in the end, or be lost time sunk into something that could work without the investment? Let’s say that having full control in this situation, as if we were so fortunate to get a % of productivity increase, would give the team a 5% speed to delivery boost and 2% less bugs.
Alright, I’ve conjured up those few thougths about each of those questions. Based on the posed end statements of each questions what would you do as the boss? What would I do? Since this is a team that is largely maintaining existing software and building some features, I’d also ponder what would keep them happier. Maintaining a CI and CD system in hours or gettinga self-hosted service, then I’d look at the over under of $40k cost per month hosted or $25k self-hosted, then nice to have thought behind self-hosted, and the upside of 5% speed to deliver and 2% bugs boost. In the end here, it would come down to developer happiness, and anything, I may even mix usage. For instance a bit of Codeship and Drone.io together. Both have a similar usage style so the learning curve would be huge, but similar giving the team a conjoined experience. I’ll leave this solution in the democratic vote of the developer team.
Scenario 3: This scenario is easy for me. If someone is spending more than 30 minutes a week dealing with Drone.io management, I’d push a shift to a managed provider. However since the Drone.io solution is already in place, if the only thing people are doing is committing code, getting builds, and have their code delivered then I’m a happy camper. Drone.io would stay as the CI and CD tooling for the team.
Scenario 4: Ok, wow, this is a big team to implement for. My immediate reaction is everything will be dependent on individual teams within the big team vote. There’s no reason to force a large team of this size to use a single solution, but there’s also many downsides to using to many different technologies to implement CI and CD. Let’s say the code this 200+ person team is working on is a single language, single stack, and singular deployment path, with just a few builds occurring per hour. If that’s the case then that would change my response toward a managed and hosted solution. But with this size, it’s likely that costs are much higher, build demand is dramatically higher, and the various internal security, admin, and related management of users, individual features, and the like lend to a self-hosted internal solution to give more control over customization specific to needs.
Summary: These are just a few of the thoughts, notions, and issues that need reflected upon to make a useful and effective decision about CI and CD tooling. These scenarios I posed are merely from the perspective of team size, organization, costs, and these higher level notions are CI and CD. In a future article I’ll dive into the specifics of how CI and CD are implemented on various platforms, and we can dig even deeper into why, what, who, where, and when to implement CI and CD for your team(s). Whatever the case, CI and CD are a fundamentally and vitally important first step or ongoing step of any project. If you’d like to discuss more, feel free to reach out to me on Twitter @Adron or contact me.
