How to Build an Effective Initial Deployment Pipeline

I love building things—what developer doesn’t? I love thinking up solutions to interesting problems, writing implementations, and creating beautiful code. However, what I don’t like is operations. Operations is everything not involved in building great software—everything from setting up servers to getting your code shipped to production.

This is interesting, because as a freelance Ruby on Rails developer, I frequently have to create new web applications and repeat the process of figuring out the DevOps side of things. Fortunately, after creating dozens of applications, I’ve finally settled on a perfect initial deployment pipeline. Unfortunately, not everyone’s got it figured out like I have—eventually, this knowledge led me to take the plunge and document my process.

Note, since it’s my pipeline, it’s also opinionated and suited to my needs; however, you can feel free to swap out anything you don’t like and replace it with whatever strikes your fancy. For my pipeline, we’ll use:

• GitLab to host code.
  • Why: My clients prefer their code to remain secret, and GitLab’s free tier is wonderful. Also, integrated free CI is awesome. Thanks GitLab!
  • Alternatives: GitHub, BitBucket, AWS CodeCommit, and many more.
• GitLab CI to build, test, and deploy our code.
  • Why: It integrates with GitLab and is free!
  • Alternatives: TravisCI, Codeship, CircleCI, DIY with Fabric8, and many more.
• Heroku to host our app.
  • Why: It works out of the box and is the perfect platform to start off on. You can change this in the future, but not every new app needs to run on a purpose-built Kubernetes cluster. Even Coinbase started off on Heroku.
  • Alternatives: AWS, DigitalOcean, Vultr, DIY with Kubernetes, and many more.

First, let’s recreate a typical application for someone who isn’t using any fancy CI/CD pipelines and just wants to deploy their application.

It doesn’t matter what kind of app you’re creating, but you will require Yarn or npm. For my example, I’m creating a Ruby on Rails application because it comes with migrations and a CLI, and I already have the configuration written for it. You’re welcome to use any framework or language you prefer, but you’ll need Yarn to do the versioning I do later on. I’m creating a simple CRUD app using only a few commands and no authentication.

And let’s test if our app is running as expected. I went ahead and created a few posts, just to make sure.

And let’s deploy it to Heroku by pushing our code and running migrations

$ heroku create toptal-pipeline Creating ⬢ toptal-pipeline... done https://toptal-pipeline.herokuapp.com/ | https://git.heroku.com/toptal-pipeline.git $ git push heroku master Counting objects: 132, done. ... To https://git.heroku.com/toptal-pipeline.git * [new branch] master -> master $ heroku run rails db:migrate Running rails db:migrate on ⬢ toptal-pipeline... up, run.9653 (Free) ... 

Finally let’s test it out in production

Pros

• Quick to set up.
• Deployments are easy.

Cons

• Not DRY: Requires repeating the same steps on every change.
• Not versioned: I’m rolling back yesterday’s deployment to last week’s isn’t very specific three weeks from now.
• Not bad-code-proof: You know you’re supposed to run tests, but no one’s looking, so you might push it despite the occasional broken test.
• Not bad-actor-proof: What if a disgruntled developer decides to break your app by pushing code with a message about how you don’t order enough pizzas for your team?
• Does not scale: Allowing every developer the ability to deploy would give them production level access to the app, violating the Principle of Least Privilege.
• No staging environment: Errors specific to the production environment won’t show up until production.

I’m going to try something different today: Let’s have a hypothetical conversation. I’m going to give you a voice, and we’ll talk about how we can improve this current flow. Go ahead, say something.

Say what? Wait—I can talk?

Yes, that’s what I meant about giving you a voice. How are you?

I’m good. This feels weird

I understand, but just roll with it. Now, let’s talk about our pipeline. What’s the most annoying part about running deployments?

Oh, that’s easy. The amount of time I waste. Have you ever tried pushing to Heroku?

Yeah, watching your dependencies downloading and application being built as part of the git push is horrible!

I know, right? It’s insane. I wish I didn’t have to do that. There’s also the fact that I have to run migrations *after* deployment so I have to watch the show and check to make sure my deployment runs through

Okay, you could actually solve that latter problem by chaining the two commands with &&, like git push heroku master && heroku run rails db:migrate, or just creating a bash script and putting it in your code, but still, great answer, the time and repetition is a real pain.

Yeah, it really sucks

What if I told you you could fix that bit immediately with a CI/CD pipeline?

A what now? What is that?

CI/CD stands for continuous integration (CI) and continuous delivery/deployment (CD). It was fairly tough for me to understand exactly what it was when I was starting out because everyone used vague terms like amalgamation of development and operations, but put simply:

• Continuous Integration: Making sure all your code is merged together in one place. Get your team to use Git and you’ll be using CI.
• Continuous Delivery: Making sure your code is continuously ready to be shipped. Meaning producing read-to-distribute version of your product quickly.
• Continuous Deployment: Seamlessly taking the product from continuous delivery and just deploying it to your servers.

Oh, I get it now. It’s about making my app magically deploy to the world!

My favorite article explaining CI/CD is by Atlassian here. This should clear up any questions you have. Anyways, back to the problem.