DevSecOps Playbook

This is a step-by-step guide to implementing a DevSecOps program for any size organization


DevSecOps Playbook – Version 1.0

This playbook will help you introduce effective DevSecOps practices in your company, regardless of size. We provide explicit guidance and actionable steps to introduce security controls, measure their effectiveness, and demonstrate value for money to your business leaders. Following this playbook will help teams build materially more secure applications, and that in the end, is the intent.

The DevSecOps-Playbook is a github repository by Paul McCarty

Some background

This playbook was inspired by several documents and I want to call them out here:

The “Minimum Viable Secure Product”, or MVSP is a project that I have profound respect for. The MVSP is a great way to determine how mature an organization’s security practices are.

Timo Pagel’s amazing “DevSecOps Maturity Model” or DSOMM is a project that I only found recently. There is some overlap between the DSOMM and this document and you should definitely browse through the DSOMM and explore the different maturity levels.

Sponsors

Sponsored with by

securetack

Shift Left

All companies and applications are unique. Blanket statements like “Shift Left” aren’t helpful without context. Enterprises and startups have different tech stacks, funding, workforces, regulation, and more. Context matters, and this document provides a roadmap to use your context to determine your next DevSecOps destination.

We want to secure our applications, from cradle to grave. To do this, we have created five categories that cover the lifecycle of a software application. We’ve also added a Compliance Addendum for anyone that is interested in aligning with a specific compliance framework.

Priority and Difficulty explained

We use two rating systems: Priority indicates the order you should implement controls and Difficulty indicates how hard implementation is for this control.

devsecops playbook

The Checklist


Development Environment

The developer’s laptop is where most of the magic happens, but also where most of the problems are introduced. If you want to shift as far left as you can this is where you want to land much of your embedded security.

ControlNamePriorityDescriptionDifficultyMaps to security frameworks
1.1Secure Code Training2Developers who receive Secure Code Training are less likely to introduce security bugs, be aware of tooling that can support them, and design systems with security in mind.MediumCIS8APRA234NIST 800-53BSSDF1.1
1.2Source Code Versioning1Version Control Systems introduce peer review processes, an auditable history, and consistent work patterns between software engineers.EasyAPRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
1.3.gitignore1.gitignore files help prevent accidental commits of sensitive, debug, or workstation specific dataEasyAPRA234CIS8NIST 800-53BSSDF1.1
1.4Pre-Commit Hook Scans2A Pre-Commit Hook for security scans provides timely feedback to engineers and helps to prevent vulnerable code being introduced to a repositoryEasyAPRA234CIS8NIST 800-53BSSDF1.1
1.5Commit Signing2Sign all commits to verify that the author is genuineEasyAPRA234CIS8NIST 800-53BSSDF1.1
1.6IDE plugins2Most IDE’s support the use of third-party plugins, and devs should implement these tools to highlight security issues as they happen in realtime while they are programming.EasyAPRA234CIS8NIST 800-53BSSDF1.1
1.7Local Software Composition Analysis1Helps you find and fix libraries with known security issuesEasyAPRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
1.8Local Static Code Analysis2Helps you find and fix security vulnerabilities in your source codeEasyAPRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
1.9Local Sensitive Data Analysis1Audits your repository for secrets, credentials, API keys and similar in dev environment. Secrets stored in source code are visible to other peopleEasyAPRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
1.10Application Baseline3Create a “recipe” for building the application from ground up that takes into consideration its risk and compliance requirements, data sensitivity, stakeholders and relationships with other systems, as well as its technical components.MediumAPRA234CIS8ISM GSDNIST 800-53BSSDF1.1

Source code management (SCM)

Most companies now store their source code in cloud based repositories like GitHub, Bitbucket or Gitlab. Even if you don’t, you will use a centralized place for your software engineers to store their code. Centralization and versioning means that these developers can work together without (mostly) stepping on each others toes. Joe and Molly can both be working on the same component, file or function but their changes won’t necessarily break the other one’s changes. SCM is also a GREAT place to deploy security functions like server side git hooks and multi-factor authentication for your developers!

ControlNamePriorityDescriptionDifficultyMaps to security frameworks
2.1Source Code Management1Use a centralized source code management (SCM) system like Bitbucket, GitHub or GitlabEasyAPRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
2.2User Roles1Create unique user and team roles so that access to source code can be tailoredEasyAPRA234CIS8ISM GSDNIST 800-53BSSDF1.1
2.3SSH2Use the SSH protocol to access your repositories instead of HTTPSEasyAPRA234CIS8ISM GSDNIST 800-53BSSDF1.1
2.4GPG Key2Add a GPG key to your central SCM provider. This will guarantee that the person commiting changes to source really are who they say they areEasyAPRA234CIS8ISM GSDNIST 800-53BSSDF1.1
2.5Multi-Factor Authentication1Make sure all developers use multi-factor authentication (MFA) when pulling, fetching or pushing code to remote. This is especially important if you use company email as your login for SCMEasyAPRA234CIS8ISM GSDNIST 800-53BSSDF1.1
2.6Server Side Git Hook3Utilize a server side git hook like update or post-receive hook to run automatic scansMediumAPRA234CIS8NIST 800-53BSSDF1.1
2.7Developer Collaboration2Use collaboration tools to document the changes to a software applicationDifficultAPRA234CIS8NIST 800-53BSSDF1.1
2.8Pull Requests1Enforce pull or merge requests so all code is verified by team lead or senior engineerEasyAPRA234CIS8ISO27001NIST 800-53BSSDF1.1
2.9Peer Reviews1Enforce peer reviews by software engineers colleagues to increase code quality and securityEasyAPRA234CIS8ISO27001NIST 800-53BSSDF1.1
2.10CODEOWNERS1Create a CODEOWNERS file that identifies people and teams that own specific parts of the repository and should be consulted via PR when those parts of the repo are modified.EasyAPRA234CIS8ISO27001NIST 800-53BSSDF1.1

CI/CD Pipelines and Automation

Modern web applications are built using modern continuous integration and deployment processes. This means that you run tests specific to whatever environment you are pushing to whether that’s DEV, STAGING or PROD.

ControlNamePriorityDescriptionDifficultyMaps to security frameworks
3.1CI/CD Pipeline1Implement a CI/CD pipelineMediumAPRA234CIS8ISM GSDISO27001SSDF1.1
3.2Application Environments2Create separate environments for dev, staging and prod, and treat each as independent with its own data, testing and requirementsMediumCIS8ISM GSDISO27001SSDF1.1
3.3Application Data Separation3Make sure that dev and test environments are not using the same data as production. If the use of live data is required then make sure that data is anonymized.DifficultCIS8ISM GSDISO27001SSDF1.1
3.4CI/CD Administration3Create and enforce user or team roles so that only the appropriate people can change or disable tests and deployment requirementsMediumCIS8ISM GSDISO27001SSDF1.1
3.5Credential Store1Create a secure encrypted place to store senstive credentials like passwords, API keys, etc.MediumAPRA234CIS8ISM GSDNIST 800-53.2bSSDF1.1
3.6Centralized Software Composition Analysis1Scan source code for vulnerable libraries and open source software from within a CD stageEasyAPRA234CIS8ISM GSDISO27001NIST 800-53.2aSSDF1.1
3.7Centralized Static Code Analysis2Scan source code for vulnerabilities in the source code itself from within a CD stageEasyAPRA234CIS8ISM GSDISO27001NIST 800-53.2bSSDF1.1
3.8Centralized Sensitive Data Analysis2Scan source code for secrets, credentials, API keys and similar from within a CD stageEasyAPRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
3.9DAST3Scan running application for vulnerabilitiesMediumCIS8ISM GSDISO27001NIST 800-53BNIST 800-53BSSDF1.1

Deployment

Applications are deployed somewhere whether that’s an AWS Lambda, S3 bucket or some old crusty server in the corner of the server room. In any case, DevSecOps best practices mean that you need to include that deployment location in your processes.

ControlNamePriorityDescriptionDifficultyMaps to security frameworks
4.1Valid SSL Certificate1Create and use a valid SSL certificate for each application URL, or implement a wildcard certEasyAPRA234CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
4.2Encrypt Traffic1Encrypt all traffic that’s public facingMediumAPRA234CIS8CIS8ISM GSDISO27001NIST 800-53BSSDF1.1
4.3Redirect to HTTPS1Configure web service to redirect all inbound requests to port 80 to the secure HTTPS endpointEasyCIS8ISM GSDISO27001NIST 800-53BSSDF1.1
4.4HSTS1Enable HSTS in your webserver, load balancer or CDNEasyCIS8ISM GSDISO27001NIST 800-53BSSDF1.1
4.5CSP1Enable content security policy (CSP) in the webserver, load balancer or CDNEasyCIS8ISM GSDISO27001NIST 800-53B
4.6Use Current Software1Use the most recent versions of application components, languages, frameworks and operating systemsDifficultCIS8ISM GSDISO27001SSDF1.1
4.7Alternative Deployment3Have tested and working alternative way to deploy changes to your application other than using your standard process with GitHub or Bitbucket in case they go down. This must include the ability to push to PROD from local in emergencies.DifficultCIS8NIST 800-53BSSDF1.1
4.8security.txt1Create a security.txt file in the root of your application so people know how to contact you about security issuesEasyCIS8ISM GSDSSDF1.1
4.9X-Forwarded-By2Configure your webservers, load balancers & web proxies to include the X-Forwarded-By: headerEasyAPRA234 ATM D-2-d-iCIS8NIST 800
4.10Logging1Collect application logs in realtime and send to centralized storage or SIEMMediumCIS8 16.11APRA234ISM GSMNIST 800SSDF1.1
4.11WAF2Implement a web application firewall (WAF) to protect your application from known attacksMediumAPRA234CIS8NIST 800-53.2a
4.12CDN3Use a content delivery network (CDN) whenever possible to add availability and security to you applicationsMediumAPRA234CIS8ISM GNNIST 800-53.2a
4.13Harden Operating System2Harden operating system using industry best practices from CIS, ISM, etcDifficultCIS8ISM GSMISM GOSHSSDF1.1
4.14Encrypt Storage3Encrypt all filesystems, disks and cloud storageMediumCIS8NIST 800-50bSSDF1.1
4.15SBOM3Generate a real-time software bill-of-materials (SBOM)MediumCIS8ISM GSDNIST 800-53BSSDF1.1
4.16Monitor Application1Monitor your application in real-time so you know when its state changes for the worse (or better). This includes uptime, performance and security monitoringMediumCIS8NIST 800-53BSSDF1.1

Organization

People don’t deploy applications, organizations do. Some steps in the DevSecOps playbook need to be owned by the Organization itself.

ControlNamePriorityDescriptionDifficultyMaps to security frameworks
5.1Penetration Testing2Have your application pentested regularlyMediumCIS8ISM GSDNIST 800-53BSSDF1.1
5.2Threat Modeling3Build a collaborative way for developers and security staff to understand the threat landscape for an individual applicationDifficultCIS8ISM GSDNIST 800-53BSSDF1.1
5.3SIEM3Implement a SIEM and send all application, system and cloud logs to itDifficultCIS8NIST 800-53BSSDF1.1
5.4Attack Surface Management1Identify public facing resources via automationEasyCIS8CIS8NIST 800-53BSSDF1.1
5.5Sovereignty1Require that all code is written in, stored in, or otherwise served from a location and/or sovereignty that aligns with your org’s requirementsMediumISM GCSRISO27001
5.6Vulnerability Disclosure1Create and publish a set of procedures to let people contact you when they find security issues in your appEasyCIS8ISM GSDSSDF1.1
5.7Bug Bounty3Setup a bug bounty program to incentivize security researchers to tell you about vulnerabilities they findMediumCIS8ISM GSDNIST 800-53BSSDF1.1
devsecops controls

Compliance – Security Framework Reference Material

Because this is meant to be a manifesto about how to do DevSecOps, we have to be cognizant that there are three groups of people that this affects: Developers, Operations and InfoSec. Historically, there are many compliance frameworks that address the InfoSec community and to a lessor extent Operations teams. But software development was never mentioned until only recently. Understanding this, I wanted to note that most commonly there is only one section of a particular compliance framework that relates to software development. This is a list of the specific sections of several larger compliance frameworks that deal with software development specifically.

  • NIST 800-53b SA 10, 11, 15, 16 and 17
  • NIST 800-218 “Secure Software Development Framework”
  • ISO 27001 Annex A Section 14
  • CIS Section 16
  • Australian ISM “Guidelines for Secure Development”

Below you will find links to several security frameworks that align with this document. I have personally spent many years implementing CIS controls into my application environments. CIS is a wonderful framework as its very prescriptive and easy for an engineer to understand. This is not to say that CIS controls are easy to implement. They are not! Regardless, you can’t deny the ubiquity of ISO27001 and SOC2 and I want this document to help orgs looking to meet those requirements as well. In fact, SecureStack has started a SOC2 program and in parallel to writing this document I am busily mapping SOC2 requirements and will eventually add them to this document.

I had a number of Australian friends suggest that I tackle the Australian ISM and APRA CPS 234, so I’ve added both of these as well. This is a work in progress and I encourage anyone that is interested to jump in and suggest mappings. You can add an issue in GitHub or simply create a PR.

NIST 800

NIST 800-218 (2022) “Secure Software Development Framework” (SSDF) version 1.1: https://csrc.nist.gov/publications/detail/sp/800-218/final

NIST 800-53b (2021): https://csrc.nist.gov/publications/detail/sp/800-53b/final
Control Families via HTML: https://csrc.nist.gov/projects/risk-management/sp800-53-controls/release-search#!/families?version=5.1

  1. SA-11: Developer Testing and Evaluation
  2. SA-15: Development Process, Standards, and Tools
  3. SA-16: Developer-Provided Training
  4. SA-17: Developer Security and Privacy Architecture and Design

NIST 800-92: https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-92.pdf
NIST 800-95: Guide to Secure Web Services (2007): https://nvlpubs.nist.gov/nistpubs/Legacy/SP/nistspecialpublication800-95.pdf

CIS Critical Security Control 16: Application Software Security

The Center for Internet Security is an organization that has been providing very prescriptive security controls since 2000. There are a total of 18 security control groups with section 16 being the group we will be referencing for this document.
https://www.cisecurity.org/controls/application-software-security

It’s not the focus of this document, but CIS maintains an amazing set of benchmarks and build playbooks for most operating systems. I have been using these templates for years and they are a great resource: https://www.cisecurity.org/cis-benchmarks/

Australian ISM

The Australian Cyber Security Centre has authored a document called the “Information Security Manual” or ISM as its commonly called. The landing page for this document is https://www.cyber.gov.au/sites/default/files/2021-12/Information%20Security%20Manual%20%28December%202021%29.pdf.

This document is large and has a very broad scope. You can download the complete ISM at https://www.cyber.gov.au/sites/default/files/2021-12/Information%20Security%20Manual%20%28December%202021%29.pdf

In late 2021 the ACSC released the “Guidelines for Software Development” (GSD). These are a general set of guidelines for embedding secure coding practices into an organization. These guidelines are far from authoritative and are not very prescriptive with my favorite snippet from the GSD being this little gem: “Platform-specific secure programming practices are used when developing software, including using the lowest privilege needed to achieve a task, checking return values of all system calls, validating all inputs and encrypting all communications.” Is that a catch all or what?! Wow! Regardless, I am respectful of the energy that went into this set of guidelines and will continue to try and bring visibility to it as much as I personally can.

You can find the GSD here: https://www.cyber.gov.au/acsc/view-all-content/advice/guidelines-software-development

APRA CPG 234

The Australian Prudential Regulation Authority (APRA) is part of the Australian government and is charged with regulating the financial industry. It published the “Prudential Standard CPS 234” document in 2019 which outlines high level information security requirements.

This document is organized in an unusual way with 8 “attachments” at the end of the doc. It is in these attachments that the security controls and expectations are laid out. You can find the APRA 234 document at https://www.apra.gov.au/sites/default/files/cpg_234_information_security_june_2019_0.pdf.

ISO27001 Annex A.14: System Acquisition, Development & Maintenance

ISO27001 is an international standard on how to manage information security. It measures the maturity of an organization with a total of 114 controls spread across 14 groupings. ISO27001 is built on the principle of gathering the documentation around these security controls in an information security management system (ISMS).

Group A.14 revolves around the acquisition and development of IT systems. It is the only part of the ISO27001 specification that mentions specific

What’s been mapped so far?

  •  mapped all of ISM GSD
  •  mapped all of SSDF
  •  mapped CIS section 16

What’s left to do?

  •  map CIS section other than 16 to all items
  •  map remaining ISO 27001 Annex 14
  •  create compliance section for ISO 27002
  •  map brand new ISO 27002 controls
  •  map remaining NIST 800-53
  •  map remaining ISM infrastructure

About the author

My name is Paul McCarty and I’m the founder of SecureStack. I created this document as a way to capture in one place the steps I took to implement DevSecOps functions into my team. If you have any questions you can contact me at hello@securestack.com or on twitter @eastside-mccarty