DevOps
{{short description|Integration of software development and operations}}
{{pp-pc1}}
DevOps is the integration and automation of the software development and information technology operations{{Efn|a portmanteau of development operations}}. DevOps encompasses necessary tasks of software development and can lead to shortening development time and improving the development life cycle.{{Cite web |last1=Courtemanche |first1=Meredith |last2=Mell |first2=Emily |last3=Gills |first3=Alexander S. |title=What Is DevOps? The Ultimate Guide |url=https://www.techtarget.com/searchitoperations/definition/DevOps |access-date=2023-01-22 |website=TechTarget |language=en}} According to Neal Ford, DevOps, particularly through continuous delivery, employs the "Bring the pain forward" principle, tackling tough tasks early, fostering automation and swift issue detection.{{Cite book |title=Building Evolutionary Architectures: Automated Software Governance |isbn=978-1492097549}} Software programmers and architects should use fitness functions to keep their software in check.{{Cite book |title=Fundamentals of Software Architecture: An Engineering Approach |publisher=O'Reilly Media |year=2020 |isbn=978-1492043454}}
Although debated,{{efn|Dyck et al. (2015) "To our knowledge, there is no uniform definition for the terms release engineering and DevOps. As a consequence, many people use their own definitions or rely on others, which results in confusion about those terms."{{Cite book |last1=Dyck |first1=Andrej |last2=Penners |first2=Ralf |last3=Lichter |first3=Horst |chapter=Towards Definitions for Release Engineering and DevOps |date=2015-05-19 |title=2015 IEEE/ACM 3rd International Workshop on Release Engineering |publisher=IEEE |page=3 |doi=10.1109/RELENG.2015.10 |isbn=978-1-4673-7070-7 |s2cid=4659735}}}}{{efn|Jabbari et al. (2016) "The research results of this study showed the need for a definition as individual studies do not consistently define DevOps."{{Cite journal |last1=Jabbari |first1=Ramtin |last2=bin Ali |first2=Nauman |last3=Petersen |first3=Kai |last4=Tanveer |first4=Binish |date=May 2016 |title=What is DevOps?: A Systematic Mapping Study on Definitions and Practices |journal=Proceedings of the 2016 Scientific Workshop |publisher=Association for Computing Machinery}}}}{{efn|Erich et al. (2017) "We noticed that there are various gaps in the study of DevOps: There is no consensus of what concepts DevOps covers, nor how DevOps is defined."{{Cite journal |last1=Erich |first1=F.M.A. |last2=Amrit |first2=C. |last3=Daneva |first3=M. |date=June 2017 |title=A Qualitative Study of DevOps Usage in Practice |journal=Journal of Software: Evolution and Process |volume=29 |issue=6 |pages=e1885 |doi=10.1002/smr.1885 |s2cid=35914007|url=https://ris.utwente.nl/ws/files/19208022/Erich_et_al_2017_Journal_of_Software_Evolution_and_Process.pdf }}}}{{efn|Erich et al. (2017) "We discovered that there exists little agreement about the characteristics of DevOps in the academic literature."}} DevOps is characterized by key principles: shared ownership, workflow automation, and rapid feedback.
From an academic perspective, Len Bass, Ingo Weber, and Liming Zhu—three computer science researchers from the CSIRO and the Software Engineering Institute—suggested defining DevOps as "a set of practices intended to reduce the time between committing a change to a system and the change being placed into normal production, while ensuring high quality".{{Cite book |last1=Bass, Len |title=DevOps: A Software Architect's Perspective |last2=Weber, Ingo |last3=Zhu, Liming |year=2015 |publisher=Addison-Wesley |isbn=978-0134049847}}
However, the term is used in multiple contexts. At its most successful, DevOps is a combination of specific practices, culture change, and tools.{{Cite journal |last1=Muñoz |first=Mirna |last2=Negrete Rodríguez |first2=, Mario |date=2021-04-01 |title=A guidance to implement or reinforce a DevOps approach in organizations: A case study |url=//onlinelibrary.wiley.com/journal/20477481 |journal=Journal of Software: Evolution and Process |publisher=Wiley |doi=10.1002/smr.2342}}
History
Proposals to combine software development methodologies with deployment and operations concepts began to appear in the late 80s and early 90s.Chapman, M., Gatti, N: A model of a service life cycle, Proceedings of TINA '93, pp. I-205–I-215, Sep., 1993.
In 2009, the first conference named DevOps Days was held in Ghent, Belgium. The conference was founded by Belgian consultant, project manager and agile practitioner Patrick Debois.{{Cite web |last=Mezak |first=Steve |date=25 January 2018 |title=The Origins of DevOps: What's in a Name? |url=https://devops.com/the-origins-of-devops-whats-in-a-name/ |access-date=6 May 2019 |publisher=devops.com}}{{Cite web |last=Debois |first=Patrick |date=9 October 2008 |title=Agile 2008 Toronto |url=http://www.jedi.be/blog/2008/10/09/agile-2008-toronto-agile-infrastructure-and-operations-presentation/ |access-date=12 March 2015 |publisher=Just Enough Documented Information}} The conference has now spread to other countries.{{Cite web |last=Debois |first=Patrick |title=DevOps Days |url=http://www.devopsdays.org/ |access-date=31 March 2011 |publisher=DevOps Days}}
In 2012, a report called "State of DevOps" was first published by Alanna Brown at Puppet Labs.{{Cite web |last1=Alana Brown |last2=Nicole Forsgren |last3=Jez Humble |last4=Nigel Kersten |last5=Gene Kim |date=2016 |title=2016 State of DevOps Report |url=https://dora.dev/research/2017-and-earlier/2016-state-of-devops-report.pdf |access-date=2024-04-24 |publisher=Puppet Labs, DORA (DevOps Research}}{{Cite web |title=Puppet - Alanna Brown |url=https://puppet.com/people/alanna-brown |access-date=2019-04-27 |publisher=Puppet Labs}}
As of 2014, the annual State of DevOps report was published by Nicole Forsgren, Gene Kim, Jez Humble and others. They stated that the adoption of DevOps was accelerating.{{Cite web |last1=Nicole Forsgren |last2=Gene Kim |last3=Nigel Kersten |last4=Jez Humble |date=2014 |title=2014 State of DevOps Report |url=https://dora.dev/research/2017-and-earlier/2014-state-of-devops-report.pdf |access-date=2024-04-24 |publisher=Puppet Labs, IT Revolution Press and ThoughtWorks}}{{Cite web |date=2015 |title=2015 State of DevOps Report |url=https://dora.dev/research/2017-and-earlier/2015-state-of-devops-report.pdf | access-date=2024-04-24 |publisher=Puppet Labs, Pwc, IT Revolution Press}} Also in 2014, Lisa Crispin and Janet Gregory wrote the book More Agile Testing, containing a chapter on testing and DevOps.{{Cite web |date=October 2014 |title=More Agile Testing |url=https://agiletester.ca/wp-content/uploads/sites/26/2014/09/TOC.pdf |access-date=2019-05-06}}{{Cite book |last1=Crispin |first1=Lisa |url=https://www.oreilly.com/library/view/more-agile-testing/9780133749571/ |title=More Agile Testing |last2=Gregory |first2=Janet |date=October 2014 |publisher=Addison-Wesley |isbn=9780133749571 |access-date=2019-05-06}}
In 2016, the DORA metrics for throughput (deployment frequency, lead time for changes), and stability (mean time to recover, change failure rate) were published in the State of DevOps report. However, the research methodology and metrics were criticized by experts.{{Cite news |last=Turner |first=Graham |date=20 November 2023 |title=Report: Software Engineers Face Backlash for Reporting Wrongdoing |url=https://www.digit.fyi/report-software-engineers-facing-retaliation-for-reporting-wrongdoing/ |access-date=5 January 2024 |work=DIGIT |language=en}}{{Cite news |last=Saran |first=Cliff |title=Software engineers worry about speaking out - Computer Weekly |url=https://www.computerweekly.com/news/366560292/Software-engineers-worry-about-speaking-out |access-date=5 January 2024 |work=ComputerWeekly.com |language=en}}{{Cite news |title=75% of software engineers faced retaliation the last time they reported wrongdoing - ETHRWorldSEA |url=https://hrsea.economictimes.indiatimes.com/news/workplace/75-of-software-engineers-faced-retaliation-the-last-time-they-reported-wrongdoing/105335733 |work=ETHRWorld.com |language=en}}{{Cite web |last=Cummins |first=Holly |title=Holly Cummins on X |url=https://twitter.com/holly_cummins/status/1448357917384744964 |access-date=5 January 2024 |website=X.com}} In response to these criticisms, the 2023 State of DevOps report {{Cite web | last1=DeBellis | first1=Derek | title=2023 State of DevOps Report | url=https://cloud.google.com/devops/state-of-devops | access-date=2024-04-24 | last2=Lewis | first2=Amanda | last3=Villalba | first3=Daniella | last4=Farley | first4=Dave | publisher=Google Cloud DevOps Research and Assessment}} published changes that updated the stability metric "mean time to recover" to "failed deployment recovery time" acknowledging the confusion the former metric has caused.{{cite web |last1=DeBellis |first1=Derek |last2=Harvey |first2=Nathan |title=2023 State of DevOps Report: Culture is everything |url=https://cloud.google.com/blog/products/devops-sre/announcing-the-2023-state-of-devops-report |website=Google Cloud Blog |access-date=2024-04-24}}
Relevant metrics
DevOps Research and Assessment (DORA) has developed a series of metrics which are intended to measure software development efficiency and reliability. These metrics include: {{Cite book |last=Forsgren |first=Nicole |title=Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations |isbn=978-1942788331}}{{Cite journal |date=2021 |title=DORA Accelerate State of DevOps 2021 |journal=DevOps Research and Assessment |issue=2021}}
- Deployment Frequency: Time between code deployments.
- Mean Lead Time for Changes: Time between code commit and deployment.
- Change Failure Rate: Percentage of deployments causing production issues.
- Failed Deployment Recovery Time (formerly Mean Time To Recover)
- Reliability (added in 2021): Measures operational performance, focusing on availability and adherence to user expectations.
Relationship to other approaches
Many of the ideas fundamental to DevOps practices are inspired by, or mirror, other well known practices such as Lean and Deming's Plan-Do-Check-Act cycle, through to The Toyota Way and the Agile approach of breaking down components and batch sizes.{{Cite journal |last=Klein |first=Brandon Thorin |date=2021-05-01 |title=The DevOps: A Concise Understanding to the DevOps Philosophy and Science |url=https://www.osti.gov/biblio/1785164/ |language=English |doi=10.2172/1785164 |osti=1785164 |s2cid=236606284 |website=Osti.gov}} Contrary to the "top-down" prescriptive approach and rigid framework of ITIL in the 1990s, DevOps is "bottom-up" and flexible, having been created by software engineers for their own needs.{{Cite web |date=5 July 2020 |title=The History and Evolution of DevOps {{!}} Tom Geraghty |url=https://tomgeraghty.co.uk/index.php/the-history-and-evolution-of-devops/ |access-date=2020-11-29 |language=en-GB}}
=Platform engineering=
{{main|Platform engineering}}
Platform engineering is an emerging discipline within software engineering that supports DevOps by building and maintaining internal developer platforms (IDPs). These platforms provide standardized tools and reusable components—such as CI/CD pipelines, infrastructure provisioning, observability, and security controls—to streamline software delivery and reduce the cognitive load on developers. The goal is to enable self-service capabilities, improve productivity, and ensure consistency across development and operations teams.{{Cite web |title=What is platform engineering? |url=https://learn.microsoft.com/en-us/platform-engineering/what-is-platform-engineering |website=Microsoft Learn |access-date=2025-04-01}}{{Cite web |title=What is platform engineering? |url=https://www.redhat.com/en/topics/devops/platform-engineering |website=Red Hat |access-date=2025-04-01}}
=Agile=
{{Main|Agile software development}}
The motivations for what has become modern DevOps and several standard DevOps practices such as automated build and test, continuous integration, and continuous delivery originated in the Agile world, which dates (informally) to the 1990s, and formally to 2001. Agile development teams using methods such as extreme programming couldn't "satisfy the customer through early and continuous delivery of valuable software"{{Cite web |title=Principles behind the Agile Manifesto |url=https://agilemanifesto.org/principles.html |access-date=2020-12-06 |website=agilemanifesto.org}} unless they took responsibility for operations and infrastructure for their applications, automating much of that work. Because Scrum emerged as the dominant Agile framework in the early 2000s and it omitted the engineering practices that were part of many Agile teams, the movement to automate operations and infrastructure functions splintered from Agile and expanded into what has become modern DevOps. Today, DevOps focuses on the deployment of developed software, whether it is developed using Agile oriented methodologies or other methodologies.
=ArchOps=
ArchOps presents an extension for DevOps practice, starting from software architecture artifacts, instead of source code, for operation deployment.{{Cite book |last1=Castellanos |first1=Camilo |title=Software Architecture |last2=Correal |first2=Dario |date=15 September 2018 |isbn=978-3-030-00760-7 |series=Lecture Notes in Computer Science |volume=11048 |pages=364–371 |chapter=Executing Architectural Models for Big Data Analytics |doi=10.1007/978-3-030-00761-4_24}} ArchOps states that architectural models are first-class entities in software development, deployment, and operations.
= Continuous Integration and Delivery (CI/CD) =
{{Main|CI/CD}}
Automation is a core principle for achieving DevOps success and CI/CD is a critical component.{{Cite book |last1=Humble |first1=Jez |title=Continuous Delivery: reliable software releases through build, test, and deployment automation |last2=Farley |first2=David |date=2011 |publisher=Pearson Education Inc. |isbn=978-0-321-60191-9}} Plus, improved collaboration and communication between and within teams helps achieve faster time to market, with reduced risks.{{Cite journal |last=Chen |first=Lianping |year=2015 |title=Continuous Delivery: Huge Benefits, but Challenges Too |journal=IEEE Software |volume=32 |issue=2 |pages=50–54 |doi=10.1109/MS.2015.27 |s2cid=1241241}}
= Mobile DevOps =
{{Main|Mobile DevOps}}
Mobile DevOps is a set of practices that applies the principles of DevOps specifically to the development of mobile applications. Traditional DevOps focuses on streamlining the software development process in general, but mobile development has its own unique challenges that require a tailored approach.{{Cite book |last1=Tak |first1=Rohin |title=Mobile DevOps: Deliver continuous integration and deployment within your mobile applications |last2=Modi |first2=Jhalak |date=2018 |publisher=Packt Publishing |isbn=9781788296243 |pages=12–18}} Mobile DevOps is not simply as a branch of DevOps specific to mobile app development, instead an extension and reinterpretation of the DevOps philosophy due to very specific requirements of the mobile world.
=Site-reliability engineering=
{{Main|Site reliability engineering}}
In 2003, Google developed site reliability engineering (SRE), an approach for releasing new features continuously into large-scale high-availability systems while maintaining high-quality end-user experience.{{Cite book |last1=Beyer |first1=Betsy |title=Site Reliability Engineering |last2=Jones |first2=Chris |last3=Petoff |first3=Jennifer |last4=Murphy |first4=Niall Richard |date=April 2016 |publisher=O'Reilly Media |isbn=978-1-4919-2909-4}} While SRE predates the development of DevOps, they are generally viewed as being related to each other. Some of the original authors of the discipline consider SRE as an implementation of DevOps.{{cite web |url=https://driftboatdave.com/2018/10/09/interview-with-betsy-beyer-stephen-thorne-of-google/ |title=Interview with Betsy Beyer, Stephen Thorne of Google |date=9 Oct 2018 |author=Dave Harrison |access-date=24 July 2024}}
=Toyota production system, lean thinking, kaizen=
{{main|Toyota Production System}}
Toyota production system, also known under the acronym TPS, was the inspiration for lean thinking with its focus on continuous improvement, kaizen, flow and small batches. The andon cord principle to create fast feedback, swarm and solve problems stems from TPS.[https://opensource.com/article/18/11/analyzing-devops Analyzing the DNA of DevOps], Brent Aaron Reed, Willy Schaub, 2018-11-14.{{Cite book |last1=Gene Kim |title=The DevOps Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations |last2=Patrick Debois |last3=John Willis |last4=Jezz Humble |date=2016}}
=DevSecOps, shifting security left=
DevSecOps is an augmentation of DevOps to allow for security practices to be integrated into the DevOps approach. Contrary to a traditional centralized security team model, each delivery team is empowered to factor in the correct security controls into their software delivery. Security practices and testing are performed earlier in the development lifecycle, hence the term "shift left". Security is tested in three main areas: static, software composition, and dynamic.
Checking software statically via static application security testing (SAST) is white-box testing with special focus on security. Depending on the programming language, different tools are needed to do such static code analysis. The software composition is analyzed, especially libraries, and the version of each component is checked against vulnerability lists published by CERT and other expert groups. When giving software to clients, library licenses and their match to the license of the software distributed are in focus, especially copyleft licenses.
In dynamic testing, also called black-box testing, software is tested without knowing its inner functions. In DevSecOps this practice may be referred to as dynamic application security testing (DAST) or penetration testing. The goal is early detection of defects including cross-site scripting and SQL injection vulnerabilities. Threat types are published by the open web application security project, e.g. its TOP10,{{Cite web |title=OWASP TOP10 |url=https://owasp.org/Top10/ |url-status=live |archive-url=https://web.archive.org/web/20230608171837/https://owasp.org/Top10/ |archive-date=June 8, 2023 |access-date=June 8, 2023}} and by other bodies.
DevSecOps has also been described as a cultural shift involving a holistic approach to producing secure software by integrating security education, security by design, and security automation.{{Cite book |last=Wilson |first=Glenn |title='DevSecOps: A leader's guide to producing secure software without compromising flow, feedback and continuous improvement' |date=December 2020 |publisher=Rethink Press |isbn=978-1781335024}}
Culture
DevOps initiatives can change how a company's operations, developers, and testers collaborate during the development and delivery processes.{{Cite report |title=Emerging Technology Analysis: DevOps a Culture Shift, Not a Technology |publisher=Gartner}}{{Cite web |last=Loukides |first=Mike |date=7 June 2012 |title=What is DevOps? |url=http://radar.oreilly.com/2012/06/what-is-devops.html |publisher=O'Reilly Media}}
DevOps attempts to support consistency, reliability, and efficiency within an organization. This is usually enabled by a shared code repository or version control.{{Cite SSRN |title=DevOps and Its Practices |last=Teja Yarlagadda |first=Ravi |date=9 March 2021 |ssrn=3798877}} Many organizations use version control to facilitate DevOps automation technologies like virtual machines, containerization (or OS-level virtualization), and CI/CD,{{Cite thesis |last=Morisio |first=Maurizio |title=DevOps: development of a toolchain in the banking domain |date=16 April 2021 |access-date=16 August 2021 |degree=laurea |url=https://webthesis.biblio.polito.it/18120/ |journal=Politecnico di Torino}} with the Git version control system and the GitHub platform referenced as examples.
GitOps
GitOps evolved from DevOps. The specific state of deployment configuration is version-controlled. Because the most popular version-control is Git, the GitOps approach has been named after Git. Changes to configuration can be managed using code review practices, and can be rolled back using version-controlling. Essentially, all of the changes to a code are tracked, bookmarked, and making any updates to the history can be made easier. As explained by Red Hat, "visibility to change means the ability to trace and reproduce issues quickly, improving overall security."{{Cite web |title=What is GitOps? |url=https://www.redhat.com/en/topics/devops/what-is-gitops |access-date=2023-03-30 |website=www.redhat.com |language=en}}
Best practices for cloud systems
The following practices can enhance productivity of DevOps pipelines, especially in systems hosted in the cloud:{{Cite book |title=Serverless Architectures on AWS |publisher=Manning |isbn=978-1617295423}}{{Cite book |title=Pipeline as Code Continuous Delivery with Jenkins, Kubernetes, and Terraform |publisher=Manning |isbn=9781638350378}}{{Cite book |title=Continuous Delivery Reliable Software Releases Through Build, Test, and Deployment Automation |isbn=9780321670229}}
- Number of Pipelines: Small teams can be more productive by having one repository and one pipeline. In contrast, larger organizations may have separate repositories and pipelines for each team or even separate repositories and pipelines for each service within a team.
- Permissions: In the context of pipeline-related permissions, adhering to the principle of least privilege can be challenging due to the dynamic nature of architecture. Administrators may opt for more permissive permissions while implementing compensating security controls to minimize the blast radius.
See also
- {{Annotated link|DataOps}}
- {{Annotated link|DevOps toolchain}}
- {{Annotated link|Infrastructure as code}}
- {{Annotated link|Lean software development}}
- {{Annotated link|List of build automation software}}
- {{Annotated link|Site reliability engineering}}
- {{Annotated link|Value stream}}
- {{Annotated link|Twelve-Factor App methodology}}
Notes
{{notelist}}
References
{{reflist|1=30em}}
Further reading
- {{Cite book |last1=Davis |first1=Jennifer |title=Effective DevOps : building a culture of collaboration, affinity, and tooling at scale |last2=Daniels |first2=Ryn |date=2016-05-30 |publisher=O'Reilly |isbn=9781491926437 |location=Sebastopol, CA |oclc=951434424}}
- {{Cite book |last1=Kim |first1=Gene |title=The DevOps handbook : how to create world-class agility, reliability, and security in technology organizations |last2=Debois |first2=Patrick |last3=Willis |first3=John |last4=Humble |first4=Jez |last5=Allspaw |first5=John |date=2015-10-07 |isbn=9781942788003 |edition=First |location=Portland, OR |oclc=907166314}}
- {{Cite book |last1=Forsgren |first1=Nicole |title=Accelerate: The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations |last2=Humble |first2=Jez |last3=Kim |first3=Gene |date=27 March 2018 |publisher=IT Revolution Press |isbn=9781942788331 |edition=First}}
{{Software engineering|state=expanded}}
{{Authority control}}