Developing and releasing software can be a complicated process, especially as applications, teams, and deployment infrastructure grow in complexity themselves. Often, challenges become more pronounced as projects grow. To develop, test, and release software in a quick and consistent way, developers and organizations have created three related but distinct strategies to manage and automate these processes.
Continuous integration focuses on integrating work from individual developers into a main repository multiple times a day to catch integration bugs early and accelerate collaborative development. Continuous delivery is concerned with reducing friction in the deployment or release process, automating the steps required to deploy a build so that code can be released safely at any time. Continuous deployment takes this one step further by automatically deploying each time a code change is made.
In this guide, we will discuss each of these strategies, how they relate to one another, and how incorporating them into your application life cycle can transform your software development and release practices.
What is Continuous Integration and Why Is It Helpful?
Continuous integration is a practice that encourages developers to integrate their code into a main branch of a shared repository early and often. Instead of building out features in isolation and integrating them at the end of a development cycle, code is integrated with the shared repository by each developer multiple times throughout the day.
The idea is to minimize the cost of integration by making it an early consideration. Developers can discover conflicts at the boundaries between new and existing code early, while conflicts are still relatively easy to reconcile. Once the conflict is resolved, work can continue with confidence that the new code honors the requirements of the existing codebase.
Integrating code frequently does not, by itself, offer any guarantees about the quality of the new code or functionality. In many organizations, integration is costly because manual processes are used to ensure that the code meets standards, does not introduce bugs, and does not break existing functionality. Frequent integration can create friction when the level of automation does not match the amount quality assurance measures in place.
To address this friction within the integration process, in practice, continuous integration relies on robust test suites and an automated system to run those tests. When a developer merges code into the main repository, automated processes kick off a build of the new code. Afterwards, test suites are run against the new build to check whether any integration problems were introduced. If either the build or the test phase fails, the team is alerted so that they can work to fix the build.
The end goal of continuous integration is to make integration a simple, repeatable process that is part of the everyday development workflow in order to reduce integration costs and respond to defects early. Working to make sure the system is robust, automated, and fast while cultivating a team culture that encourages frequent iteration and responsiveness to build issues is fundamental to the success of the strategy.
What is Continuous Delivery and Why Is It Helpful?
Continuous delivery is an extension of continuous integration. It focuses on automating the software delivery process so that teams can easily and confidently deploy their code to production at any time. By ensuring that the codebase is always in a deployable state, releasing software becomes an unremarkable event without complicated ritual. Teams can be confident that they can release whenever they need to without complex coordination or late-stage testing. As with continuous integration, continuous delivery is a practice that requires a mixture of technical and organizational improvements to be effective.
On the technology side, continuous delivery leans heavily on deployment pipelines to automate the testing and deployment processes. A deployment pipeline is an automated system that runs increasingly rigorous test suites against a build as a series of sequential stages. This picks up where continuous integration leaves off, so a reliable continuous integration setup is a prerequisite to implementing continuous delivery.
At each stage, the build either fails the tests, which alerts the team, or passes the tests, which results in automatic promotion to the next stage. As the build moves through the pipeline, later stages deploy the build to environments that mirror the production environment as closely as possible. This way the build, the deployment process, and the environment can be tested in tandem. The pipeline ends with a build that can be deployed to production at any time in a single step.
The organizational aspects of continuous delivery encourage prioritization of "deployability" as a principle concern. This has an impact on the way that features are built and hooked into the rest of the codebase. Thought must be put into the design of the code so that features can be safely deployed to production at any time, even when incomplete. A number of techniques have emerged to assist in this area.
Continuous delivery is attractive because it automates the steps between checking code into the repository and deciding on whether to release well-tested, functional builds to your production infrastructure. The steps that help assert the quality and correctness of the code are automated, but the final decision about what to release is left in the hands of the organization for maximum flexibility.
What is Continuous Deployment and Why Is It Helpful?
Continuous deployment is an extension of continuous delivery that automatically deploys each build that passes the full test cycle. Instead of waiting for a human gatekeeper to decide what and when to deploy to production, a continuous deployment system deploys everything that has successfully traversed the deployment pipeline. Keep in mind that while new code is automatically deployed, techniques exist to activate new features at a later time or for a subset of users. Deploying automatically pushes features and fixes to customers quickly, encourages smaller changes with limited scope, and helps avoid confusion over what is currently deployed to production.
This fully automated deploy cycle can be a source of anxiety for organizations worried about relinquishing control to their automation system of what gets released. The trade-off offered by automated deployments is sometimes judged to be too dangerous for the payoff they provide.
Other groups leverage the promise of automatic release as a method of ensuring that best practices are always followed and to extend the testing process into a limited production environment. Without a final manual verification before deploying a piece of code, developers must take responsibility for ensuring that their code is well-designed and that the test suites are up-to-date. This collapses the decision of what and when to commit to the main repository and what and when to release to production into a single point that exists firmly in the hands of the development team.
Continuous deployment also allows organizations to benefit from consistent early feedback. Features can immediately be made available to users and defects or unhelpful implementations can be caught early before the team devotes extensive effort in an unproductive direction. Getting fast feedback that a feature isn't helpful lets the team shift focus rather than sinking more energy into an area with minimal impact.
Key Concepts and Practices for Continuous Processes
While continuous integration, delivery, and deployment vary in the scope of their involvement, there are some concepts and practices that are fundamental to the success of each.
Small, Iterative Changes
One of the most important practices when adopting continuous integration is to encourage small changes. Developers should practice breaking up larger work into small pieces and committing those early. Special techniques like branch by abstraction and feature flags (see below) help to protect the functionality of the main branch from in-progress code changes.
Small changes minimize the possibility and impact of integration problems. By committing to the shared branch at the earliest possible stage and then continually throughout development, the cost of integration is diminished and unrelated work is synchronized regularly.
With trunk-based development, work is done in the main branch of the repository or merged back into the shared repository at frequent intervals. Short-lived feature branches are permissible as long as they represent small changes and are merged back as soon as possible.
The idea behind trunk-based development is to avoid large commits that violate of concept of small, iterative changes discussed above. Code is available to peers early so that conflicts can be resolved when their scope is small.
Releases are performed from the main branch or from a release branch created from the trunk specifically for that purpose. No development occurs on the release branches in order to maintain focus on the main branch as the single source of truth.
Keep the Building and Testing Phases Fast
Each of the processes relies on automated building and testing to validate correctness. Because the build and test steps must be performed frequently, it is essential that these processes be streamlined to minimize the time spent on these steps.
Increases in build time should be treated as a major problem because the impact is compounded by the fact that each commit kicks off a build. Because continuous processes force developers to engage with these activities daily, reducing friction in these areas is a worthwhile pursuit.
When possible, running different sections of the test suite in parallel can help move the build through the pipeline faster. Care should also be taken to make sure the proportion of each type of test makes sense. Unit tests are typically very fast and have minimal maintenance overhead. In contrast, automated system or acceptance testing is often complex and prone to breakage. To account for this, it is often a good idea to rely heavily on unit tests, conduct a fair number of integration tests, and then back off on the number of later, more complex testing.
Consistency Throughout the Deployment Pipeline
Because a continuous delivery or deployment implementations is supposed to be testing release worthiness, it is essential to maintain consistency during each step of the process—the build itself, the deployment environments, and the deployment process itself:
- Code should be built once at the beginning of the pipeline: The resulting software should be stored and accessible to later processes without rebuilding. By using the exact same artifact in each phase, you can be certain that you are not introducing inconsistencies as a result of different build tools.
- Deployment environments should be consistent: A configuration management system can control the various environments, and environmental changes can be put through the deployment pipeline itself to ensure correctness and consistency. Clean deployment environments should be provisioned each test cycle to prevent legacy conditions from compromising the integrity of the tests. The staging environments should match the production environment as closely as possible to reduce unknown factors present when the build is promoted.
- Consistent processes should be used to deploy the build in each environment: Each deployment should be automated and each deployment should use the same centralized tools and procedures. Ad-hoc deployments should be eliminated in favor of deploying only with the pipeline tools.
Decouple Deployment and Release
Separating the deployment of code from its release to users is an extremely powerful part of continuous delivery and deployment. Code can be deployed to production without initially activating it or making it accessible to users. Then, the organization decides when to release new functionality or features independent from deployment.
This gives organizations a great deal of flexibility by separating business decisions from technical processes. If the code is already on the servers, then deployment is no longer a delicate part of the release process, which minimizes the number of individuals and the amount of work involved at the time of release.
There are a number of techniques that help teams deploy the code responsible for a feature without releasing it. Feature flags set up conditional logic to check whether to run code based on the value of an environmental variable. Branch by abstraction allows developers to replace implementations by placing an abstraction layer between resource consumers and providers. Careful planning to incorporate these techniques gives you the ability to decouple these two processes.
Types of Testing
Continuous integration, delivery, and deployment all rely heavily on automated tests to determine the efficacy and correctness of each code change. Different types of tests are needed throughout these processes to gain confidence in a given solution.
While the categories below in no way represent an exhaustive list, and although there is disagreement on the exact definition of each type, these broad categories of tests represent a variety of ways to evaluate code in different contexts.
Smoke tests are a special kind of initial checks designed to ensure very basic functionality as well as some basic implementation and environmental assumptions. Smoke tests are generally run at the very start of each testing cycle as a sanity check before running a more complete test suite.
The idea behind this type of test is to help to catch big red flags in an implementation and to bring attention to problems that might indicate that further testing is either not possible or not worthwhile. Smoke tests are not very extensive, but should be extremely quick. If a change fails a smoke test, its an early signal that core assertions were broken and that you should not devote any more time to testing until the problem is resolved.
Context-specific smoke tests can be employed at the start of any new phase testing to assert that the basic assumptions and requirements are met. For instance, smoke tests can be used both prior to integration testing or deploying to staging servers, but the conditions to be tested will vary in each case.
Unit tests are responsible for testing individual elements of code in an isolated and highly targeted way. The functionality of individual functions and classes are tested on their own. Any external dependencies are replaced with stub or mock implementations to focus the test completely on the code in question.
Unit tests are essential to test the correctness of individual code components for internal consistency and correctness before they are placed in more complex contexts. The limited extent of the tests and the removal of dependencies makes it easier to hunt down the cause of any defects. It also is the best time to test a variety of inputs and code branches that might be difficult to hit later on. Often, after any smoke tests, unit tests are the first tests that are run when any changes are made.
Unit tests are typically run by individual developers on their own work station prior to submitting changes. However, continuous integration servers almost always run these tests again as a safe guard before beginning integration tests.
After unit tests, integration testing is performed by grouping together components and testing them as an assembly. While unit tests validate the functionality of code in isolation, integration tests ensure that components cooperate when interfacing with one another. This type of testing has the opportunity to catch an entirely different class of bugs that are exposed through interaction between components.
Typically, integration tests are performed automatically when code is checked into a shared repository. A continuous integration server checks out the code, performs any necessary build steps (usually performing a quick smoke test to make sure the build was successful) and then runs unit and integration tests. Modules are hooked together in different combinations and tested.
Integration tests are important for shared work because they protect the health of the project. Changes must prove that they do not break existing functionality and that they interact with other code as expected. A secondary aim of integration testing is to verify that the changes can be deployed into a clean environment. This is frequently the first testing cycle that is performed off of the developer's own machines, so unknown software and environmental dependencies can also be discovered during this process. This is usually also the first time that new code is tested against real external libraries, services, and data.
Once integration tests are performed, another level of testing called system testing can begin. In many ways, system testing acts as an extension to integration testing. The focus of system tests are to make sure that groups of components function correctly as a cohesive whole.
Instead of focusing on the interfaces between components, system tests typically evaluate the outward functionality of a full piece of software. This set of tests ignores the constituent parts in order to gauge the composed software as a unified entity. Because of this distinction, system tests usually focus on user- or externally-accessible interfaces.
Acceptance tests are one of the last tests types that are performed on software prior to delivery. Acceptance testing is used to determine whether a piece of software satisfies all of the requirements from the business or user's perspective. These tests are sometimes built against the original specification and often test interfaces for the expected functionality and for usability.
Acceptance testing is often a more involved phase that might extend past the release of the software. Automated acceptance testing can be used to make sure the technological requirements of the design were met, but manual verification also usually plays a role.
Frequently, acceptance testing begins by deploying the build to a staging environment that mirrors the production system. From here, the automated test suites can be run and internal users can access the system to check whether it functions the way they need it to. After release or offering beta access to customers, further acceptance testing is performed by evaluating how the software functions with real use and by collecting feedback from users.
While we've discussed some of the broader ideas above, there are many related concepts that you may come across as you learn about continuous integration, delivery, and deployment. Let's define a few other terms you are likely to see:
- Blue-Green Deployments: Blue-green deployments is a strategy for testing code in a production-like environment and for deploying code with minimal downtime. Two sets of production-capable environments are maintained, and code is deployed to the inactive set where testing can take place. When ready to release, production traffic is routed to the servers with the new code, instantly making the changes available.
- Branch by Abstraction: Branch by abstraction is a method of performing major refactoring operations in an active project without long-lived development branches in the source code repository, which continuous integration practices discourage. An abstraction layer is built and deployed between consumers and the existing implementation so that the new implementation can be built out behind the abstraction in parallel.
- Build (noun): A build is a specific version of software created from source code. Depending on the language, this might be compiled code or a consistent set of interpreted code.
- Canary Releases: Canary releases are a strategy for releasing changes to a limited subset of users. The idea is to make sure everything works correctly with production workloads while minimizing the impact if there are problems.
- Dark launch: Dark launching is the practice of deploying code to production that receives production traffic but does not impact the user experience. New changes are deployed alongside existing implementations and the same traffic is often routed to both places for testing. The old implementation is still hooked up to the user's interface, but behind the scenes, the new code can be evaluated for correctness using real user requests in the production environment.
- Deployment Pipeline: A deployment pipeline is a set of components that moves software through increasingly rigorous testing and deployment scenarios to evaluate its readiness for release. The pipeline typically ends by automatically deploying to production or providing the option to do so manually.
- Feature Flags or Feature Toggles: Feature flags are a technique of deploying new features behind conditional logic that determines whether or not to run based on the value of an environmental variable. New code can be deployed to production without being activated by setting the flag appropriately. To release the software, the value of the environmental variable is changed, causing the new code path to be activated. Feature flags often contain logic that allows for subsets of users to gain access to the new feature, creating a mechanism to gradually roll out the new code.
- Promoting: In the context of continuous processes, promoting means moving a software build through to the next stage of testing.
- Soak Test: Soak testing involves testing software under significant production or production-like load for an extended period of time.
In this guide, we introduced continuous integration, continuous delivery, and continuous deployment and discussed how they can be used to build and release well-tested software safely and quickly. These processes leverage extensive automation and encourage constant code sharing to fix defects early. While the techniques, processes, and tools needed to implement these solutions represent a significant challenge, the benefits of a well-designed and properly used system can be enormous.
开发和发布软件可能是一个复杂的过程，特别是当应用程序，团队和部署基础架构本身复杂化时。 随着项目的增长，挑战越来越明显。 为了以快速而一致的方式开发，测试和发布软件，开发人员和组织已经创建了三个相关但不同的策略来管理和自动化这些流程。
持续整合的重点是将个人开发人员的工作整合到主存储库，每天多次，以及时捕捉整合错误，并加速协同开发。 持续交付涉及减少部署或释放过程中的摩擦，自动执行部署构建所需的步骤，以便可以随时安全地释放代码。 在每次进行代码更改时，通过自动部署，持续部署将进一步扩展。
这个想法是通过早日考虑来最大限度地降低整合成本。 开发人员可以尽早发现新代码和现有代码之间的边界冲突，而冲突仍然比较容易协调。 一旦冲突得到解决，工作就可以保证新代码能够满足现有代码库的要求。
频繁集成代码本身不会对新代码或功能的质量提供任何保证。 在许多组织中，集成是昂贵的，因为使用手动过程来确保代码符合标准，不会引入错误，并且不会破坏现有功能。 当自动化水平不符合质量保证措施时，频繁的集成可能会产生摩擦。
为了在整合过程中解决这种摩擦，实际上，持续集成依赖于强大的测试套件和自动化系统来运行这些测试。 当开发人员将代码合并到主存储库中时，自动化过程将启动新代码的构建。 之后，针对新版本运行测试套件，以检查是否引入了任何集成问题。 如果构建或测试阶段失败，则会警告小组，以便他们可以修复构建。
持续交付是持续整合的延伸。 它专注于自动化软件交付过程，以便团队可以随时轻松自信地将其代码部署到生产中。 通过确保代码库始终处于可部署状态，释放软件将成为无复杂仪式的不起眼的事件。 团队可以相信，只要他们需要没有复杂的协调或后期阶段测试，他们就可以释放。 与持续集成一样，持续交付是一种需要技术和组织改进的混合才能有效的做法。
在技术方面，持续的交付大大地依赖于部署管道来自动化测试和部署过程。 部署管道是一个自动化系统，可以针对构建运行越来越严格的测试套件，作为一系列顺序阶段。 这在连续集成脱离的情况下获得了回报，因此可靠的连续集成设置是实施连续交付的先决条件。
在每个阶段，构建或者不通过测试，这会提醒团队，或者通过测试，从而将自动升级到下一阶段。 随着构建运行通过管道，后续阶段将构建部署到尽可能接近生产环境的环境中。 这样可以一起测试构建，部署过程和环境。 管道以可以在一个步骤中随时部署到生产中的构建结束。
持续交付的组织方面鼓励将“可部署性”作为主要关注点。 这对构建功能并挂接到其他代码库的方式有影响。 必须考虑到代码的设计，即使在不完整的情况下，随时可以将功能安全地部署到生产中。 已经出现了一些技术来协助这一领域。
持续部署是连续交付的一种延伸，可以自动部署通过完整测试周期的每个构建。 连续部署系统不是等待人类网守来决定何时和何时部署到生产，而是部署了已经成功遍历部署管道的所有内容。 请记住，当自动部署新代码时，会存在稍后或部分用户激活新功能的技术。 自动部署功能并快速修复客户，鼓励在范围有限的情况下进行更小的更改，并有助于避免与目前部署到生产中的内容混淆。
其他组织利用自动发布的承诺，作为确保始终遵循最佳实践的方法，并将测试过程扩展到有限的生产环境中。 在部署一段代码之前，没有最终的手动验证，开发人员必须承担责任，确保他们的代码设计精良，测试套件是最新的。 这破坏了什么和什么时候承诺主要存储库的决定，什么和什么时候将生产释放到一个牢固地掌握在开发团队手中的单一点。
持续部署还使组织能够从一致的早期反馈中受益。 功能可以立即向用户提供，缺陷或无益的实施可以在团队在无效的方向下付出巨大努力的早期被捕获。 获得快速反馈，功能不是很有帮助，让团队转移焦点，而不是将更多的精力投入到影响最小的区域。
采取持续整合的最重要的做法之一是鼓励小的变化。 开发人员应该把更大的工作分解成小件，早日实现。 通过抽象和特征标记（见下文）分支的特殊技术有助于保护主分支的功能不被进行中的代码更改。
在可能的情况下，并行运行测试套件的不同部分可以更快地移动构建通过管道。 还要注意确保每种类型的测试的比例是有意义的。 单元测试通常非常快，维护开销最小。 相比之下，自动化系统或验收测试往往很复杂，易于破损。 为了解决这个问题，通常很有可能依靠单元测试，进行相当多的集成测试，然后退回后来更复杂的测试。
- 代码应该在管道开始时建立一次：生成的软件应该被存储和访问以供后续流程而不重建。 通过在每个阶段使用完全相同的工件，您可以确定您不会因为不同的构建工具而引入不一致。
- 部署环境应该是一致的：配置管理系统可以控制各种环境，环境变化可以通过部署管道本身进行，以确保正确性和一致性。 应为每个测试周期配置清洁的部署环境，以防止遗留条件损害测试的完整性。 阶段性环境应尽可能接近生产环境，以减少构建提升时存在的未知因素。
- 应该使用一致的流程来在每个环境中部署构建：每个部署应该是自动的，每个部署应该使用相同的集中式工具和过程。 应该消除专门部署，以便仅使用管道工具进行部署。
将代码部署从其发行版分离到用户是连续交付和部署中非常强大的一部分。 代码可以部署到生产中，而无需最初激活它或使其可供用户访问。 然后，组织决定何时发布独立于部署的新功能或功能。
有许多技术帮助团队部署负责功能的代码，而不会释放它。 功能标志设置条件逻辑，以检查是否根据环境变量的值运行代码。 通过抽象分支，开发人员可以通过在资源消费者和提供者之间放置抽象层来替换实现。 仔细规划并入这些技术使您能够将这两个过程分离。
这种测试背后的想法是帮助捕获实施中的大红旗，并引起注意可能表明进一步测试是不可能或不值得的问题。 烟雾测试不是很广泛，但应该非常快。 如果一个变化失败了烟雾测试，它的核心断言的早期信号被打破，你不应该再花费更多的时间进行测试，直到问题解决。
单元测试负责以孤立且高度针对性的方式测试代码的各个元素。 各个功能和类的功能自己测试。 任何外部依赖关系都被替换为存根或模拟实现，将测试完全集中在所涉及的代码上。
单元测试对于内部一致性和正确性在被放置在更复杂的上下文中之前测试各个代码组件的正确性至关重要。 测试的有限程度和依赖性的消除使得更容易追查任何缺陷的原因。 它也是测试各种可能难以打击的输入和代码分支的最佳时机。 通常，在任何烟雾测试之后，单元测试是进行任何更改时运行的第一个测试。
单元测试后，通过将组件分组在一起并将其作为组件进行测试来执行集成测试。 当单元测试隔离验证代码的功能时，集成测试确保组件在彼此接口时协作。 这种类型的测试有机会捕获通过组件之间的交互暴露的完全不同类别的错误。
通常，当将代码检入共享存储库时，会自动执行集成测试。 连续集成服务器检查代码，执行任何必要的构建步骤（通常执行快速烟雾测试以确保构建成功），然后运行单元和集成测试。 模块以不同的组合钩在一起并进行测试。
集成测试对共享工作很重要，因为它们保护了项目的健康。 更改必须证明它们不会破坏现有的功能，并且与预期的其他代码进行交互。 集成测试的第二个目标是验证这些更改可以部署到一个干净的环境中。 这是经过开发人员自己的机器执行的第一个测试周期，因此在此过程中也可以发现未知的软件和环境依赖关系。 这通常也是第一次根据真正的外部库，服务和数据测试新的代码。
一旦执行了集成测试，可以开始另一个称为系统测试的测试级别。 在许多方面，系统测试作为集成测试的扩展。 系统测试的重点是确保组件组合作为一个整体的整体正常运行。
系统测试通常不会专注于组件之间的接口，而是评估整个软件的外部功能。 这套测试忽略了组成部分，以便将组合软件评估为统一实体。 由于这种区别，系统测试通常侧重于用户或外部可访问的接口。
验收测试是在交付前在软件上执行的最后一个测试类型之一。 验收测试用于确定一件软件是否满足业务或用户观点的所有要求。 这些测试有时是根据原始规范构建的，并且经常测试接口的预期功能和可用性。
通常，通过将构建部署到反映生产系统的分段环境开始。 从这里可以运行自动测试套件，内部用户可以访问系统，以检查它们是否按照需要的方式运行。 在发布或提供对客户的beta访问后，通过评估软件如何正常使用并收集用户的反馈来进一步验收测试。
- 蓝绿色部署：蓝绿色部署是在类似生产环境中测试代码并以最少的停机时间部署代码的策略。 维护两套具有生产能力的环境，并将代码部署到可以进行测试的非活动集合。 当准备发布时，生产流量将通过新代码路由到服务器，立即使更改可用。
- 分支抽象：分支抽象是一种在活动项目中执行主要重构操作的方法，而在源代码存储库中没有长期存在的开发分支，这种连续集成实践阻止了这一点。 在消费者和现有实现之间构建和部署抽象层，以便并行地在抽象背后构建新的实现。
- 构建（名词）：构建是从源代码创建的软件的特定版本。 根据语言，这可能是编译代码或一组一致的解释代码。
- 金丝雀发布：金丝雀版本是发布对有限子集的更改的策略。 这个想法是确保一切都能与生产工作负载正常工作，同时尽可能减少影响，如果有问题。
- 黑暗发射：黑暗发射是将代码部署到生产流程的做法，但不会影响用户体验。 新的更改与现有实施一起部署，相同的流量通常路由到两个地方进行测试。 旧的实现仍然与用户界面挂钩，但在幕后，可以使用生产环境中的真实用户请求来评估新代码的正确性。
- 部署管道：部署管道是一组组件，通过越来越严格的测试和部署方案来移动软件，以评估其准备发布。 流水线通常以自动部署到生产或提供手动执行的选项来结束。
- 功能标志或特征切换功能标志是一种在条件逻辑之后部署新功能的技术，可根据环境变量的值确定是否运行。 新代码可以部署到生产中，而不必通过适当地设置标志来激活。 要释放软件，环境变量的值被改变，导致新的代码路径被激活。 功能标志通常包含允许用户子集访问新功能的逻辑，创建一种逐渐推出新代码的机制。
在本指南中，我们介绍了持续集成，持续交付和持续部署，并讨论了如何安全，快速地构建和发布经过良好测试的软件。 这些过程利用广泛的自动化，并鼓励持续的代码共享早日解决缺陷。 虽然实施这些解决方案所需的技术，流程和工具是一个重大的挑战，但设计良好且使用正确的系统的好处可能是巨大的。