CICD的思考


CICD 的思考

CICD(Continuous Integration, Continuous Delivery),作为工程人员大家都一致认可 CICD 的重要性,由于 CICD 的涉及面太过广泛,每个工程人员对 CICD 的第一反应都是不尽相同的。在设计 CICD 框架之初,希望可以对 CICD 有一个比较全面而结构化的认识,基于这样的认识进行 CICD 的思考和设计。

CICD 是什么

下面是引用自 Wikipedia 对于 CICD 的解释:

In software engineering, CI/CD or CICD generally refers to the combined practices of continuous integration and either continuous delivery or continuous deployment.

CI/CD bridges the gaps between development and operation activities and teams by enforcing automation in building, testing and deployment of applications. Modern day DevOps practices involve continuous development, continuous testing, continuous integration, continuous deployment and continuous monitoring of software applications throughout its development life cycle. The CI/CD practice or CI/CD pipeline forms the backbone of modern day DevOps operations.1

CICD 存在于软件工程实践中,它的目的是帮助开发团队更快、更可靠地发布软件迭代,从 DevOps 或者敏捷开发的角度来讲,CICD 是最佳实践,这个毋庸置疑的。从Wikipedia的解释中可以看出软件的生命周期包括:

  • continuous development
  • continuous testing
  • continuous integration
  • continuous deployment
  • continuous monitoring

RedHat对上述软件生命周期进行了按时序归类,如下图所示。

CICD flow
CICD flow

CI 是什么

Continuous integration is a coding philosophy and set of practices that drive development teams to implement small changes and check in code to version control repositories frequently. Because most modern applications require developing code in different platforms and tools, the team needs a mechanism to integrate and validate its changes.

The technical goal of CI is to establish a consistent and automated way to build, package, and test applications. With consistency in the integration process in place, teams are more likely to commit code changes more frequently, which leads to better collaboration and software quality.2

CD 是什么

CD 可以是 Continuous Delivery 的缩写,也可以是 Continuous Deployment 的缩写。CD 是 CI 的一个扩展,这个扩展可以持续地确保快速地将软件迭代版本发布到用户。

Continuous delivery picks up where continuous integration ends. CD automates the delivery of applications to selected infrastructure environments. Most teams work with multiple environments other than the production, such as development and testing environments, and CD ensures there is an automated way to push code changes to them.2

Continuous deployment goes one step further than continuous delivery. With this practice, every change that passes all stages of your production pipeline is released to your customers. There’s no human intervention, and only a failed test will prevent a new change to be deployed to production.3

Continuous Delivery 和 Continuous Deployment 的区别如下图所示。

Continuous Delivery V.S. Continuous DeploymentContinuous Delivery V.S. Continuous Deployment

Gain V.S. Cost

CI,CD 的分析表格如下所示3

Subject Cost Gain
Continuous integration – Your team will need to write automated tests for each new feature, improvement or bug fix.
– You need a continuous integration server that can monitor the main repository and run the tests automatically for every new commits pushed.
– Developers need to merge their changes as often as possible, at least once a day.
– Less bugs get shipped to production as regressions are captured early by the automated tests.
– Building the release is easy as all integration issues have been solved early.
– Less context switching as developers are alerted as soon as they break the build and can work on fixing it before they move to another task.
– Testing costs are reduced drastically – your CI server can run hundreds of tests in the matter of seconds.
– Your QA team spend less time testing and can focus on significant improvements to the quality culture.
Continuous delivery – You need a strong foundation in continuous integration and your test suite needs to cover enough of your codebase.
– Deployments need to be automated. The trigger is still manual but once a deployment is started there shouldn’t be a need for human intervention.
– Your team will most likely need to embrace feature flags so that incomplete features do not affect customers in production.
– The complexity of deploying software has been taken away. Your team doesn’t have to spend days preparing for a release anymore.
– You can release more often, thus accelerating the feedback loop with your customers.
– There is much less pressure on decisions for small changes, hence encouraging iterating faster.
Continuous deployment – Your testing culture needs to be at its best. The quality of your test suite will determine the quality of your releases.
– Your documentation process will need to keep up with the pace of deployments.
– Feature flags become an inherent part of the process of releasing significant changes to make sure you can coordinate with other departments (Support, Marketing, PR…).
– You can develop faster as there’s no need to pause development for releases. Deployments pipelines are triggered automatically for every change.
– Releases are less risky and easier to fix in case of problem as you deploy small batches of changes.
– Customers see a continuous stream of improvements, and quality increases every day, instead of every month, quarter or year.

对比下来 Continuous Deployment 是一个比较理想的状态,对像蚂蚁这样的复杂基础设施架构的公司来讲,CICD 的投入会非常巨大,所以暂且以 Continuous Delivery 作为CICD的目标。

CICD 通用框架

通过对 CICD 的通盘了解和分析,无论是 CI 还是 CD 它的最终目的都是:

“establish a consistent and automated way to build, package, and test applications.”

归纳一下 CICD 关键字包括 consistent, automated, build, package, test

  • consistent: 一致的代码仓库、一致的测试环境、一致的发布规则
  • automated: 自动化
  • build: 构建
  • package: release二进制
  • test: UT, AT, ST

对 CICD 的理解

经过上述的分析,我认为 CICD 就是一个自动化的 pipeline,该流水线帮助软件开发从代码提交开始就进行构建、测试、release等整个流程。

鉴于蚂蚁基础设施以及开发环境的特殊性,对 CICD 的细节做了一些调整:

  • 由于生产环境的稳定性的考量,自动化的部署到生产环境( CD 的目标之一)当前不做考虑;
  • pipeline 原则上应该是以 commit 为粒度来触发整个自动化流程,考虑蚂蚁开发的 collaboration 都是以 PR(Pull Request)/MR(Merge Request) 为基本单元,所以从效率的角度,pipeline 修改为以 PR/MR 为粒度进行快速迭代;

CICD 通用框架

通用的 CICD pipeline 如下图所示。

通用 CICD 框架通用 CICD 框架

复盘code review


复盘code review


近两周都code review,CR中挣扎,主要还是所有的工作都被CR严重拖累,被问了太多为什么,而不是有价值的comment,每次review都会经历这样的事情,而且是非收敛式的。心里真想骂娘。

reviewer的逻辑

  • 始终带着一个问题“这个feature如果我来开发,我会怎么写”
  • 依赖于代码命名来理解代码
  • 自己没法理解或者看不懂的就是代码设计不好
  • “听风就是雨”——会带入自己遇到的问题,希望commiter的代码可以解决自己的问题

committer的逻辑

  • 功能逻辑是否满足要求
  • 是否有未考虑的异常场景
  • 代码细节是否有问题(code style rules
  • 是否有测试覆盖

思考总结

千万不要以committer的角度来review代码,reviewer还是需要本着尊重作者的角度来查漏补缺(逻辑,代码规范,测试,重大的设计问题)。

特别说明:这里针对的是团队内部的交付场景


#反思总结

About OpenSSL(Part 1)


关于OpenSSL社区

OpenSSL功能简介,OpenSSL外围包提供了如下三种功能:
1. 命令行工具,用来完成各种各样密码学相关的任务。例如,创建证书、解析证书、加密文件、加密算法测试等。
2. 全面的、可扩展的密码学库(libcrypto)。覆盖了大多数标准定义的加密算法,可用硬件扩展或者加速。
3. 符合SSL/TLS协议的加密通讯库(libssl)。提供客户端和服务端进行加密通讯的的能力。
Continue reading “About OpenSSL(Part 1)”

Python程序性能提升的一种方法


python程序性能分析

虽然运行速度慢是 Python 与生俱来的特点,大多数时候我们用 Python 就意味着放弃对性能的追求。但是,就算是用纯 Python 完成同一个任务,老手写出来的代码可能会比菜鸟写的代码块几倍,甚至是几十倍(这里不考虑算法的因素,只考虑语言方面的因素)。很多时候,我们将自己的代码运行缓慢地原因归结于python本来就很慢,从而心安理得地放弃深入探究。

但是,事实真的是这样吗?面对python代码,你有分析下面这些问题吗:

程序运行的速度如何?
程序运行时间的瓶颈在哪里?
能否稍加改进以提高运行速度呢?
为了更好了解python程序,我们需要一套工具,能够记录代码运行时间,生成一个性能分析报告,方便彻底了解代码,从而进行针对性的优化(本篇侧重于代码性能分析,不关注如何优化)。

Continue reading “Python程序性能提升的一种方法”

Python metaclass


学懂元类,你只需要知道两句话:

  • 道生一,一生二,二生三,三生万物
  • 我是谁?我从哪来里?我要到哪里去?

在Python世界,拥有一个永恒的道,那就是“type”,请记在脑海中,type就是道。如此广袤无垠的python生态圈,都是由type产生出来的。 Continue reading “Python metaclass”