of our deployment pipeline ii. Enabling fast & reliable automated testing iii. Enabling & practicing continuous integration & testing iv. Automating, enabling, and architecting for low-risk releases mastery of application & environment interactions 3. Ch. 10 Enable Fast and Reliable Automated Testing a. Undesired outcomes result from finding and fixes errors in a separate test phase. b. Delayed Hard line: no changes would be accepted into GWS without accompanying automated tests. Spread Testing Grouplet team approach. Google – single, share repository of billions of files. 50% of code is changed

production services become fragile 5. Ch. 17 – Integrate Hypothesis-Driven Development and A/B Testing into Our Daily Work a. INTRODUCTION i. Jez Humble, “The most inefficient way to test a business the 3 month US tax season in 2010 with website conversion rates up 50% b. A BRIEF HISTORY OF A/B TESTING i. Pioneered in direct response marketing ii. Early efforts required sending thousands of postcards/flyers Campaigns would modify text, layouts, packaging, etc. 2. Very expensive c. INTEGRATING A/B TESTING INTO OUR FEATURE TESTING i. Random subset of users shown differing versions. User groups define a cohort. Statistical

US-EAST availability zone went down; Netflix was unaffected. Since 2008 they were building and testing with the Chaos Monkey; they architected for failure, tested for failure, and evolved beyond it usually dangerous—ones happen to emerge.” iii. Resilience – requires defining failure modes and then testing to ensure they operate as expected iv. Great Amazon Reboot of 2014 – 10% of Amazon EC2 servers Configuration standards for libraries, infrastructure, and environments 2. Deployment tools 3. Testing standards and tools, including security 4. Deployment pipeline tools 5. Monitoring and analysis

other approach as well and compare. We are comfortable with the ideathat we will break things in testing, because the end result of doing so is that we can make a decision more quickly and with more good used to be natural for ops specialists to do ops, developers to do development, and testers to do testing. Now, crossover skills are increasingly important. On a delivery team, employees will help each each other across roles in the interest of shipping code. If there is a backlog in exploratory testing, people who normally do development will help test. Software engineers will oversee their code in production

deploy, operate, and monitor. In DevSecOps, testing and security are shifted to the left through automated unit, functional, integration, and security testing - this is a key DevSecOps differentiator since

not in its organization structures, but in developing capability and habits in its people.” f. TESTING, OPERATIONS, AND SECURITY AS EVERYONE’S JOB, EVERY DAY i. In high-performing organizations – shared