driven actions or system specific 3. Warn – conditions that could become an error and will likely trigger an alert 4. Error – error conditions such as API failures, internal issues 5. Fatal – forces a termination Everyone (development, managers, architects, ops, etc.) in the value stream shares responsibility for handling operational incidents and gets visceral feedback on upstream decisions. (Pedro Canahuati, Facebook 2001 study – “paired programmers are 15% slower than two independent individual programmers, while ‘error-free’ code increased from 70% to 85%. Since testing and debugging are often many times more costly

Bad Apple Theory – you cannot eliminate error by eliminating the people who caused the error. 1. “Human error is not our cause of troubles; instead, human error is a consequence of the design of the

fixes errors in a separate test phase. b. Delayed feedback fades the cause & effect link of the error; now requires firefighting and archaeology. c. Worst effect – lose the ability to learn from mistakes to find the error as early as possible, in the fastest category as possible ii. Later stages require more scarce and complex environment to detect and reproduce issues iii. When an error is found, create create a test in an earlier, faster stage to detect the error iv. Ideal vs. Non-Ideal Testing Pyramids h. ENSURE TESTS RUN QUICKLY (IN PARALLEL, IF NECESSARY) i. Design tests to run in parallel ii

whose specification has to be givenProgram verification • Program verification does not mean error-proof programs • There is no magic here either, mathematical proofs can be faulty • Even perfect

Examples of ineffective quality controls include: a. Requiring another team to complete tedious, error-prone, and manual tasks that could be easily automated and run as needed by the team who needs the

by these early Agile adopters. Broader, more successful Agile execution will take time, trial and error, and shaping of processes, policies, and culture, but with support from oversight and process owners