Algorithmic Complexity Amir Kirsh & Adam Segoli Schubert © All rights reserved image source: https://www.ac.tuwien.ac.at/people/szeider/cartoon Algorithmic Complexity @ CppCon 2021 Why this talk? 2Algorithmic Complexity @ CppCon 2021 Why this talk? Performance is the name of the game 3 Algorithmic Complexity @ CppCon 2021 Why this talk? Performance is the name of the game You all (hopefully) (hopefully) know that O(n) is better than O(n2) 4Algorithmic Complexity @ CppCon 2021 Performance is the name of the game You all (hopefully) know that O(n) is better than O(n2) But there is still important

Amortized O(1) Complexity Andreas Weis CppCon 20242/32 Runtime Complexity f ∈ O(g) ⇐⇒ ∃ C > 0. ∃ x0 > 0. ∀ x > x0 : |f (x)| ≤ C · |g(x)|3/32 Child’s play!4/32 What is amortized complexity?4/32 What amortized complexity?4/32 What is amortized complexity?4/32 What is amortized complexity?4/32 What is amortized complexity?4/32 What is amortized complexity?4/32 What is amortized complexity?5/32 Amortized push_back (3);10/32 Cost of vector::push back v.push_back (3);11/32 Worst case complexity?11/32 Worst case complexity? O(n) n: Number of elements in the vector12/32 push back with an O(n) budget

Quantifying Accidental Complexity: An empirical look at teaching and using C++ Herb Sutter 9/21/2020 1 Herb Sutter 4 We’re “paying taxes” all the time Productivity Correctness and quality Tooling Tooling Teaching, learning, hiring, training 2 4Quantifying Accidental Complexity: An empirical look at teaching and using C++ Herb Sutter 9/21/2020 2 5 Common claim: “C++ is too complex” This talk's count 6 Essential complexity Inherent in the problem, present in any solution Accidental complexity Artifact of a specific solution design 5 6Quantifying Accidental Complexity: An empirical look

of the software task, those concerned with fashioning abstract conceptual structures of great complexity. I suggest: • Using rapid prototyping as part of a planned iteration in establishing software properties of this irreducible essence of modern software systems: complexity, conformity, changeability, and invisibility.”Complexity • Software entities are more complex for their size than perhaps no two parts are alike • The complexity of software is an essential property, not an accidental one • Descriptions of a software entity that abstract away its complexity often abstract away its essence

• Semantically based interoperability guarantees • Efficient • No penalty for generality • Complexity guarantees at the interface level • Natural • C/C++ machine model and programming paradigm • • Semantically based interoperability guarantees • Efficient • No penalty for generality • Complexity guarantees at the interface level • Natural • C/C++ machine model and programming paradigm • Copyright © 2021 Bob Steagall Complexity and the Big-O Notation • Complexity refers to the runtime cost of an algorithm • Big-O notation expresses the relative complexity of an algorithm 16 Type Notation

predictability and consistent behavior (once pattern is recognized) Enables system scaling (in size and complexity)Charley Bay - charleyb123 at gmail dot com The Roles of Symmetry And Orthogonality In Design cppcon obvious for where a desired action should be placed • Are the steps guaranteed to occur? • If yes, complexity is reduced (edge cases are removed) • Can a step be “empty” (e.g., “do nothing” or “default” behavior cases and complexity: • Can a step be conditionally skipped? • Can the steps be reordered? • Can new (user-custom) steps be inserted? ctor() dtor() … Increasing Complexity Decreasing Complexity Domain-specific

priorities, or technical considerations. Story Points – Unit of measurement to estimate relative complexity of user stories Velocity – The amount of work the team estimates it can deliver in a sprint representatives, analyze each user story and use measures known as story points to estimate the relative complexity of developing that capability. The development team assigns points to each story following discussion support frequent collaboration. One or a few contractor(s) or teams can perform development. Complexity Many contractors are required to develop program elements. Program can leverage test infrastructure

QUESTIONS 1. Am I using any standard algorithms? Which concept(s) do they require? also informs complexity guarantees, maybe different for different concepts 30 . 1ITERATOR CONCEPT QUESTIONS ITERATOR CONCEPT QUESTIONS 1. Am I using any standard algorithms? Which concept(s) do they require? also informs complexity guarantees, maybe different for different concepts 2. Do I look at an element after moving past QUESTIONS 1. Am I using any standard algorithms? Which concept(s) do they require? also informs complexity guarantees, maybe different for different concepts 2. Do I look at an element after moving past

September 6, 2023 50 / 57 The Curse of Dimensionality Handling complexity Involve many variables, how can we handle this complexity without get- ting into trouble. Optimization and Integration Usually Overview September 6, 2023 51 / 57 How to Handle Complexity Properly dealing with complexity is a crucial issue for machine learning. Limiting complexity is one approach Use a model that is complex enough if can find out how to reduce the large number of variables to a small number. Averaging over complexity is the Bayesian approach. Use as complex a model might be needed, but don’t choose a single parameter

constant complexity – begin() has constant complexity – end() has constant complexity • Guarantees for Views – Initialization has constant complexity – begin() has amortized constant complexity – end() end() has amortized constant complexity [range.range]: Given an expression t such that decltype((t)) is T&, T models concept std::range only if ... (3.2) — both ranges::begin(t) and ranges::end(t) are amortized constant complexity • begin() is thread safe (using mutable) – Performance issue: Makes begin() very expensive • No caching at all – Performance issue: Some use cases have quadratic complexity • Reverse