Asymptotic analysis & growth rates

What Big O tells you

The supplied lecture emphasizes scaling: how the resource requirement changes when more data is supplied. Time complexity models work; space complexity models memory. Neither directly gives a wall-clock duration without additional assumptions.

Constants can matter for small inputs, while a larger growth rate eventually dominates any fixed constant. Therefore benchmark practical sizes and use asymptotic analysis to understand the trend.

Big O: an eventual upper bound

For nonnegative cost functions, f(n) is O(g(n)) if there exist constants c > 0 and n₀ such that f(n) ≤ c g(n) for every n ≥ n₀. The constants must be independent of n.

For f(n) = 3n + 2, take c = 5 and n₀ = 1 to prove O(n). This function is also O(n²), but the linear bound is tighter. Big O does not inherently mean worst case: specify which cost function you are bounding.

Big Omega and Big Theta

Ω(g(n)) is an eventual lower bound. Θ(g(n)) gives matching upper and lower bounds up to positive constant factors. A statement of Θ(n²) describes a tight growth class.

For 3n + 2 and n ≥ 1, the function lies between 3n and 5n. That proves Θ(n). An algorithm’s best-case time and a problem’s lower bound are different concepts; do not use the terms interchangeably.

Reading loops as sums

A loop that increments i once from 0 to n−1 performs n iterations. If an inner loop also runs n times for each outer iteration, multiply to get n². If the inner bound is i, sum the varying inner counts to get a triangular number.

When a positive counter doubles each iteration, it reaches n after about log₂ n iterations. If the counter increases by consecutive integers, its value after k steps is k(k+1)/2, so reaching n takes Θ(√n) steps.

• Do not infer complexity from the number of loops alone.
• An inner loop may have a bound independent of n.
• Early return can make best and worst cases different.
• Separate preprocessing from per-query work.

Logarithms count repeated division

The equation 2ᵏ = n is equivalent to k = log₂ n. A sorted region of 1024 items can be halved ten times to reach one item. A final inspection may still be needed, depending on the exact search convention.

Changing the logarithm base multiplies it by a constant, which does not affect Θ(log n). The supplied workbooks retain the original examples, formulas and cached values for studying powers and logarithms.

Sorting and partitioning

The source references insertion, bubble and selection sorting and emphasizes partitioning. Selection sort repeatedly selects the next minimum and performs quadratic comparisons. Insertion sort inserts each next element into an already sorted prefix; it can be linear on already sorted input.

Merge sort divides the input, sorts each half and merges in linear work per level, giving Θ(n log n). Quicksort partitions around a pivot. Balanced partitions lead to Θ(n log n), while repeatedly extreme partitions can give Θ(n²). The supplied Quick implementation shuffles before partitioning, so its expected performance assumes that randomization works as intended.

Three bubble-sort variants in the source

Pages 29–45 develop bubble sort by comparing adjacent values and exchanging inverted pairs. The first pseudocode runs n−1 passes, each with n−1 comparisons, for (n−1)² comparisons even if the input is already sorted.

The first enhancement shortens the inner loop after each pass because the largest remaining value has reached its final position. Its comparison count becomes (n−1) + (n−2) + … + 1 = n(n−1)/2. This reduces the count but keeps Θ(n²) growth.

The second enhancement records whether any exchange occurs. A pass with no exchanges proves the array is sorted, permitting early termination. Already sorted input then takes Θ(n) time, while reverse order still takes Θ(n²). All three versions use Θ(1) auxiliary space. The exact original pseudocode remains on the source slides.

• Trace [5, 3, 8, 1]: the first pass gives [3, 5, 1, 8].
• The next pass gives [3, 1, 5, 8], and the third gives [1, 3, 5, 8].
• A strict greater-than comparison preserves the relative order of equal keys, making the adjacent-swap version stable.
• A smaller leading coefficient is a practical improvement even when the asymptotic class stays the same.

The source’s merge and partition traces

The merge-sort slides show eight elements passing through three merge levels: 8 × 3 = 24 append operations in that illustrative merge-only cost model. Since log₂ 8 = 3, this makes the n log₂ n pattern visible. Comparisons, allocation and copying can add work without changing the growth class.

The quicksort sequence begins with [5, 2, 7, 6, 1, 9, 4, 8] and illustrates partitioning around 5. Subsequent slides show an extreme partition pattern with 7 + 6 + 5 + 4 + 3 + 2 + 1 = 28 operations for eight items. The general triangular sum explains the quadratic worst case.

The source’s recursive QuickSort(Data, Left, Right) stops when Left is no longer less than Right, and recurses on intervals ending before and starting after PivotPosition. Excluding the pivot is what ensures progress. The preserved slide walkthrough and the Java Quick implementation use their own partition details; do not assume their exact swap sequences are identical.

Remember for revision

Define the function and the input model before writing a bound. O is an upper bound, Ω a lower bound and Θ a tight bound. Show the loop count, sum or recurrence that justifies the result.

Original implementations

Read the complete original code and its documentation. Core algorithm pages add a walkthrough, complexity discussion and a worked example.

Check your understanding

Supplementary practice. Try each question before opening the answer.