COMS21103: Summary of Lecture 4
Little “oh”
We’ve started by looking at strict upper bounds, so we defined o(g) to mean strict upper bounds.
This notion is defined properly in the notes for lecture 3.
Asymptotic analysis
We started by looking at the Insertion sort algorithm. The algorithm is presented at page at page
17 of CLRS. A more careful analysis than we have done in class, but which essentially uses the
abstractions that we have discussed is given at pages 23-25.
Pseudocode. The algorithm is written in pseudocode, a very convenient way of writing algorithms
so that we can focus on the substance of the algorithms and not get bogged down in the details of
the implementation.
Random Access Machine(RAM). We analyze the execution of such algorithm on an idealized
machine called the Random Access Machine (RAM). This machine has an infinite amount of mem-
ory, the basic instructions (i.e. assignment) take constant time, and accessing any memory location
also takes constant time.
I made the following points in the first lecture, but let me restate them here, since this is the
proper context.
Worst-case analysis. We will mostly be concerned with worst-case asymptotic behaviour of
algorithms. This means that for each algorithm we have to determine the time that the algorithm
takes on the worst possible input (i.e. the input that makes the algorithm run longer). This analysis
guarantees upper-bounds on the running time of the algorithm (and upper-bounds is really what
we care for). Notice that it’s important to give as tight of an upper-bound as possible: it’s better
to know that the running time of an algorithm is O(n2) than O(n3).
Average case analysis. This is an alternative to worst-case analysis where, if we know that the
inputs to the algorithm come from some distribution, we look at the time that the algorithm is
expected to run, when its inputs are selected from the distribution.
Here’s an example that should clarify what this means: Say that we have some algorithm A
for which the inputs come from some domain D={x1, x2, . . . , xl}, and assume that on input xi
the algorithm takes time ti. Imagine that the input that is actually passed to the algorithm is
determined by a distribution on D, meaning that the input xiis passed to Awith some probability
pi. Then, the expected running time of Ais how much time it takes on the average, i.e.:
l
X
i=1
pi·ti
1
