Chapter 2

These two sections of chapter two deals with the efficiency of algorithms, as well as how an algorithm's running time changes as its input size changes. Both of these are important concepts, as algorithms must work quickly no matter the scale of the problem.

This section is all about efficiency, even taking several attempts at defining it. The first attempt reads as follows:

“Proposed Definition of Efficiency (1): An algorithm is efficient if, when implemented, it runs quickly on in real input instances.“

This definition does not work because it has no concept of scale. An inefficient process could meet this standard if run on a very small data sample, since the computer could process things quickly. The next attempt reads as:

“Proposed Definition of Efficiency (2): An algorithm is efficient if it achieves qualitatively better worst-case performance, at an analytical level, than brute-force search.“

This definition is simply too vague to work. It does not provide guidelines for that counts as “qualitatively better worst-case performance”, and is thus subjective. The final definition fixes this vagueness by providing a way of quantifying efficiency:

“Proposed Definition of Efficiency (3): An algorithm is efficient if it has a polynomial running time.“

This is the ideal definition for our purposes because of its specificity. It allows us to look at algorithms quantitatively so that we can determine whether a process is feasible. This definition also makes problems negatable. Thus, using this definition, we can decide when there is no efficient algorithm for a problem.

This section looks at how an algorithm's worst-case running time grows as the input size grows. For our purposes as computer scientists, we do not need to stress out about finding the exact number of steps it takes to solve a problem. Instead, we make accurate generalizations in the form of O-notation so that algorithms' run times become easier to compare. This section begins by describing the specific process through which we can define an algorithm's upper and lower bounds, then describes how to find an algorithm's “tight bounds”. An algorithm has tight bounds if its upper-bound and lower-bound are the same function. This bound can sometimes also be found by calculating a limit as n goes to infinity.

The book then goes on to mention some of the properties of the asymptotic growth rates. It discusses transitivity, and sums of functions.

This section practically applies all of the ideas we have discussed thus far. It proves that the Stable Matching problem can be solved by an algorithm that runs in O(n²) time.

But first, the book discusses the nuances of two basic data structures that will be imperative to this algorithm: array lists and linked lists. Array lists are good for finding the i^th item in the list in constant time. However, they do not resize dynamically and thus end up taking more space than they need. The book decides that the men and women's preferences would be well contained in an array list because the list will never change sizes, and being able to find the i^th item in constant time would be useful. Linked lists, on the other hand, must find the i^th item in O(i) time because it has to follow the links all the way to node that contains the item. However, linked lists are dynamically sized, so the would be a good implementation for storing the list of single men. Since the man you start with is arbitrary anyway, we can access him in O© time, so the time penalty is rendered inert.

This section goes on to give a basic implementation of the algorithm. In order to keep runtime under O(n²), we must be able to perform four operations in constant time:

   (1) Identify a free man.
   (2) For any given man, identify the highest-ranked woman to whom he has not yet proposed.
   (3) For any given woman, we must be able to identify whether or not she has a partner, and identify her partner.
   (4) When presented two men, decide which of the two a given woman prefers.

If these four operations can be performed in constant time, the algorithm will perform correctly in O(n²) time.

Overall, this section was more interesting than others because it practically applied the theories and proofs we have discussed.

This section covers the primary running times, and how to achieve them. Throughout the section, it primarily compares each running time to the “brute force” method.

It starts off with the most intuitive running time, linear (O(n)). This algorithm is relatively easy to achieve, as just limiting the algorithm to one pass over the data. A single for-loop will result in linear running time, as long as there is not another loop nested within.

The next running time discussed is O(n log n). This running time is marginally worse than linear. Algorithms that operate under this running time usually sort the data first in O(log n) time, and then deal with the sorted data in O(n) time. The result yields O(n log n).

The section then discusses quadratic running time (O(n²)). This running time naturally arises out of nested for-loops, i.e. an O(n) operation repeated an O(n) number of times.