Sorting
Summary: We consider a variety of techniques used to put a list or vector in order, using a binary comparison operation to determine the ordering of pairs of elements.
The Problem of Sorting
Sorting a collection of values -- arranging them in a fixed order, usually alphabetical or numerical -- is one of the most common computing applications. When the number of values is even moderately large, sorting is such a tiresome, error-prone, and time-consuming process for human beings that the programmer should automate it whenever possible. For this reason, computer scientists have studied this application with extreme care and thoroughness.
One of the clear results of their investigations is that no one algorithm for sorting is best in all cases. Which approach is best depends on whether one is sorting a small collection or a large one, on whether the individual elements occupy a lot of storage (so that moving them around in memory is time-consuming), on how easy or hard it is to compare two elements to figure out which one should precede the other, and so on. In this course we'll be looking at two of the most generally useful algorithms for sorting: insertion sort, which is the subject of this reading, and merge sort, which we will consider in another reading. In our general exploration of sorting, we may also discuss other sorting algorithms.
Imagine first that we're given a collection of values and a rule
for arranging them. The values might actually be stored in a
list, vector, or file. Let's assume first that they are in a list.
The rule for arranging them typically takes the form of a predicate with
two parameters that can be applied to any two values in the collection
to determine whether the first of them could precede the second when
the values have been sorted. (For example, if one wants to sort a
collection of real numbers into ascending numerical order, the rule
should be the predicate <=; if one wants to
sort a collection of strings into alphabetical order, ignoring case,
the rule should be string-ci<=?; if one wants
to sort a collection of real numbers into descending numerical order,
the rule should be >=; and so on.)
The Insertion Sort Algorithm
As one might guess, the key operation in insertion sort is that of insertion. We envision separating the elements to be sorted into two collections: those that are not yet sorted, and those that are already sorted. We repeatedly insert one value from the unsorted collection into the proper place in the sorted collection.
The way in which we represent the unsorted and sorted collections has an effect on the way in which we implement the insertion operation. Let's start by representing collections as lists. Midway through the sorting process, we have one list of unsorted values (the values we have not yet processed) and a second, sorted, list. To insert a value into the sorted list, we step over the elements that should precede the value, stopping when we find an element that should follow the value (or when we run out of elements). We then cons the new value onto that section of the list and rebuild the prior elements.
Inserting Elements
We'll start by considering the specific case of sorting a list of real numbers in ascending order.
We begin with the procedure used to insert a new value into a list that
is already in order. Because we're inserting the numbers in ascending
order, we can use <= as the ordering predicate.
;;; Procedure:
;;; insert-number
;;; Parameters:
;;; sorted, a list of real numbers
;;; new-element, a real number
;;; Purpose:
;;; Insert new-element into sorted.
;;; Produces:
;;; new-ls, a new list of real numbers
;;; Preconditions:
;;; sorted is a list of numbers arranged in increasing order. That is,
;;; (<= (list-ref sorted i) (list-ref sorted (+ i 1)))
;;; for all reasonable values of i. [Unverified]
;;; new-element is a number. [Unverified]
;;; Postconditions:
;;; new-ls is a list of numbers arranged in increasing order.
;;; new-ls is a permutation of (cons new-element sorted).
(define insert-number
(lambda (sorted new-element)
(cond
[(null? sorted)
(list new-element)]
[(<= new-element (car sorted))
(cons new-element sorted)]
[else
(cons (car sorted)
(insert-number (cdr sorted) new-element))])))
In English: If the list into which the new element is to be inserted is empty, return a list containing only the new element. If the new element can precede the first element of the existing list, then, since the existing list is assumed to be sorted already, it must also be able to precede every element of the existing list, so attach the new element onto the front of the existing list and return the result. Otherwise, we haven't yet found the place, so issue a recursive call to insert the new element into the cdr of the current list, then re-attach its car at the beginning of the result.
Sorting a list
Now let's return to the overall process of sorting an entire list. The insertion sort algorithm simply takes up the elements of the list to be sorted one by one and inserts each one into a new list, which is initially empty:
;;; Procedure:
;;; numbers-insertion-sort
;;; Parameters:
;;; numbers, a list of real numbers
;;; Purpose:
;;; Sorts numbers
;;; Produces:
;;; sorted, a list of real numbers
;;; Preconditions:
;;; (none)
;;; Postconditions:
;;; sorted is a list of real numbers.
;;; sorted is organized in increasing order. That is,
;;; (<= (list-ref sorted i) (list-ref sorted (+ i 1)))
;;; for all reasonable values of i.
;;; sorted is a permutation of numbers.
(define numbers-insertion-sort
(lambda (numbers)
(let kernel ([unsorted numbers] ; The remaining unsorted values
[sorted null]) ; The sorted values
(if (null? unsorted)
sorted
(kernel (cdr unsorted) (insert-number sorted (car unsorted)))))))
Formalizing the Problem of Sorting
Of course, the insertion sort procedure we just wrote works only
for lists of real numbers, and only sorts such lists into increasing
order. What if we want a more general version? As you may recall,
we were able to generalize the process of searching by adding a
may-precede? parameter. We can add a similar
parameter to our sorting routine.
Rather than moving directly to a generalized implementation of
insertion-sort, let's first develop the
documentation for that procedure. The name, parameters, purpose,
and produced value are fairly straightforward.
;;; Procedure: ;;; list-______-sort ;;; Parameters: ;;; lst, a list to be sorted ;;; may-precede?, a binary predicate ;;; Purpose: ;;; Sort lst. ;;; Produces: ;;; sorted, a list.
Now, on to the harder part. What conditions must hold in order for us
to be able to sort? Well, we need to be able to use
may-precede? to compare values. That means that we
must be able to apply it to any two elements of the list. We also
want to make sure that the may-precede? predicate
is sensible. That is, it should be transitive (if a may precede b and
b may precede c, then a may precede c) and must order any two elements
(for any two elements, a may precede b or b may precede a). What should
we do about equal elements? We'll say that each may precede the other.
;;; Preconditions: ;;; may-precede? can be used with the elements of lst. That is for ;;; all values a and b in lst, (may-precede? a b) successfully ;;; returns a truth value. ;;; may-precede? is transitive. That is, for all values a, b, and ;;; c in lst, if (may-precede? a b) and (may-precede? b c), then ;;; (may-precede? a c). ;;; may-precede? is sensible. That is, for all values a and b, ;;; either (may-precede? a b), (may-precede? b a), or both.
All that is left is for us to formalize the postconditions. What does it
mean to sort a list? The result must be in order (which we specify using
may-precede?). In addition, sorting should neither add
nor remove values from the list. The easiest way to formalize that idea
is to say that the result must be a permutation of the original list.
We might also indicate that we do not modify the original list, but that's
implicit. (That is, we traditionally only indicate when we modify
parameters, not when we fail to modify them.)
;;; Postconditions: ;;; sorted is sorted by may-precede?. That is, for all i such that ;;; 0 <= i < (- (length lst) 1), ;;; (may-precede? (list-ref sorted i) (list-ref sorted (+ i 1))) ;;; sorted is a permutation of lst.
As you may recall, we also looked at searching through collections of keyed values for a value with a particular key. Since we search for keyed values, we might also want to sort by key, and use the key as an explicit parameter to the procedure.
;;; Procedure: ;;; list-keyed-_____-sort ;;; Parameters: ;;; lst, a list ;;; get-key, a procedure ;;; may-precede?, a binary predicate ;;; Purpose: ;;; Sort lst. ;;; Produces: ;;; sorted, a list ;;; Preconditions: ;;; get-key? can be applied to each element of lst. ;;; may-precede? can be used with the values returned by get-key. That is ;;; for all values a and b in lst, (may-precede? (get-key a) (get-key b)) ;;; successfully returns a truth value. ;;; may-precede? is transitive. That is, for all keys a, b, and ;;; c, if (may-precede? a b) and (may-precede? b c), then ;;; (may-precede? a c). ;;; may-precede? is sensible. That is, for all keys a and b, ;;; (may-precede? a b) holds, (may-precede? b a) holds, or both ;;; hold. ;;; Postconditions: ;;; sorted is sorted by key using may-precede?. That is, for all i ;;; such that 0 <= i < (- (length lst) 1), ;;; (may-precede? (get-key (list-ref sorted i)) ;;; (get-key (list-ref sorted (+ i 1)))) ;;; sorted is a permutation of lst.
Insertion Sort, Generalized
Okay, let's now implement the two forms of generalized insertion sort.
This time, we'll make the insert procedure local,
since it need not be used outside of insertion sort.
(define list-insertion-sort
(lambda (lst may-precede?)
(letrec ([insert
(lambda (lst val)
(cond
[(null? lst)
(list val)]
[(may-precede? val (car lst))
(cons val lst)]
[else
(cons (car lst) (insert (cdr lst) val))]))]
[kernel
(lambda (unsorted sorted)
(if (null? unsorted)
sorted
(kernel (cdr unsorted) (insert sorted (car unsorted)))))])
(kernel lst null))))
What changes do we need to make for keyed insertion sort? Not many.
The only differences are (a) we need to add the get-key
parameter and (b) every time we used may-precede?, we
must now add calls to get-key. Fortunately, there's
only one call to may-precede?, so the update is
small.
(define list-keyed-insertion-sort
(lambda (lst get-key may-precede?)
(letrec ([insert
(lambda (lst val)
(cond
((null? lst)
(list val))
((may-precede? (get-key val) (get-key (car lst)))
(cons val lst))
(else
(cons (car lst) (insert (cdr lst) val)))))]
[kernel
(lambda (unsorted sorted)
(if (null? unsorted)
sorted
(kernel (cdr unsorted) (insert sorted (car unsorted)))))])
(kernel lst null))))
Of course, this code is quite close to that of list-insertion-sort.
Hence, rather than doing a cut-paste-edit on that code, we might instead
call list-insertion-sort, providing
it with a predicate that extracts keys and then compares.
(define list-keyed-insertion-sort
(lambda (lst get-key may-precede?)
(list-insertion-sort lst
(lambda (v1 v2)
(may-precede? (get-key v1) (get-key v2))))))
Why use the second strategy? One obvious reason is that
it's smaller. A second is that it may actually be less work
to write this version than to figure out what to change in
list-insertion-sort. But the best reason to use
this strategy (that is, calling another function with a well-designed
parameter, rather than cutting, pasting, and changing) is that it
makes updates much easier. If we realize that we made a mistake in
list-insertion-sort or simply find a better way
to do some part of it, we only have one procedure to update, rather
than two.
Sorting a Vector
Of course, as we saw in our exploration of binary search, in order to do binary search, we need a sorted vector, rather than a sorted list. To sort a vector, we could turn that vector into a list, sort the list, and then build a new vector.
However, in sorting a vector, the goal of sorting is often to overwrite the old arrangement of those values with a new, sorted arrangement of the same values. This type of sorting is often called in-place sorting. Let's consider how we might use the ideas of insertion sort to sort a vector in place.
As you may recall, insertion sort requires us to have two collections: one of which is sorted and the other of which is unsorted. We will partition the original vector into two sub-vectors: a sorted sub-vector, in which all of the elements are in the correct order relative to one another, and an unsorted sub-vector in which the elements are still in their original positions. The two sub-vectors are not actually separated; instead, we just keep track of a boundary between them inside the original vector. Items to the left of the boundary are in the sorted sub-vector; items to its right, in the unsorted one. Initially the boundary is at the left end of the vector. The plan is to shift the boundary, one position at a time, to the right end. When it arrives, the entire vector has been sorted.
Here's the plan for the main algorithm.
;;; Procedure:
;;; vector-insertion-sort!
;;; Parameters:
;;; vec, a vector
;;; may-precede?, a binary predicate
;;; Purpose:
;;; Sorts the vector.
;;; Produces:
;;; [Nothing; sorts in place]
;;; Preconditions:
;;; vec is a vector.
;;; may-precede? can be applied to any two elements of vec.
;;; may-precede? is transitive.
;;; Postconditions:
;;; The final state of vec is a permutation of the original state.
;;; vec is sorted. That is,
;;; (may-precede? (vector-ref vec i) (vector-ref vec (+ i 1)))
;;; for all reasonable values of i.
(define vector-insertion-sort!
(lambda (vec may-precede?)
(let ([len (vector-length vec)])
(let kernel ([boundary 1]) ; The index of the first unsorted value
(cond
[(< boundary len) ; If we have elements left to sort
(vector-insert! vec
(vector-ref vec boundary)
boundary
may-precede?)
(kernel (+ boundary 1))]
[else
vec])))))
The insert! procedure takes four parameters: an element
to be inserted into the sorted part of the vector, the vector itself,
the current boundary position, and the comparison procedure. The new
element can be inserted at any position up to and including the current
boundary position, but it must be placed in the correct order relative
to elements to the left of that boundary. This means that any elements
that should follow the new one should be shifted one position to the
right in order to make room for the new one. (Elements that precede
the new one can keep their current positions.)
;;; Procedure:
;;; vector-insert!
;;; Parameters:
;;; vec, a vector of values
;;; new-element, a value
;;; boundary, an index into the vector
;;; may-precede?, a binary predicate
;;; Purpose:
;;; Insert new-element into the portion of vec between 0 and
;;; boundary, inclusive.
;;; Produces:
;;; [Nothing; called for side effects.]
;;; Preconditions:
;;; 0 <= boundary < (vector-length vec)
;;; The elements in positions 0..boundary-1 of vec are sorted.
;;; That is, (may-precede? (vector-ref vec i) (vector-ref vec (+ i 1)))
;;; for all 0 <= i < boundary-2.
;;; may-precede? is transitive and sensible.
;;; Postconditions:
;;; The elements in positions 0..boundary of vec are sorted.
;;; That is, (may-precede? (vector-ref vec i) (vector-ref vec (+ i 1)))
;;; for all 0 <= i < boundary.
;;; The elements in positions 0..boundary of vec after insert! finishes
;;; are a permutation of new-element and the elements that were in
;;; positions 0..(boundary-1) before the procedure started.
(define vector-insert!
(lambda (vec new-element boundary may-precede?)
(let kernel ((pos boundary))
(cond
; If we've reached the left end of the vector, we've run out of
; elements to shift. Insert the new element.
[(zero? pos)
(vector-set! vec pos new-element)]
; If we've reached a point at which the element to the left
; is smaller, we insert the new element here.
[(may-precede? (vector-ref vec (- pos 1)) new-element)
(vector-set! vec pos new-element)]
; Otherwise, we shift the current element to the right and
; continue.
[else
(vector-set! vec pos (vector-ref vec (- pos 1)))
(kernel (- pos 1))]))))
How does this work? We assume that there's a “space” at position
pos of the vector. (That is, that we can safely insert
something there without removing anything from the vector.) We know
that the condition holds at the beginning from the description. That is,
the postcondition specifically ignores the value that was in the
boundary position. We also know that the conditional holds from the way
insert! was called from insertion-sort!, since
the boundary is initially the position of the value we insert.
Now, what do we do? If the position is at the left end of the vector,
there's nothing smaller in the vector, so we just put the new value there.
If the thing to the left of the current position is smaller, we know we've
reached the right place, so we put the value there. In every other case,
the value to the left is larger than the value we want to insert, so we
shift that value right (into the pos position) and continue
working one position to the left. Since we've copied the value to the right,
it is remains safe to insert something in the position just vacated
(that is, pos-1).
You will have a chance to explore this procedure further in the laboratory.
Self Checks
These checks might take you a little bit longer, but they're not complex. However, they are importantly designed to help you understand sorting before starting the lab, so please make your best effort to complete them.
Check 1: Verifying Insert
a. Make a copy of insertion-sort-lab.rkt.
b. Test the insert-number procedure from
the reading by inserting the number 42
- into an empty list;
- into a list of numbers larger than 42, arranged in ascending order;
- into a list of numbers smaller than 42, arranged in ascending order;
- into a list of numbers both smaller and larger than 42, arranged in ascending order; and
-
into a list that contains only three copies of 42 (that is, the list created by
(list 42 42 42).
c. Why do you think we had you do each of these tests. (That is,
why would one want to check that insert-number
works on each of these lists?)
d. What would you expect to happen if the list is
not in ascending order when
insert-number is invoked?
e. Check your answer experimentally.
Check 2: Displaying Steps in Insertion Sort
a. Add calls to the display and
newline procedures to the body of the local
helper kernel of
numbers-insertion-sort, so that it displays the
values of unsorted and sorted,
appropriately labeled, at each step of the sorting process.
For example, you might write
(display (list 'sorted sorted 'unsorted unsorted)) (newline)
b. To sort the values 7, 6, 12, 4, 10, 8, 5, and 1 (given in that
order), what will the values of sorted and
unsorted be for the first few iterations of the kernel?
c. Use your revised numbers-insertion-sort
procedure to verify your prediction and see all the iterations.
Check 3: Checking Potential Problems
a. When we use a new procedure, we often want to test it on a
variety of cases. We've seen that
numbers-insertion-sort works on a few simple
cases. But we should also check some “special cases”,
cases that might stress the algorithm. Come up with some lists that
a poorly-implemented insertion sort procedure might have difficulty
with.
b. As some potential problems, predict what
numbers-insertion-sort will do for each of the
following cases. Then check your answer experimentally.
- An empty list.
- A list containing only one element.
- A list containing all equal values.
- A list in which the elements are originally in descending numerical order.
- A list in which the elements are already in ascending numerical order.
- A list containing some duplicate elements.