Return to the Lecture Notes Index

15-200 Lecture 17 (Monday, March 6, 2006)

An Introduction to Sorts

You probably did at least a small amount of sorting in your introductory course. Sorting a collection of items is an important computing application. There are many commonly-used sorting algorithms. Some of them are good for sorting small sets of data, while others are good for sorting large sets of data. Some are good under certain conditions and horrid in other conditions. Some take about the same length of time to run no matter what the conditions. Some are simple, others are a bit more complicated.

In order to keep things simple and focus our attention on sorting, we'll sort an array of integers, from lowest to highest, in our examples. In reality, you can sort any collection of Objects (remember Comparable).

Bubble Sort

A bubble sort traverses the array, or a selected portion of the array, length-1 times (a less efficient version of the bubble sort could pass through the array length times and still produce a sorted array). With each pass through the array, the bubble sort compares adjacent values, swapping them if necessary. This actually results in "bubbling" the highest value in the array up to the highest index (length-1) by the end of the first pass through the array, the second-highest value up to the second-highest index (length-2) after the second pass through the array, and so on. By the time the bubble sort has made length-1 passes through the array (or a selected portion of the array), every item, including the lowest item, is guaranteed to be in its proper place in the array.

Let's see how the bubble sort works on an array of integers. We start at the lowest pair of indexes, and work our way up to the highest.

The following shows one entire pass through the array using a bubble sort. Turn your head sideways to the left to read this array. The array indexes are the left-hand, red column. The two adjacent items currently being compared are bold. Note that there are eight total items in the array. Notice that we will make seven comparisons (there are eight columns after the red column) during this first pass.

7 7 7 7 7 7 7 7 9
6 4 4 4 4 4 4 9 7
5 5 5 5 5 5 9 4 4
4 6 6 6 6 9 5 5 5
3 3 3 3 9 6 6 6 6
2 1 1 9 3 3 3 3 3
1 2 9 1 1 1 1 1 1
0 9 2 2 2 2 2 2 2

What has happened to the 9 that started at index 0? It happened to be the highest value in the array, so every time we compared adjacent values we had to swap until the 9 was finally in the last position in the array. After one pass through the array, we have successfully moved the highest value into the highest index.

On our next pass through the array, we'll move the second-highest value into the second-highest index. This time we will stop at the second-to-last pair of values, since we already know that the highest value is at the highest index. The 9 is in blue, since it is already at its proper index.

7 9 9 9 9 9 9 9
6 7 7 7 7 7 7 7
5 4 4 4 4 4 6 6
4 5 5 5 5 6 4 4
3 6 6 6 6 5 5 5
2 3 3 3 3 3 3 3
1 1 2 2 2 2 2 2
0 2 1 1 1 1 1 1

On the second pass through, the second-highest value was already in the second-highest position in the array, but we did move some of the other values one step closer to being in the correct position. Notice that this time, we made only six comparisons (see that there are only seven columns after the red column). There's no need to make the seventh comparison, since the 9 is already in its proper place. 

We continue this process until we have moved every value to the correct index.

Bubble Sort Code

public void bubbleSort(int[] numbers)
{
     /*traverse the array (or a subset of the array) length-1 times*/
     for (int highest_unsorted=numbers.length-1; highest_unsorted != 0; highest_unsorted--)
     {    
        /*makes the necessary comparisons for one pass through the array*/      
        for (int best_so_far=0; best_so_far < highest_unsorted; best_so_far++)
        {
                        /*compare adjacent items and swap them if necessary*/
                        if (numbers[best_so_far] > numbers[best_so_far+1])
                                swapNumbers (best_so_far, best_so_far+1);
        }
     }
}


public void swapNumbers(int i, int j)
{
        int temp = numbers[i];  /*put numbers[i] somewhere to keep it safe*/
        numbers[i] = numbers[j];        /*put numbers[j] at index i*/
        numbers[j] = temp;               /*put numbers[i] at index j*/
}

The Big-O of Bubble Sorts

The bubble sort is a sorting algorithm which is very easy to implement, but how good is this algorithm? In order to answer this question, we need to look at how much work the algorithm has to do. When we analyze the running time of an algorithm, we want to look at the bigger picture, so we will make some approximations and ignore constant factors in order to simplify the analysis.

Let's say we have 10 numbers. The outer for loop of the bubble sort has to run 9 times (once when last_one = 9, once when it = 8, etc., up until it = 1, and when it hits 0 it stops). If we had 100 numbers, the outer loop would run 99 times. If we had 1000 numbers, the outer loop would run 999 times. In general, if there are N numbers, the outer loop will run (N-1) times, but to simplify our math we will say that it runs N times.

If the outer loop runs N times, then the total running time is N times the amount of work done in one iteration of the loop. So how much work is done in one iteration? Well, one iteration of the outer for loop includes one complete running of the inner for loop. The first time through, the inner loop goes 9 times, the second time through it goes 8 times, then 7, and so on. On average, the inner loop goes N/2 times, so the total time is N for the outer loop times N/2 for the inner loop times the amount of work done in one iteration of the inner loop.

For one iteration of the inner loop, we either do nothing if the number in the less than the one after it, or we set three values if we need to swap. In the worst case, we will need to swap at every step, so we will say that the one iteration of the inner loop requires 3 operations. That makes the total time for the bubble sort algorithm N*(N/2)*3 operations, or (3/2)*N2. So in the end it is Big-O O(n2)

Selection Sort

Now let's look at another easy sort, called the selection sort. With Bubble sort, we remembered the "smallest value so far" by swapping it into current position of the list during each step of each iteration. Although this works, it take a lot of effort to swap numbers around -- especially if all we really need to know is the index of the "smallest value so far". This is the critical difference in the two sorts -- Selection Sort just remembers the position of the "best so far", instead of swapping it into the current position. This reduces the amount of work required during each step of the sort. Then, at the end of each pass, it swaps just once.

So, let's take a look at this step-by-step. In the selection sort, we find the smallest value in the array and move it to the first index, then we find the next-smallest value and move it to the second index, and so on. We start at the first index and walk through the entire list, keeping track of where we saw the lowest value. If the lowest value is not currently at the first index, we swap it with the lowest value.

Now, let's take a look at an example. We will start with the same set of numbers we used before.

0 1 2 3 4 5 6 7
9 2 1 3 6 5 4 7

So the first step in the selection sort is to get the lowest value into the lowest index (index 0 in this case). We will loop through all of the values, keeping track of which index contains the lowest value. Since we are trying to fill index 0, we will assume that the lowest value is already there unless we find one that islower - we initialize our best index to be index 0.

As we scan through, we find that the lowest value is the 1 which is at index 2, so we swap the value at index 0 with the value at index 2.

0 1 2 3 4 5 6 7
1 2 9 3 6 5 4 7

Now the lowest value is at index 0, so the next step is to get the next-lowest value into index 1. When we scan through the numbers, we find that the next-lowest value, the 2, is already at index 1, so we do not need to swap.

If we continue through the array one index at a time, we will eventually move every value into the appropriate index, resulting in a sorted array.

Selection Sort Code

public void selectionSort(int[] numbers)
{
    /*For every index in the array, with the exception of the last one,*/
    for(int searcherIndex=0; searcherIndex < numbers.length-1; searcherIndex++)
    {
        /*Assume that the number is where it's supposed to be*/
        int correctIndex = searcherIndex;
        
        /*Try out other candidate indexes*/
        for (int candidateIndex=searcherIndex+1; candidateIndex < numbers.length; candidateIndex++)
        {
                /*If you find a smaller number, make it's index the correct index*/
                if(numbers[candidateIndex] < numbers[correctIndex])
                                correctIndex = candidateIndex;
        }
        /*At this point correctIndex really will be the correct index*/
        swapNumbers (searcherIndex, correctIndex);
    }
}

The Big-O of Selection Sort

Selection sort slightly better than the bubble sort, because we swap at most once for every index, instead of potentially once for each item. We find the correct index for that particular number. Then we swap it into its correct place in the array.

But the Big-O is exactly the same: O(n2). Each pass through the list, we fix the position of only one more item, so we still need to make n passes. And, each pass, we must compare ourselves to, n/2 other items (amortized cost, since the list shrinks from n to 1). As before, this leaves us with 1/2*n2, but we drop the coefficient and focus on the outer-bound behavior for Big-O.

Insertion Sort

Insertion sort works, as before, by viewing the list as two sub-lists, one of unsorted items, initially full, and one of unsorted items, initially empty. As before, each iteration, the next item from the unsorted list enters the sorted list.

Bascially, we remove the next item from the unsorted list, and that gives us an extra slot in the sorted list. We then move each item, one at a time, into that slot, until we make a "hole" in the right place to insert the new item. When we do, we place it there.

Much like Bubble sort and Selection sort, Insertion sort is O(n2). We move one item to the sorted list each time, so we need to make n passes. And then, each pass, we need to compare it to n/2 items (amortized over all runs). So, it requires 1/2*n*n operations for a Big-O of O(n2).

It is however, quite a bit more expensive in real time. Displacing an item in an array is quite expensive -- everything after it needs to be copied.

But, what is special about insertion sort is that it can be started before all of the data arrives. New data can still be sorted, even if it is lower (or higher) than those already fixed in place.

Also, the cost of displacing an item isn't present, if a Linked List is being sorted.

Here's an example:

4 7 9 2 5 8 1 3 6
4 7 9 2 5 8 1 3 6
4 7 9 2 5 8 1 3 6
4 7 9 2 5 8 1 3 6
4 7 9 2 5 8 1 3 6
2 4 7 9 5 8 1 3 6
2 4 7 9 5 8 1 3 6
2 4 5 7 9 8 1 3 6
2 4 5 7 9 8 1 3 6
2 4 5 7 8 9 1 3 6
2 4 5 7 8 9 1 3 6
1 2 4 5 7 8 9 3 6
1 2 4 5 7 8 9 3 6
1 2 3 4 5 7 8 9 6
1 2 3 4 5 7 8 9 6
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9

Insertion Sort Code

 public void insertionSort(int[] numbers)
  {
    /*going through all of the items in the array*/
    for (int insertMe=1; insertMe < numbers.length; insertMe++)
    {
      /*find the correct index for the item*/
      for (int newPosn=0; newPosn < insertMe; newPosn++)
      {
        /*stop when you come to an item greater than the item in question*/
        if (numbers[insertMe] < numbers[newPosn])
        {
          /*put the item in question somewhere for safe keeping*/
          int temp = numbers[insertMe];
          
          /*move everything after the correct index down to make room*/
          for (int shift=insertMe; shift > newPosn; )
            numbers[shift] = numbers[--shift];

          /*put the item in its correct index*/
          numbers[newPosn] = temp;

          /*You've found the right index for the item and it's time to stop*/
          break;
        }
      }
    }
  }