## CS302 Lecture Notes - Topological Sort

• November 19, 2013.
• Latest Revision: November, 2018
• James S. Plank
• Directory: /home/plank/cs302/Notes/Topological

Topological sorts work on directed, acyclic graphs, and they are very simple. It is a sorting of the vertices of a graph, such that if there is an edge from a to b, then a comes before b in the sorting. Since the graph is acyclic, a topological sort is guaranteed to exist, although it is not guaranteed to be unique.

For example, consider the following graph (from the Topcoder problem ConvertibleStrings, which we use as an example of Dynamic progrmaming). The numbers in green show a valid topological sort of the graph:

As I said before, the sortings do not have to be unique. For example, you could swap the 3rd and 4th nodes in the sort, and you would still have a valid sort.

To perform a topological sort, you maintain a list of nodes with no incoming edges. Then, until that list is empty, you do the following:

• Remove a node from the list and append it to the topological sorting.
• For each edge coming out of the node, remove the edge from the graph. If that edge was to a node that now has no more incoming edges, put that node on the list.
That's how I got the ordering in the graph above.

This is guaranteed to work, because the graph is acyclic. The running time is O(|V|+|E|). Like DFS and BFS, it visits each node once, and each edge once.

There are some problems that you can solve with topological sort:

• Scheduling. Often the graphs represent precedence. For example, maybe the nodes are CS courses, and the edges are prerequisite relationships. A topological sorting is a valid way to schedule the classes without violating the prerequisites.

• Shortest path from a source node. You start with all nodes having distance ∞ from the source, except the source, which has a distance 0. Then you do a topological sort. When you remove an edge from a to b, update the minimum distance to b if it is improved by going through a. When you process a node, you have processed all nodes before it in the graph, so you know the shortest path to that node.

Is that better than Dijkstra's algorithm? Yes and no. See the analysis below.

• Number of distinct paths from the source to each node. There is one path to the source. Set the number of paths to each other node to zero. Then do a topological sort. When you remove an edge from a to b, add the number of paths to a to the number of paths to b. This one is in the topcoder problem IncreasingSubsequences, for which I have lecture notes.

### Shortest paths with Dijkstra or Topological Sort?

If our graph is directed and acyclic, then we can calculate shortest paths using either Dijkstra's algorithm or with topological sort. If all we cared about was worst-case running time, we'd use topological sort, because O(|E|+|V|) is a better than O(|E|log|V|). However, we are not always dealing with worst-case running times. Think about it:

• Topological sort has to process every node and edge between the source and the destination.
• Dijkstra's algorithm has to process every path that is less than or equal to the shortest path, but it doesn't process any path that is greater.
Let's explore this a little.

What I've done is write two programs: topo.cpp and dijkstra.cpp (I don't let you see dijkstra.cpp -- for myself, see my personal 302 page for how to get dijkstra.cpp back..).

These take the following command line arguments:

 ```topo|dijkstra n maxcap mincap window seed print(y|n) ```

The programs create random directed, acyclic graphs with n nodes, numbered 0 through n-1. The edges all have random capacities uniformly distributed between mincap and maxcap.

The structure of the graph depends on window:

• If window is positive, then each node i has edges to nodes i+1 through i+window.
• If window is negative, then we partition the nodes into n/window layers, and between layers, the nodes are fully connected. I'll have pictures later in the lecture.
Let's focus firt on the graphs where window is positive. For example, take a look at a small graph:

 ```UNIX> topo 4 50 1 2 1 y Node 0: [1,35][2,44] Node 1: [2,26][3,6] Node 2: [3,48] Node 3: Total edges in graph: 5 Shortest Path: 41 Edges Processed: 5 Graph Creation Time: 0.000 Shortest Path Time: 0.000 UNIX> dijkstra 4 50 1 2 1 n Total edges in graph: 5 Shortest Path: 41 Edges Processed: 4 Graph Creation Time: 0.000 Shortest Path Time: 0.000 UNIX> ```

It's pretty easy to see that the shortest path is 0 -> 1 -> 3. And you can see the difference between topological sort and Dijkstra -- topological sort has to process every edge. Dijkstra on the other hand, does not visit node 2, because the shortest path to node 3 is shorter than the one to node 2. For that reason, the edge from 2 to 3 is not processed.

Let's look at a larger example to see a class of graphs where Dijkstra's algorithm will outperform topological sort: Those where window equals n. Here's an example where n equals 8:

```UNIX> topo 8 10 1 8 8 y
Node 0: [1,7][2,7][3,1][4,10][5,3][6,9][7,4]
Node 1: [2,7][3,5][4,3][5,3][6,2][7,6]
Node 2: [3,5][4,7][5,1][6,2][7,4]
Node 3: [4,3][5,6][6,1][7,2]
Node 4: [5,4][6,6][7,5]
Node 5: [6,8][7,3]
Node 6: [7,1]
Node 7:
Total edges in graph:          28
Shortest Path:                  3
Edges Processed:               28
Graph Creation Time:        0.000
Shortest Path Time:         0.000
UNIX> dijkstra 8 10 1 8 8 n
Total edges in graph:          28
Shortest Path:                  3
Edges Processed:               14
Graph Creation Time:        0.000
Shortest Path Time:         0.000
UNIX>
```
To help visualize this, I'm drawing the graph below, where the edges are colored according to their weights:

What you can see here is that while topological sort has to process all 28 edges, Dijkstra's algorithm only processes the edges from nodes 0, 3 and 6. Let's extrapolate and time. In each of these tests, maxcap is 1000, mincap is one, window is equal to n.

I'm not super-proud of that graph, BTW -- the Dijkstra numbers are averages of 50 runs each, but there's still so enough randomness in the graphs that you see wavy lines. However, what you are seeing is that Dijkstra's algorithm process so many fewer edges than topological sort, that it is over ten times faster on the larger graphs.

Now, let's instead construct graphs that favor topological sort. Let's make n big, but limit window to 64. Here's an example:

```UNIX> topo 10000 1000 1 64 1 n
Total edges in graph:      637920
Shortest Path:               2102
Edges Processed:           637920
Graph Creation Time:        0.048
Shortest Path Time:         0.010
UNIX> dijkstra 10000 1000 1 64 1 n
Total edges in graph:      637920
Shortest Path:               2102
Edges Processed:           635321
Graph Creation Time:        0.053
Shortest Path Time:         0.015
UNIX>
```
Now, you can see that Dijkstra's algorithm is processing nearly all of the edges on the graph. Since it has to do map operations, which are O(log m) (where m is the size of the map), it is slower than topological sort, which is doing O(1) operations for each edge.

Let's look at how the timings scale with n when we keep window fixed at 64:

It's no longer a 10-fold improvement, but the topological sort clearly outperforms Dijkstra.

These are nice examples of showing how the structure of the graph impacts the performance of the two algorithms.

If we give a negative window size to topo and dijkstra, then the graph is composed of w fully-connected layers. For example, if I specify n to be 52 and w to be -5, then the graph looks like this:

I'd like to use this graph to demonstrate topological sort outperforming Dijkstra, but I'm having a hard time, so I'll just leave this text here. When the number of layers is 500, and mincap/maxcap are both large values, topological sort definitely outperforms Dijkstra:

Again, this is because both Dijkstra and topological sort process similar numbers of edges. Interestingly, when I shrink the number of layers, Dijkstra's performance relative to topological sort improves to the point where it's better. I may have to look at that in more detail some day.