CS302 Lecture Notes - Breadth First Search

• James S. Plank
• November 2, 2009. Latest Modification: Tue Oct 28 09:01:14 EDT 2014
• Directory: /home/plank/cs302/Notes/BFS

Topcoder Practice Problems

Breadth First Search (BFS) is complementary to Depth First Search (DFS). DFS works by visiting a node and then recursively visiting children. You can view it as relying on a stack -- push a node onto a stack, then go through the following algorithm:

• Pop a node off the stack.
• Do some processing on the node.
• Push all of the node's children onto the stack.
• Repeat until the stack is empty.
In fact, it will be useful to revisit DFS with this view. Let's use the following graph as an example:

 Ex-Graph.txt ```NNODES 12 EDGE 8 10 EDGE 4 11 EDGE 4 6 EDGE 3 8 EDGE 0 2 EDGE 2 9 EDGE 5 6 EDGE 4 8 EDGE 1 11 EDGE 6 7 EDGE 0 11 EDGE 3 7 EDGE 3 9 EDGE 8 9 EDGE 6 8 ```

A recursive visiting of all nodes using DFS starting with node zero will look as follows:

Were we to print out the nodes, they would be printed out in the order in which they are visited:

0, 2, 9, 8, 10, 3, 7, 6, 4, 11, 1, 5

Instead of recursion, let's use a stack. We'll push 0 onto the stack, then repeatedly pop a node off the stack, visit the node, then push the non-visited children onto the stack. (We push the children onto the stack in reverse order so that the order of visiting is the same as the recursive case).

 Node Visited 0 2 9 8 10 3 7 6 4 11 1 5 6 4 3 11 Action Start Push 11 and 2 Push 9 Push 3 and 8 Push 4, 6, 3 and 10 Do nothing Push 7 Push 6 Push 5 and 4 Push 11 Push 1 Do nothing Do nothing Already visited Already visited Already visited Already visited Stack 0 2, 11 9, 11 8, 3, 11 10, 3, 6, 4, 3, 11 3, 6, 4, 3, 11 7, 6, 4, 3, 11 6, 6, 4, 3, 11 4, 5, 6, 4, 3, 11 11, 5, 6, 4, 3, 11 1, 5, 6, 4, 3, 11 5, 6, 4, 3, 11 6, 4, 3, 11 4, 3, 11 3, 11 11 Print 0 2 9 8 10 3 7 6 4 11 1 5

As you see, the order of the nodes is the same as in the recursive case.

Now, breadth-first search works in the same manner, only we use a queue instead of a stack. See how this differs:

 Node Visited 0 2 11 9 1 4 8 3 8 6 10 3 6 7 5 7 Action Start Append 2 and 11 Append 9 Append 1 and 4 Append 8 and 3 Do nothing Append 8 and 6 Append 10, 3 and 6 Append 7 Already visited Append 5, and 7 Do nothing Already visited Already visited Do nothing Do nothing Already visited Queue 0 2, 11 11, 9 9, 1, 4 1, 4, 8, 3 4, 8, 3 8, 3, 8, 6 3, 8, 6, 10, 3, 6 8, 6, 10, 3, 6, 7 6, 10, 3, 6, 7 10, 3, 6, 7, 5, 7 3, 6, 7, 5, 7 6, 7, 5, 7 7, 5, 7 5, 7 7 Print 0 2 11 9 1 4 8 3 6 10 7 5

The order of the nodes is now 0, 2, 11, 9, 1, 4, 8, 3, 6, 10, 7, 5. The algorithm still visits all nodes and edges, but it does so in order of distance from the starting node. Think about it.

• The first two nodes visited are those that are one edge from node 0: nodes 2 and 11.
• Next are the nodes that are two edges away: nodes 9, 1 and 4.
• Next are the nodes that are three edges away: nodes 8, 3 and 6.
• Finally come the nodes that are four edges away: 10, 7 and 5.
Thus, breadth-first search is a convenient way to find the shortest path from a starting node to all the other nodes in the graph. To do so, you can store a back-edge in each node n -- this is the edge that first put n onto the queue. You can also maintain node n's distance to the starting node. The BFS algorithm becomes:
• For all nodes, set their backedges to NULL and their distances to -1.
• Set node 0's distance to zero and put it on the queue.
• Repeat the following:

• Remove a node n from the queue.
• For each edge e from n to n2 such that n2's distance is -1:

• Set n2's distance to n's distance plus one.
• Set n2's backedge to e.
• Append n2 to the queue.
When the algorithm terminates, each node contains its shortest distance to node zero, and the path to node zero can be obtained by traversing the backedges.

The PDF file BFS-Run.pdf contains an example from the graph above. It shows every step along the way. The final state is below:

Once the BFS finishes, we know the shortest distance of every node from node zero, and we can use the backedges to find the paths. For example, the shortest path from node 0 to node 7 is:

(0,2)(2,9)(9,3)(3,7)

Dijkstra's Algorithm

Dijkstra's algorithm is a simple modification to breadth first search. It is used to find the shortest path from a given node to all other nodes, where edges may have non-negative weights. The modification uses a multimap instead of the queue. The multimap uses the distance to from the starting node to the node as a key, and the node itself as a val. The algorithm is as follows:
• For all nodes, set their backedges to NULL, their distances to -1, and their "visited" field to be false.
• Set node 0's distance to zero and put it on the multimap.
• Repeat the following:

• Remove a node n from the front of the multimap and set its visited field to true.
• For each edge e from n to n2 such that n2 has not been visited. Let d be n's distance plus the weight of edge e. If n2's distance is -1, or if d is less than node n2's current distance:

• If n2 was in the multimap, remove it. [* We're going to revisit this below. *]
• Set n2's distance to d.
• Set n2's backedge to e.
• Insert n2 into the multimap, keyed on distance.
When the algorithm terminates, all the nodes will contain their shortest distance to node 0, and their backedges will define the shortest paths.

[* Revisiting Here *]: You actually have a choice of whether to remove a node from the multimap or not. It is often easier to code up Dijkstra's algorithm to leave nodes on the multimap rather than remove them. In that case, when a node reaches the front of the multimap for you to process, you need to check its distance versus its key in the multimap. If they differ, you simply ignore the node, because you have processed it already.

The tradeoff is memory and potentially performance vs coding complexity. When you code, it is much easier to leave the node on the multimap. However, if you end up replacing a lot of nodes on the multimap, it can make performance and memory consumption suffer. Ideally, it is better to remove the node before you re-insert it. To do that properly, you need to store an iterator to the node's place in the multiple, in the node's class definition. Think about that, especially if you decide to remove the node in your own implementation. BTW, I do advocate that you try removing the node in your lab. Not only is the code better, but it forces you to think about data structure design..

As an example, suppose we enrich the graph above with edge weights:

Then the PDF file Dijkstra-Run.pdf shows how Dijkstra's algorithm runs on the graph. In the PDF file, we are removing nodes when we replace them). When it finishes, here is the state of the system:

As with the BFS run above, you can use the backedges to find the shortest paths. For example, the shortest path from node 0 to node 3 has a distance of 20, and contains the edges:

(0,11)(11,4)(4,8)(8,9)(9,3)

Running Times

The running time of BFS (and therefore the unweighted shortest path problem) is O(|V| + |E|). As with DFS, it visits each node and edge once. The running time of Dijkstra's algorithm (and therefore the weighted shortest path problem) is a little more complex: O(|V| + |E|log(|V|)). This is because Dijkstra's algorithm visits each node and edge once, and at each edge, it potentially inserts a node into the multimap. Memorize those running times!