Graph Algorithm 101 for Busy Engineers (Part 1)

This post is for the experienced software engineers who want to refresh their memory on graph algorithms quickly. I will not go into the details of any algorithms, rather than I will list down the most basic algorithms and will briefly tell you how they work. If you are working in the industry for a few years, chances are you have already forgotten most of the algorithms. However, I hope, this post will help you to bring back some of the memories. And certainly, you can google yourself later and learn the details.

(If you are a Bengali reader, you can check out the graph algorithm book which i wrote in 2016. Also, I have lots of articles about graph algorithm in my Bengali blog)

I assume the readers of this post are competent engineers, and they roughly know what a graph is. So, I will start with graph representation and will describe some algorithms very briefly. When I mention their complexity, I will use the variables $V$ and $E$, where $V$ is the number of nodes in the graph and $E$ is the number of edges in the graph.

Graph Representation

There are two significant ways to represent a graph; the first one is an Adjacency matrix, and the second one is an Adjacency list. Adjacency matrix is a $V \times V$ matrix where $matrix[u][v]$ represents if there is an edge between $u$ and $v$.

adjacency matrix

For a weighted graph, the matrix contains the weight of the edges. It takes $O(V^2)$ memory. While the matrix makes it easy to check the cost or existence of the edge between $u$ and $v$, the operation of finding all the edges connected to a node becomes expensive as you need to traverse the whole row. On the contrary, the Adjacency list can solve this problem. For the above graph, the Adjacency list will look like this:

Adjacency list

In C++, you can use a vector or, in Java, you can use an ArrayList to make this kind of list. It takes only $O(E)$ memory. But the downside is, you need to traverse the list to check if there is an edge between $u$ to $v$.

Now, the question is which representation is to use? Well, it depends on the type of problem you are solving and the type of operations you need to perform.

Breadth-first-search (BFS)

What does it do?

BFS is the most straightforward algorithm to find the shortest path in an unweighted graph. The problem we are trying to solve here is, “Given an undirected graph and a source node, find the shortest path from the source to every other node of the graph.”. It is a “Single source shortest path” (SSSP) algorithms.

How does it work?

Have a look at the graph below. Assume $1$ is the source node. bfs graph

BFS will traverse the graph level-by-level. Here, the level 1 nodes have distance $1$ from the source, then, the level $2$ has distance $2$ and so on. It uses a queue to achieve this. At first, put the source into the queue. Now while the queue is not empty, pop out the first element of the queue and push every connected node in the queue (don’t push the same node twice!!). Doing this, it will ensure you that you will push lower level nodes in the queue before higher level nodes. As a result, every time you push a node, you can calculate the level by adding $1$ to the level of the parent node.

And, the complexity of this algorithm is $O(V + E)$.

Dijkstra

What does it do?

Dijkstra is quite similar to BFS. It also finds the shortest path, but in a weighted graph. Like BFS, it is also an SSSP algorithm.

How does it work?

As I said, it’s similar to BFS. But instead of a queue, it will use a priority queue. This priority queue will contain node numbers and distance of the nodes from the source. The members of the queue will be sorted by the distance.

For example, let’s assume $d[u]$ is the distance from the source to $u$. Every time you go from node $u$ to $v$, not only you push v to the queue, but also you need to push $d[v]$ to the queue. That means every time you pop a node from the queue, you will get the closest node from the source.

For this algorithm, the complexity is $O(V \times logV + E)$

Floyd-Warshall

What does it do?

Floyd Warshall is an all pair shortest path (APSP) algorithm. It can find the shortest path from all node to all other nodes in a directed path.

How does it work?

Even though the Floyd-Warshall algorithm is internally pretty complex, however, the code of this algorithm is surprisingly simple. First of all, you need to represent your graph using adjacency matrix, other representations won’t work. If two nodes don’t have a connection, you need to put infinite in those cells. Now you need to run $3$ nested loops. The first loop will select a node $k$ as the intermediate node. Then, the other two nested loop will select two nodes $u$ and $v$. Now you will check if it’s a good idea to visit $k$ while you are going from u to v. More specifically, you need to check if $distance[u][v] > distance[u][k] + distance[k][v]$. If the condition is true, you need to update $distance[u][v]$ and move on. In the end, the matrix will contain the shortest path between every node.

Here, the complexity of Floyd Warshall is $O(V^3)$.


Bellman-Ford

What does it do?

Bellman form is another SSSP algorithm. But the speciality of this algorithm is, apart from finding the shortest path from the source, it can also detect the existence of a negative cycle in a graph. (A negative cycle is a sequence of edges which start and ends at the same node and the sum of the weight of the edges of the sequence is negative.)

How does it work?

Let’s assume $d[u]$ is the distance from the source to $u$. Initially, $d[source]$ is $0$, and for all other nodes, $d[u]$ is infinite. Then, imagine you are going from $u$ to $v$ and weight is cost(u, v). Now you can update the value of $d[v]$ if $d[u] + cost[u][v] < d[v]$. That means you found a path which is shorter. This is called “edge relaxation”. Now perform this relaxation for every node and keep updating the values in $d$. Then, you will get the shortest path from source to all nodes which uses at most 1 edge. Repeat these operations $V-1$ times and after that, you will have the correct shortest path. Remember, the shortest path can have at most $V-1$ edges.

Now, the concern is, how to detect a negative cycle? For this, first of all, you need to relax all the nodes one more time. Then, if the negative cycle is absent (no negative cycle at all), the value of any index of $d$ won’t update, because, there is no shortest path that has $V$ edges. And, if there is a negative cycle, at least one index will update. In the case, where a negative cycle is reachable from the source, the shortest path is not defined anymore as you can make the path as short as you want by going around the cycle many times.

The complexity of the algorithm is $O(V*E)$. If there is no specific reason, such as, the possibility of a negative cycle, don’t use this algorithm.

Topological Sorting

What does it do?

Imagine you have a list of tasks, each task is dependent on some other tasks. You can represent it with a graph like below:

topsortNow you need to find a correct order to complete the tasks. There can be more than one possible solutions. Topological sort or topsort can help you to do that.

How does it work?

Let us assume that there is no cycle in the graph, otherwise, there will be no valid order. Because a cycle will create a circular dependency, hence no valid ordering will be possible. At first, you need to find all the tasks which don’t depend on any other tasks. To do this, we count the in-degree of each node (the number of nodes has an edge towards the current node). The tasks with indegree $0$ can be performed first. Next, push those tasks in the list of answers and remove the outward nodes from those nodes. Now we will get some new tasks with in-degree zero, and we can perform them. Keep repeating this until you complete all the tasks.

Complexity of this algorithm is $O(V^2)$. In order to find all the possible orders of the tasks, you need to do backtracking.

That’s all for part 1, and I will talk about minimum spanning three, depth-first-search and some other algorithms in the next part.

Shafaet Ashraf

Leave a Reply

Your email address will not be published. Required fields are marked *