Graph Algorithm 101 for Busy Engineers (Part 1)

This post is for the experienced software engineers who want to refresh their memory on graph algorithms quickly. I will not go into the details of any algorithms, rather than I will list down the most basic algorithms and will briefly tell you how they work. If you are working in the industry for a few years, chances are you have already forgotten most of the algorithms. However, I hope, this post will help you to bring back some of the memories. And certainly, you can google yourself later and learn the details.

(If you are a Bengali reader, you can check out the graph algorithm book which i wrote in 2016. Also, I have lots of articles about graph algorithm in my Bengali blog)

I assume the readers of this post are competent engineers, and they roughly know what a graph is. So, I will start with graph representation and will describe some algorithms very briefly. When I mention their complexity, I will use the variables $V$ and $E$, where $V$ is the number of nodes in the graph and $E$ is the number of edges in the graph.

Graph Representation

There are two significant ways to represent a graph; the first one is an Adjacency matrix, and the second one is an Adjacency list. Adjacency matrix is a $V \times V$ matrix where $matrix[u][v]$ represents if there is an edge between $u$ and $v$.

adjacency matrix

For a weighted graph, the matrix contains the weight of the edges. It takes $O(V^2)$ memory. While the matrix makes it easy to check the cost or existence of the edge between $u$ and $v$, the operation of finding all the edges connected to a node becomes expensive as you need to traverse the whole row. On the contrary, the Adjacency list can solve this problem. For the above graph, the Adjacency list will look like this:

Adjacency list

In C++, you can use a vector or, in Java, you can use an ArrayList to make this kind of list. It takes only $O(E)$ memory. But the downside is, you need to traverse the list to check if there is an edge between $u$ to $v$.

Now, the question is which representation is to use? Well, it depends on the type of problem you are solving and the type of operations you need to perform.

Breadth-first-search (BFS)

What does it do?

BFS is the most straightforward algorithm to find the shortest path in an unweighted graph. The problem we are trying to solve here is, “Given an undirected graph and a source node, find the shortest path from the source to every other node of the graph.”. It is a “Single source shortest path” (SSSP) algorithms.

How does it work?

Have a look at the graph below. Assume $1$ is the source node. bfs graph

BFS will traverse the graph level-by-level. Here, the level 1 nodes have distance $1$ from the source, then, the level $2$ has distance $2$ and so on. It uses a queue to achieve this. At first, put the source into the queue. Now while the queue is not empty, pop out the first element of the queue and push every connected node in the queue (don’t push the same node twice!!). Doing this, it will ensure you that you will push lower level nodes in the queue before higher level nodes. As a result, every time you push a node, you can calculate the level by adding $1$ to the level of the parent node.

And, the complexity of this algorithm is $O(V + E)$.

Dijkstra

What does it do?

Dijkstra is quite similar to BFS. It also finds the shortest path, but in a weighted graph. Like BFS, it is also an SSSP algorithm.

How does it work?

As I said, it’s similar to BFS. But instead of a queue, it will use a priority queue. This priority queue will contain node numbers and distance of the nodes from the source. The members of the queue will be sorted by the distance.

For example, let’s assume $d[u]$ is the distance from the source to $u$. Every time you go from node $u$ to $v$, not only you push v to the queue, but also you need to push $d[v]$ to the queue. That means every time you pop a node from the queue, you will get the closest node from the source.

For this algorithm, the complexity is $O(V \times logV + E)$

Floyd-Warshall

What does it do?

Floyd Warshall is an all pair shortest path (APSP) algorithm. It can find the shortest path from all node to all other nodes in a directed path.

How does it work?

Even though the Floyd-Warshall algorithm is internally pretty complex, however, the code of this algorithm is surprisingly simple. First of all, you need to represent your graph using adjacency matrix, other representations won’t work. If two nodes don’t have a connection, you need to put infinite in those cells. Now you need to run $3$ nested loops. The first loop will select a node $k$ as the intermediate node. Then, the other two nested loop will select two nodes $u$ and $v$. Now you will check if it’s a good idea to visit $k$ while you are going from u to v. More specifically, you need to check if $distance[u][v] > distance[u][k] + distance[k][v]$. If the condition is true, you need to update $distance[u][v]$ and move on. In the end, the matrix will contain the shortest path between every node.

Here, the complexity of Floyd Warshall is $O(V^3)$.


Bellman-Ford

What does it do?

Bellman form is another SSSP algorithm. But the speciality of this algorithm is, apart from finding the shortest path from the source, it can also detect the existence of a negative cycle in a graph. (A negative cycle is a sequence of edges which start and ends at the same node and the sum of the weight of the edges of the sequence is negative.)

How does it work?

Let’s assume $d[u]$ is the distance from the source to $u$. Initially, $d[source]$ is $0$, and for all other nodes, $d[u]$ is infinite. Then, imagine you are going from $u$ to $v$ and weight is cost(u, v). Now you can update the value of $d[v]$ if $d[u] + cost[u][v] < d[v]$. That means you found a path which is shorter. This is called “edge relaxation”. Now perform this relaxation for every node and keep updating the values in $d$. Then, you will get the shortest path from source to all nodes which uses at most 1 edge. Repeat these operations $V-1$ times and after that, you will have the correct shortest path. Remember, the shortest path can have at most $V-1$ edges.

Now, the concern is, how to detect a negative cycle? For this, first of all, you need to relax all the nodes one more time. Then, if the negative cycle is absent (no negative cycle at all), the value of any index of $d$ won’t update, because, there is no shortest path that has $V$ edges. And, if there is a negative cycle, at least one index will update. In the case, where a negative cycle is reachable from the source, the shortest path is not defined anymore as you can make the path as short as you want by going around the cycle many times.

The complexity of the algorithm is $O(V*E)$. If there is no specific reason, such as, the possibility of a negative cycle, don’t use this algorithm.

Topological Sorting

What does it do?

Imagine you have a list of tasks, each task is dependent on some other tasks. You can represent it with a graph like below:

topsortNow you need to find a correct order to complete the tasks. There can be more than one possible solutions. Topological sort or topsort can help you to do that.

How does it work?

Let us assume that there is no cycle in the graph, otherwise, there will be no valid order. Because a cycle will create a circular dependency, hence no valid ordering will be possible. At first, you need to find all the tasks which don’t depend on any other tasks. To do this, we count the in-degree of each node (the number of nodes has an edge towards the current node). The tasks with indegree $0$ can be performed first. Next, push those tasks in the list of answers and remove the outward nodes from those nodes. Now we will get some new tasks with in-degree zero, and we can perform them. Keep repeating this until you complete all the tasks.

Complexity of this algorithm is $O(V^2)$. In order to find all the possible orders of the tasks, you need to do backtracking.

That’s all for part 1, and I will talk about minimum spanning three, depth-first-search and some other algorithms in the next part.

Promote empathy and be a better colleague

Recently I have read lot’s of articles on the corporate working environment. They advice on what do to climb the ladders, how to become the star employee, the ugly truth of human resource policies, etc. The thing I found dispiriting is almost advised to not to treat the co-workers as friends. They always suggest to keep a professional relationship with colleagues and hide anything personal, including any own problems. The logic they pose is simple and seems convincing at first. Your co-workers are your competitors for the next promotion and pay-rise. If you grow a personal relationship with them and share too much information, they can abuse the knowledge and back-stab you. No matter how close someone is, wait until the next promotion, and you will see they are not a friend.

I find this as a dangerous idea, and it promotes a trust-less environment. It asks you to forget the fact that your co-worker is another human just like you. Remember how dehumanizing it is to call an employee a resource? Promoting non-caring heartless relationship similarly dehumanize people.

Let’s face it, most of us spend at least 40 hours with colleagues every week which is 24% of total hours in a week and a whopping 35% if the time you are awake (assuming you sleep 8 hours a day). Is this healthy to spend so much time in an environment where everyone is worried about back-stabbing? Can you guarantee this habit won’t trickle down in your personal life too?

Now, why should you care about your co-workers? The simple answer is because they are humans and the only way to make the world a better place is caring and empathy. Your work life will be enjoyable when you have great people around you who care for each other and when you know they will defend you when you mess up. Show empathy to the co-worker who is having a bad week due to some personal issue and help them share the workload, most of the time they will return the favor when you have a bad day. Don’t be afraid to share your weakness, your fear with them, be open and help each other to grow as a better person.

I can assure you will be less stressed when you learn to treat the co-workers as friends, as human and stop worrying about back-stabbing. Are their downsides? Of course, there is, there will always be some person who is so-called “professional” and will take advantage of you. Probably you will get sabotaged a few times. But you will still hold up the moral high ground, and in your heart, you will know you did the right thing. So you want to get stressed and have a cold relationship with the people you spend so much time with? Or you want to be open and friendly but take the risk of potential back-stabbing? The choice is yours.

I had made some fantastic friends in my short professional career so far, and they helped a lot in difficult times. I can’t say how much grateful I am to them. Maybe some of them will someday something to hurt my career, but I would rather get hurt by trusting them rather than work in a stressful environment where everyone is ‘just a co-worker’.

It’s time we stop wearing the mask and promote empathy.

Problem Solving: Consecutive Letters (MIST Inter University Contest 2019)

This problem appeared in 2019 inter-university contest hosted by MIST university, Dhaka, Bangladesh. Initially, our assumption was it will be medium difficulty problem but seems like the contestants found it easier. Around 53 team out of 120 was able to solve it. Let’s see the problem statement:

The time limit for this problem is $3$ seconds.

If you find the 1st query hard to understand, let me try to put it another way, you have to find the size of the largest substring which includes index $i$ and all the characters in the substring are same as $S[i]$.

Let’s see the first sample input:

I give you the query “1 0”. Now you have to find the size of the largest substring which includes index $i = 0$ and contains only the character S[0] = ‘A’. In this case the substring is S[0:1] = AA” and has size $2$ .

Now the next query is “2 1”, so you need to replace $S[1]$ with ‘#’ character, so the string becomes:

After then we again query “1 0”. Now the substring is S[0:0] = A” and has size 1.

Naive Solution

The first solution comes to mind is brute force. For every query of type 1, just loop from index $i$ towards left and right and see how many $S[i]$ you can find.

The time complexity for this code is O(Q*N) where N is the size of the string. As Q and N are very big, they will fail. As a problem setter, I had to be careful to make sure this code doesn’t pass. So I made some test cases where the loops for searching left and right will be very long. Those test cases have very few queries of type 2 and almost all the characters in the string are same.

Expected solution

There are several ways to solve the problem. I have solved it using a set and binary search. There is also a nice solution using a disjoint set. I will describe the solution using binary search first.

In brute force solution, you looped towards the left to find the starting point of the segment. Now I will show you how to find the $left$ point faster. Let’s forget about type 2 query for a moment.

For any given string, we can divide it by segments containing the same characters. See the following string for example:

I have highlighted the starting point of each segment.

After you take the original string as input, you can easily find these points and save them, right? Let’s say we have saved them in a sorted array which looks like $[0, 3, 6, 7]$.

Now let’s say I give you a query “1 4”.

Can you see that the $left$ point is the largest number in the set which is smaller than $i = 4$? In this case, it’s $3$. And how to find the $right$ point? It’s just the next number if the array, in this case, it’s $6$. So the size of the segment will be $right – left  = 6 – 3 = 2$

As you have a sorted array already, you don’t need to search one by one, use the power of binary search. You can either write your own binary search or use lower_bound function available in your language and modify it.

Now how about query 2? Let’s say we put a “#” in $i = 1$. Now you can imagine this is creating a new segment from index $i = 1$.

Notice that not only it’s creating a segment from index $1$, it’s creating another segment from index $1 + 1 = 2$. So we insert $1$ and $2$ in our array $[0, 1, 2, 3, 6, 7]$.

Note that we never do type 1 query on a segment consisting “#”. There is no harm if you consider two consecutive “#” as two different segments. We can assume for type $1$ query, index $i$ will always create a new segment, index $i + 1$ may or may not create a new segment but we don’t need to handle it specially if you use a set.

If we use normal array to maintain the list of segment starts it will be very difficult to insert new segments as we need to keep them sorted. That’s why we will use a set. All the programming languages have some built-in set where you can insert a number in sorted way in O(LogN) complexity and also ensures uniqueness of the number.

Let’s see the code. To make our life easier, we will imagine that there is a segment starting at the very end of the string. That will save us pain of handling some special cases.

The complexity for both query of type 1 and type 2 is logN which makes the total time complexity O(Q*LogN) which is good enough to pass under 1 seconds if you use fast I/O.

Alternate solution using disjoint set

There is a nice offline solution using disjoint sets. It “offline” because you have to read all the queries first before answering them. Let’s see another example XXBBFFF, but now we will see the string as a graph:

All the characters converterd into nodes and there is edge between them if the characters are same. Now let’s say we have 3 queries:

Now we read all the type 2 queries first and we will erase the links with those index. In this case they are index $3$ and $5$.

Now treat the connected nodes as a connected components. You can use disjoint sets to maintain the member and the size of the components. Now we have these sets $\{0, 1\}, \{2\}, \{4\}, \{5\}, \{7\}$.

Now we answer the queries from last to first! The last query is “1 5”. What is the size of the component which contains the node 6? The answer is 1.

The next query is “2 6”, now we have to unblock the 5-th node and add back the edges.

Using union find algorithm, we will update the sets, now we have $\{0, 1\}, \{2\}, \{4\}, \{5, 6, 7\}$. Now we query “1 5” again, this time the answer is 3.

In this way we will climb from the end, keep unblocking the nodes and use union find to update the sets. So the type 1 query will be just finding the size of the set. In the end we have to print the queries in original order.

Union find operations have logN complexity, so in the end we have O(QlogN) time complexity. You can see a sample code here.

There is a third solution using segment tree. Code for that is significantly more complex compared to the solution described above. Also it takes much more extra memory and runs slower. During the contest we have let those solutions pass too. I am not going to describe that solution here.

That’s it for now, good wishes for your next contest.

Tech Retro: Pull Request Ethics

Pull requests are part and parcel of a soft engineer’s life unless you work in a horrible place where people just merge the codes to master directly. How a team handles the pull request and code reviews tells a lot about the team’s culture.

Last week we were discussing pull request ethics in our bi-weekly tech retros in Traveloka.com. Some of the points we discussed are, they should have meaning commit message and description and criticism about code should never offend anyone. If your team is relatively new, you can think about setting some ethical standard about pull requests to avoid conflicts and maintain healthy relationships. Here are my two cents about pull request ethics:

Continue Reading