Site Search:

Greedy and relaxation, positive and negative

In order to find the extreme aspect (shortest path, longest path, max flow, min cut) we need greedy algorithm. The algorithm have to be greedy at each step in order to achieve the global extreme. For example, Dijkstra's algorithm always adds the edge with the shortest distance to the original point into the SPT at each step; Prime's algorithm always adds the edge with the shortest distance to the current MST to the MST at each step; Kruskal's algorithm always add the shortest edge in the graph into the current MST.

Greedy algorithm has a bad name, but it is actually not greedy, it always do the easiest thing and leave plenty of room. At each step, the algorithm expands the options, but defer the decision at the later stage. For example, we start at a small village, want to reach a big city across the mountain. The road through the mountain is very dangerous, tigers, poisonous mists...might cost your life. Dijkstra's algorithm will register this option, then look at other alternatives, 2 miles away, there is a small village, across the big river there is a small town. So let's do the easiest thing, walk to the village and see what will happen. Once we get to the village, we got more options, there is another village 5 miles away, a bridge leading to the small town we know earlier. So We again do the easiest thing -- we across the bridge and go to the small town and see what will happen. We go there, tons of options are opened. We can buy a bus ticket to the big city we want to go earlier, we can bug a fair and go to the big city in a boat, we can walk to an airport 5 miles away and see what will happen, we can walk to many neighboring cities and see what options we got. Who knows, a villager rumored a travel agency is hiring people somewhere. Any path to the big city don't have to cost your life by passing through the mountain. Be flexible to alternatives won't make you a tiger slayer sort of hero, but it will bring you to the goal alive. That's the key of greedy algorithm, it opens to options and takes the easiest route for the next step without goal in mind, until it eventually reached the goal by accident. If the goal of reaching the big city is everything, Dijkstra's algorithm wasted lots of steps exploring, but once reaching the big city, you have seen everything in your journey in the easiest possible way. This is how Dijkstra's algorithm builds the SPT.

As we can see, take the lowest energy path is a natural intendancy. It is favored by physics and statistics. Given a group of dots, by chance the 2 dots with lowest energy barrier will first connect, then the 2 dots with next lowest energy barrier. Eventually all dots are connected in the order of their energy barrier. That is the way Kruskal's algorithm builds the MST. Imagine our universe has many civilizations besides earth, how they will eventually connect to each other. The closest civilization will have the best chance to connect. We can imagine 2 scenarios. These civilizations are seeded at the same time and developed similarly. They will cross the communication barrier following Kruskal's algorithm. In scenarios 2, one civilization developed much more earlier, it reaches out to rest of the civilizations. It first reaches the closest neighbor, brings that civilization up-speed, then the combined civilizations start to reach their closet neighbor, combine, reach next one...This is how Prime's algorithm build the MST.

These greedy algorithm has success because they are relaxed and flexible, but they also suffer from it when negative edge is present. When going from A to B always cost something, greedy algorithm do a good job saving cost. How about getting from A to B gain something? Greedy algorithm will get trapped, they will never find the sweet point that saves enough. It is like hit rock and water. In case of water, you are never strong enough.