Download Brute Force Design Strategies in Computer Architecture | CPSC 5155G and more Study notes Computer Architecture and Organization in PDF only on Docsity!
Chapter 3: Brute Force
The brute-force design strategy is often the simplest and occasionally the best approach to solving a problem. This strategy is not sufficiently discussed in many algorithm design courses, possibly due to the mistaken idea that it is not sufficiently “advanced”, whatever that might mean. We study the method for a number of reasons, including:
- It occasionally produces a good solution, and
- We use it as a “yardstick” against which to compare all more sophisticated methods. As the book notes, it is very difficult to find a solvable problem that cannot be solved by some sort of brute-force algorithm. Indeed, “unsolvable by brute-force” is almost a definition of the term “unsolvable by any algorithm”. As an example of a problem that is well solved by brute force, consider the “dot product” of two N-dimensional vectors, each represented as an array with N elements. Algorithm DotProduct ( A[0..N – 1], B[0..N – 1] ) Dot = 0 For J = 0 To (N – 1) Do Dot = Dot + A[J]*B[J] Return Dot Selection Sort and Bubble Sort We now study a few solutions to the problem of sorting a list by comparison only. As always, the search and sort problems can be used as sources of many interesting examples. The selection sort is the first algorithm that we shall study in this section. The strategy of selection sort is to process an unsorted sublist of the list of numbers by placing the smallest element at the beginning of the list. Here is a slightly modified version of the textbooks algorithm. It is more efficient in that it reduces array references. Algorithm SelectionSort( A[0.. N – 1] ) For I = 0 to (N – 2) Do Min = I A_Min = A[I] For J = (I + 1) To (N – 1) Do If A[J] < A_Min Then A_Min = A[J] Min = J End If End Loop over J Swap A[I] and A[Min] // For algorithms swap is a End Loop over I // basic operation. Page 1 of 10 CPSC 5115 Last Revised November 28, 2020
It is easily seen that the algorithm produces a sorted list, taking (^) 2 N = 2
N N 1
comparisons to complete the process. Thus, we say that the time complexity of this brute- force algorithm is (NN^2 ). Later, we shall study some “more advanced” sort algorithms that are known to have time complexity (NNlog N). While the latter algorithms are usually more efficient than selection sort, it is common knowledge that for small values of N, selection sort is faster. The commonly accepted estimate is that selection sort is better for sorting 10 or fewer items, and possibly retains its advantage up to about 25 items. We now discuss bubble sort briefly. As noted in the textbook, the algorithm is generally considered as inferior and would not be discussed were it not for the catchy title. Indeed, one textbook illustrates the algorithm by sorting the names of volcanoes, thus the volcanoes are seen to “bubble up”. The key strategy of this algorithm is the repeated application of the “bubble” operation. If A[J + 1] < A[J] Then Swap A[J + 1] and A[J] The book presents the standard version of the bubble-sort algorithm and mentions an obvious way to improve its performance. We shall present this slightly improved version, which can have some advantages should the array be almost sorted. Algorithm Bubble (A[0.. N – 1]) // Change the outer for loop to a Repeat Until loop I = 0 Repeat Sorted = True For J = 0 to (N – 2 – I) Do If A[J + 1] < A[J] Then Swap A[J+1] and A[J} Sorted = False End If End For I = I + 1 Until ( I > (N – 2) ) Or Sorted Here we make the observation that if the list has been divided into two parts A 0 …. AK-1 | AK … AN- in which the list AK … AN-1 has been sorted and an examination of the list A 0 …. AK does not show any elements out of order, then we conclude that the entire list is sorted.
Some Remarks on the Closest Pair Problem The book discusses the problem of examining a set of points to determine the two that are closest together. Let P = { (NX 1 , Y 1 ), (NX 2 , Y 2 ), … (NXN, YN) }, with N ≥ 2, be the set of points. It is usually assumed that any pair of points differ in at least one coordinate, but such a constraint is not required. On page 107, the book presents a brute-force algorithm based on the “hypotenuse rule” that the distance between two points is given by the square root of a sum of squares. Thus the obvious algorithm, with a few subtle faults, is shown below. Dmin // This gives me some grief For I = 1 to (N – 1) Do For J = (I + 1) To N Do D = Sqrt( (XI – XJ)^2 + (YI – YJ)^2 ) If ( D < Dmin) Then Dmin = D IX1 = I IX2 = J End If Next J // Visual Basic syntax Next I Return (IX1, IX2) // The indices of the closest pair There are a few comments to make on this. The first is that this algorithm shows the brute- force approach to comparing pairs of items in an array. This strategy will guarantee that every pair of points in the list is considered. For I = 1 to (N – 1) Do For J = (I + 1) To N Do // Do something here Next J // Visual Basic syntax Next I We should note that any algorithm based on this framework will be (NN^2 ), possibly (NN^2 ), so that the improvements we shall discuss, while very practical, do not change the time complexity of the algorithm. The first objection to the algorithm is the problem with taking the square root. We theorists say that the square root is just a basic operation that should be counted the same as addition or multiplication. The fact is that the square root operation is computationally intensive (Ntry to take a square root by hand) and can slow an algorithm considerably. The solution to this first objection rests of the fact that for positive numbers, both the square and square root functions are monotonically increasing functions ; thus for A ≥ 0 and B ≥ 0, we have A^2 ≥ B^2 if and only if A ≥ B. So we compare the squares of the distances.
DSmin // This gives me some grief For I = 1 to (N – 1) Do For J = (I + 1) To N Do DS = (XI – XJ)^2 + (YI – YJ)^2 If ( DS < DSmin) Then DSmin = DS IX1 = I IX2 = J End If Next J // Visual Basic syntax Next I Return (IX1, IX2) // The indices of the closest pair We now have a perfectly good algorithm for the problem. I want to divert from our purely theoretical approach and discuss some very real problems with the above when it is considered as a fragment of a real program. My main complaint concerns the first statement which assigns plus infinity to the proposed minimum for the square of the distance. I distrust any algorithm that involves assigning a value of either positive or negative infinity. The following code moves towards my preferred algorithm. If (N < 2) Then Return (1, 1) // // I always like to start with a real value when computing // either the minimum or maximum of anything. // DSmin (X2 – X1)^2 + (Y2 – Y1)^2 For I = 1 to (N – 1) Do For J = (I + 1) To N Do DS = (XI – XJ)^2 + (YI – YJ)^2 If ( DS < DSmin) Then DSmin = DS IX1 = I IX2 = J End If Next J // Visual Basic syntax Next I Return (IX1, IX2) // The indices of the closest pair The observant student will note that the above calculates (X2 – X1)^2 + (Y2 – Y1)^2 twice. For me personally, this is a small cost to avoid silly code. The student is invited to contemplate this issue, particularly considering what numeric value to assign to infinity. At a company where I previously worked, we determined that = 1.0 1038. All I can say is that we had no errors in either our code or results of our computation due to this assignment. Another consideration is whether to exit the loop if one computes DS = 0. Note that I excluded this possibility earlier, but one might want to consider it.
First suppose that XJ = XK. Then the value of SSQ is changed by 2(NXK – XJ) + 2 = 2. This shows that making the values more different increases the value of SSQ. For XJ = XK + 1, this becomes (NXJ)^2 + (NXK)^2 + 2(N– 1) + 2 = (NXJ)^2 + (NXK)^2. This is not a surprise as all we have done here is to swap the values. Suppose XJ = XK + Z, with Z > 1. Then SSQ changes by 2(N – Z) + 2 < – 2 + 2 = 0 and the value of SSQ is decreased by the operation. Although not directly connected to the problem, it is this result that gave me the clue as to how to generate an example to falsify the claim. Exhaustive Search Exhaustive search is a method for generating the answer to a problem by generating every possible solution and examining that solution in comparison with other possible solutions. We sometimes refer to these as either combinatorial problems, because they involve the generation of combinatorial objects, or exponential-time problems. We first note that the two notations are roughly equivalent. Remember that the factorial function is to be considered in the class of exponential functions, specifically that the function N! is (N2N). We show this by proving that N! > 2N^ for N > 3. The proof is by simple induction, with a significantly different choice of the base case. Base case: N = 4 We observe that 4! = 24 > 16 = 2^4. Induction: Assume N ≥ 4. Then (NN + 1)! = (NN + 1)N! > 2N!. By the inductive hypothesis N! > 2N, so 2N! > 2 2 N^ = 2N+1. We now mention two examples of problems best solved by exhaustive search as well as a variant of one of the problems that is known to have a very easy exact solution. Knapsack Problem The first problem is well-known and, as is the case with all interesting problems, can be reinterpreted to apply to a large number of very important applications. The knapsack problem is stated as follows: “Given N items of known weights (NW 1 , W 2 , …, WN) and values (NV 1 , V 2 , …., VN) and a knapsack of capacity W, find the most valuable subset of the items that fit into the knapsack. There is an implied condition that we should consider. Suppose that the items are ordered by non-decreasing weight, so that W 1 W 2 … WN. Then W 1 W < (^) N K
W K
1
. We have an easy solution if either all items fit or no items fit. There are two important variants of the knapsack problem. The fractional (continuous) knapsack problem that has an easy linear-time solution. The 0/1 (discrete) knapsack can be solved only by exhaustive search. We shall use the fractional knapsack to make statements about solving 0/1 knapsack.
Traveling Salesman Problem We now consider the optimization version of the Traveling Salesman Problem. The only known way to generate the lowest cost tour of N cities is to generate each of the (NN – 1)! possible tours and investigate the cost of each of them. Here is a four-city example of TSP, in which the only way to solve the problem is the brute-force method of generating all 3! = 6 possible tours and tracking the lowest cost. Note that for small values of N, that TSP for N cities is not a hard problem. For N = 20, this becomes a very tedious problem. In our solution, we apply brute-force generation, but do not generate the solutions blindly. We recall that we can start the tour at any city and arbitrarily choose to begin at city A. The first step in any exhaustive search must be the generation of a complete tour. A B C D A Cost = 2 + 8 + 1 + 12 = 23 Best = 23 A B D C A Cost = 2 + 3 + 1 + 5 = 11 Best = 11 Now consider tours beginning with A C, the partial cost of which is 5. A C B the partial cost of this is 5 + 8 = 13. We already have a tour with less cost, so we do not consider this any more. A C D the partial cost of this is 5 + 1 = 6. This looks good, so we continue the tour with the only possible choices. A C D B A. The total cost of this is 6 + 3 + 2 = 11. This matches the best. A D there are two tours beginning with this choice. But note that the partial cost at this point is 12, which is greater than the known best solution, so we discard both solutions that begin with the link A D. There are two optimal trips, each with total cost of 11. A B D C A, and A C D B A. We next present a problem that was given as a puzzle on the NPR show “Car Talk”. This problem is one that is most easily solved by searching the solution space; fortunately that search space can be proven to be quite small.
Observation 3 The number of dogs cannot be less than one or more than six. We have already specified that D > 0 or D 1. Suppose that D > 6. Then D 7 (Nas the count of dogs is an integer) and 15D 105 which immediately implies that the sum 15D + C + 0.25M 105, as each of D, C, and M is a positive integer. This is not consistent with the sum equation, so we conclude 1 D 6. This last observation shows that there are only six possibilities for the number of dogs in the solution to this set of equations. Thus, we solve the equations by guessing possible values for the number of dogs and solving the modified equations. Recall that the equations are 15 D + C + 0.25M = 100 D + C + M = 100 Let D = 1. Then the equations become C + 0.25M = 85 C + M = 99 or 0.75M = 14. But 14 is not a multiple of 3, so this solution is not allowed. Let D = 2. Then the equations become C + 0.25M = 70 C + M = 98 or 0.75M = 28, not a multiple of 3. Let D = 3. Then the equations become C + 0.25M = 55 C + M = 97 or 0.75M = 42 and M = 56. Thus we have one solution: D = 3, C = 41, and M = 56. Let D = 4. Then the equations become C + 0.25M = 40 C + M = 96 or 0.75M = 56, not a multiple of 3. Let D = 5. Then the equations become C + 0.25M = 25 C + M = 95 or 0.75M = 70, not a multiple of 3. Let D = 6. Then the equations become C + 0.25M = 10 C + M = 94 or 0.75M = 84 or Z = 112, too many mice. We have covered all six possible solutions to the set of equations and shown that only one solution satisfies all of the requirements. Thus, the solution exists and is unique: three dogs, forty-one cats, and fifty-six mice.
