int sum(int n) { int i, partial_sum; partial_sum = 0; /* Line 1 */ for (i = 1; i <= n; i++) { /* Line 2 */ partial_sum += i*i*i; /* Line 3 */ } return partial_sum; /* Line 4 */ }This returns the sum from i = 1 to n of i cubed. Suppose we count the number of statements that are executed in this procedure. You have one for line 1 and one for line 4. The for loop on line 2 can be more easily counted if you re-write it as:
i = 1; while (i <= n) { ... i++ }You should now be able to see that its statements are executed 2n+2 times:
In terms of big-O notation, this is O(n). Why? Informally it is because n is the largest degree polynomial. Formally, it is because if we choose c=4, then we see that cn > 3n+4.
Of course, it is also O(n*log(n)) and O(n*n), since n*n > n*log(n) > 3n+4. However, to best characterize it, we choose the smallest function: it is O(n).
Given the definitions of last class, you'll also see that it is Omega(n), and therefore Theta(n) too. Why Omega(n)? Because if we choose c=1, then we see that cn < 3n+4. Since 3n+4 = O(n), and 3n+4 = Omega(n), 3n+4 = Theta(n) too.
Now, in line three, should we count each execution of that statement as one or more? Certainly executing ``partial_sum += i*i*i'' should be more expensive than executing ``partial_sum = 0.'' In fact, the book counts each execution of line three as four statements -- two multiplications, an addition, and an assignment. This is reasonable. However, in terms of big-O, it doesn't matter. Why? Well, suppose you do count it as 4 statements. Then line three contributes 4n statements to the running time, and the total running time is now 1+(2n+2)+4n+1 = 6n+4.
In terms of big-O, Omega and Theta, this changes nothing: 6n+4 = O(n) (choose c to be 7); 6n+4 = Omega(n) (choose c to be 1), and therefore 6n+4 = Theta(n) too.
The bottom line is that in terms of big-O, you can count statements as one, and not worry about their relative execution time, as long as it is O(1).
for (i = 0; i < n; i++) { j = sum(n); }Here the body of the loop is 3n+4 since sum requires 3n+4 statements. I could add an additional statement for the assignment to j but I think that complicates matters, especially since it only adds a constant to the running time of the body. Since the loop iterates n times, the running time of the body is n * (3n + 4) = 3n^{2} + 4n. As before the for statement requires 2n+2 statements so the total running time is 3n^{2} + 4n + (2n + 2) = 3n^{2} + 6n + 2, which is O(n^{2}).
We also could have analyzed the above loop slightly differently. We could include the increment of i and the comparison of i < n as part of each loop iteration. In this case the running time of the loop body would be 3n+6 and since the loop iterates n times, the running time of the body becomes n * (3n + 6) = 3n^{2} + 6n. The extra 2 statements come from the initial assignment of 1 to i and the final comparison of i with n that causes the loop to exit.
Now look at:
for (i = sum(0); sum(i) < sum(n); i++) { j = sum(n); }Now, the loop body is 3n+4 and the test is 3i+4+3n+4. Since i is always less than or equal to n, we can say the loop test is definitely better than 6n+8. The increment is one, and it executes n times, so the total running time of the loop body is n*(3n+4+6n+8+1) = 9n^{2}+13n. We also have to count the initial execution of i = sum(0) and the final execution of sum(i) < sum(n). sum(0) executes 3(0) + 4 = 7 statements. The final execution of sum(i) < sum(n) occurs when i = n, so this comparison requires 2 * (3n + 4) = 6n + 8 statements. Summing it all up, we get a total running time of:
running time = 9n^{2} + 13n + (7) + (6n + 8) = 9n^{2} + 19n + 15Again, this is O(n^{2}).
Look at:
for (i = sum(0); sum(i) < sum(n); i++) { for (k = 0; k < n; k++) { j = sum(n); } } }Earlier we analyzed the inner loop and determined that it executes 3n^{2}+6n+2 statements. The test in the outer loop requires 6n+8 statements and iterates n times, so the running time of the outer loop is n(3n^{2}+6n+2+6n+8+1) = 3n^{3}+12n^{2}+9n. Did we forget something? Yes! We forgot to include the execution time for the initial assignment in the outer loop, which is i = sum(0), and the final comparison, when i = n. We also did this analysis earlier and determined that the initial assignment requires 7 statements and the final comparison requires 6n+8 statements. Our final running time is therefore:
running time = 3n^{3}+12n^{2}+9n + 7 + (6n + 8) = 3n^{3}+12n^{2}+15n + 15which is O(n^{3}). Often you get sloppy counting statements, because more often than not, if you skip the initial assignment, the final comparison, or the loop increment, you don't affect the running time in terms of the big-O notation. In this course I will expect you to count everything however.
if (i == 3) { j = 4; } else { j = sum(n); }is 1 plus the maximum of the running time of ``j = 4'' and ``j = sum(n)''. Taking the running time of sum(n) to be 3n+4, we get the running time of ``j = sum(n)'' to be 3n+4, and thus the running time of the if-then-else statement to be 3n+5. Of course, that is O(n), so we can just say it is O(n).
As you get better at this stuff, you learn to just quantify everything as big-O, and you use the following identities. (You might think about how you could prove these).
Suppose line 2 of the sum program was:
for (i = 0; i <= n; i++)Does that change the running time of the program? No, because the for loop is still O(n). Similarly, if line 2 is
for (i = 0; i <= n/2; i++)then it is still O(n).
If we change line 2 to be:
for (i = 1; i <= n; i *= 2)then the loop iterates log(n) times. Now the running time of the program is O(1)+O(log(n))+O(1) = O(log(n)).
Since 4n+1 and 1000n+400000 are both O(n) does this mean that you shouldn't care if your program's running time is one or the other? No. If you can get it to be 4n+1 rather than 1000n+400000, you should. However, a more fundamental thing is to make sure that your program's running time is O(n) instead of O(n*log(n)) or O(n*log(n)) instead of O(n*n) if that is possible.