CS302 Lecture notes -- NP Completeness
- James S. Plank
- December 1, 2009.
- Latest revision: Dec 2 01:05:24 EST 2015
This is not a complete treatment of NP-Completeness. Like the Halting Problem
lecture notes, they introduce you to a concept that you will see later in
your CS careers and will provide you with fodder for endless conversations
around the family dinner table.
As always, you can spend quite a bit of time reading Wikipedia on the
subject. Their page is
in here.
This is not required reading, but (as of 2015) is a nice treatment of the topic.
P, NP, NP-Complete and NP-Hard are sets of problems, defined as follows:
- P: problems whose solution is polynomial time in the size of their inputs.
- NP: problems whose solutions can be verified in polynomial time.
(NP stands for non-deterministic polynomial time).
- NP-Complete: A collection of problems in NP whose solutions may or may not
polynomial time. We don't know. However, if we can prove that one of them may be solved
in polynomial time, then all of them can.
- NP-Hard: A collection of problems that are not in NP, whose solutions are at least as hard
as the NP-Complete problems.
In this lecture, we are going to see what it takes to prove that problems belong
to these sets.
Suppose you have a problem to solve, and you want to know its complexity class.
This takes two steps:
- Prove that it is in NP. Typically the problem is couched as
a yes or no problem involving a data structure, such
as ``does there exist a simple cycle through a
given directed graph that visits all the nodes?''
To prove it is in NP, you need to show that
a yes solution can be checked in polynomial time.
In the above example, you can check to see if a given path through the graph
is indeed a simple cycle in linear time. Therefore, the problem is in
NP. You don't have to prove anything about the no solutions,
and you don't have to prove anything about how you'd calculate a solution.
- Transform a known NP-Complete problem to this one in polynomial time.
Suppose the problem in question is Q,
and that L is a well-known NP-Complete problem like
the 3-satisfiability problem. You need to show that if you have
any instance of problem L, you can transform it into an instance
of problem Q in polynomial time. Thus, if you could solve problem
Q in polynomial time, you could solve problem L in polynomial
time.
If you can do both of these things, then you have proved that a problem is
NP-Complete. If you can prove that either of these things cannot be done, then you
have proved that a problem is not NP-Complete. Sometimes you can't do come up
with good proofs, and you just don't know.
The complexity classes P and NP-Hard may be put in terms of the above:
- P: If we can prove that the solution to a problem may be calculated
in polynomial time, then the problem is in P. All of the algorithms that we
have studied in this class, with the exception of enumeration, are in P.
- NP-Hard: These are problems that are not in NP; however, we can
perform the transformation in step 2 of a known NP-Complete problem to these
problems. Thus, they are at least as hard as the NP-Complete problems.
3-SAT - A Canonical NP-Complete Problem
3-SAT is a very simple NP-Complete problem. You are given a boolean expression,
which is a big AND (∧) of clauses:
E = C_{0} ∧
C_{1} ∧ ... ∧
C_{m-1}
Each clause C_{i} is the OR (∨) of three literals, where a literal is
either a variable x_{i} or the negation of a variable ¬ x_{i}
(or sometimes the negation of a is denoted a).
Here is an example with three clauses and three variables. To make it easier to read, I'm simply
calling the variables a, b and c .
E = ( a ∨ b ∨ c )
∧
( a ∨
b ∨
c )
∧
( a ∨
b ∨
c )
Given this definition, 3-SAT is simple -- is there an assignment of the variables so that E
is true? In the above example, it's easy to find such an assignment. For example, set a
and c to TRUE and b to FALSE (I'm coloring the true statements red -- you can
see that there is always at least one TRUE in each clause).
E = ( a ∨ b ∨ c )
∧
( a ∨
b ∨
c )
∧
( a ∨
b ∨
c )
In general, 3-SAT can be a very difficult problem to solve. Here's a harder example
with seven clauses and four variables.
E = ( a ∨ b ∨ c )
∧
( a ∨
b ∨
d )
∧
( a ∨
c ∨
d )
∧
( b ∨
c ∨
d )
∧
( a ∨
b ∨
c )
∧
( b ∨
c ∨
d )
∧
( b ∨
c ∨
d )
One correct assignment is setting a and c to FALSE,
and b and d to TRUE:
E = ( a ∨ b ∨ c )
∧
( a ∨
b ∨
d )
∧
( a ∨
c ∨
d )
∧
( b ∨
c ∨
d )
∧
( a ∨
b ∨
c )
∧
( b ∨
c ∨
d )
∧
( b ∨
c ∨
d )
From our lecture notes on enumeration, we can
answer whether an instance of 3-SAT is true or false by crafting a
solution that is exponential in the number of variables. Call that number n.
View each setting of the variables as
an n-bit number, where bit i represents the assignment of variable
x_{1}. There are 2^{n} of these numbers, and they represent
all possible settings of the variables. So, enumerate them and test to see if E is true.
However, is there a polynomial time solution? No one knows.
It is an easy matter to prove that 3-SAT is in NP. How many different clauses can there
be? (4/3) * n * (n-1) * (n-2) -- we'll go over that in class. That's a polynomial of
n. If we have a solution, we can test its validity by simply setting the variables and
seeing if E is true. That test is polynomial time, so 3-SAT is in NP.
As for proving that 3-SAT is NP-Complete, that is well beyond the scope of this class. However,
3-SAT is a very popular problem for proving that other problems are NP-Complete.
How would we do that?
Suppose I have a problem, like
The Independent Set Decision Problem (ISDP): Given a graph G
and a number k, can we find a set of k vertices in G such that there are
no edges between any two of the vertices.
Here's how we use 3-SAT to prove it is NP-Complete.
First, prove it's in NP: If you give me a set of k vertices, I can easily check to verify
that there are no edges between two nodes in k. That will be O(|E|) in the
worst case, which is most definitely polynomial in |V|.
Next, I need to figure out how to take an instance of 3-SAT, and convert it into an instance
of ISDP, so that if you can solve the ISDP instance in polynomial time, then you can solve the
instance of 3-SAT in polynomial time. Here's one way:
- Turn each clause into three nodes, and label the nodes with their literals (including
the not). Add an edge between each of these nodes.
- For every pair of nodes with the same, but negated, literals, add an edge between that
pair of nodes.
- Any independent set of size k=n will correspond to an assignment of the literals for
which the 3-SAT expression is true.
Here's the simple three-clause 3-SAT problem above, converted to a graph, with an example
3-node independent set colored magenta. You'll note that the set corresponds to a setting of
the variables that makes the 3-SAT equation true:
Below, I also convert the more complicated 7-node expression to a graph for the ISDP
problem.
I have the clauses clumped together going clockwise around the graph, starting at roughly
1:00. I also have colored inter-clause edges according to the literals that they connect:
I've colored the nodes in the Independent Set gray. You should be able to verify that:
- The set is indeed independent.
- The assignment of literals makes the expression true.
This works because you can only have one node per clause in the Independent Set. Moreover,
if you have a in the set, then you cannot have
a and vice versa.
Finally, think about the size of the graph. It will have 3m nodes, and a maximum
of something like 3m + 3(m/2)(m/2-1)/2 edges, which is clearly polynomial in m.
Thus, if I can solve ISDP in polynomial time, then I can solve 3-SAT in polynomial time.
Neat, no?
Who Cares?
NP-Complete problems usually have easy-to-write exponential solutions. However, we cannot prove
that they do not have polynomial time solutions. This is embodied in the equation:
P = NP?
It is a famous open question in theoretical computer science. Does its solution have practical
worth? Maybe -- a lot of these problems pop up very naturally (Spellseeker from Lab B
comes to mind...), and if we could solve them in polynomial time rather than
exponential, then that would be something!