\documentstyle[12pt]{article}

\include{PCMInotes}

\begin{document}

\lecturetitle{Advanced}{11}{Measuring the Complexity of
Proofs}{July}{31}

\section{How Do We Prove a Tautology?}

Consider formulas with Boolean variables, the connectives NOT, AND and
OR (usually written $\neg$, $\land$ and $\lor$), but no quantifiers.
For example, 
\[ 
   F = x \land y \implies x \lor y
\]
is such a formula where the connective $A \implies B$ is simply seen
as an abbreviation for $\neg A \lor B$.  (We are assuming the
following precedence rule: $\neg$, then $\land$ and $\lor$, then
$\implies$.)  How do you convince someone that this formula is always
true, i.e., that it evaluates to true no matter what Boolean values
are assigned to the variables?  Formulas with this property are
called {\em tautologies}.

One way would be to present that person with a {\em truth table}:
\[
  \begin{array}{cc|ccc}
    x & y & x \land y & x \lor y & F \\\hline
    0 & 0 & 0 & 0 & 1 \\
    0 & 1 & 0 & 1 & 1 \\
    1 & 0 & 0 & 1 & 1 \\
    1 & 1 & 1 & 1 & 1 
  \end{array}
\]

Now what about the formula 
\[ 
   F_n = x_1 \land x_2 \land \cdots \land x_n \implies x_1 \lor x_2
         \lor \cdots \lor x_n?
\]  
We could again present a truth table for this formula, but that table
would have size at least $n2^n$, which is exponential in the size of
$F_n$.  So just to look at the entire table requires time exponential
in the size of the formula.

Can we do better?  For example, can we convince someone that $F_n$ is
a tautology by presenting a {proof} that has size polynomial in the
size of $F_n$?  The answer is yes and here is how.  First, let us go
back to $F$.  Consider the following sequence of steps:
\begin{formalproof}
\line $x \implies x$
\line $x \land y \implies x$
\line $x \land y \implies x \lor y$
\end{formalproof}
It is clear that the first line is a tautology and that this implies
that the second line is a tautology too.  The third line follows from
the second one since it is clear that if $x \land y \implies x$ is a
tautology, then $x \land y \implies x \lor y$ is one too.  What we
have here is a more sophisticated type of proof, at least when
compared to truth tables.  We are relying on accepted {\em axioms}
like $x \implies x$ and on {\em inference rules} such as ``if $x
\implies x$, then $x \land y \implies x$''.

Note that the above proof is convincing even if $x$ and $y$ are
replaced by arbitrary formulas.  This is because the axiom $A \implies
A$ and the inference rules ``if $A \implies B$, then $A \land C
\implies B$'' and ``if $A \implies B$, then $A \implies B \lor C$''
all make sense even if $A$ and $B$ are replaced by arbitrary formulas.
Therefore, what we have is a convincing proof of $A \land B \implies A
\lor B$, where $A$ and $B$ are arbitrary formulas.

Now let $A = x_1$ and $B = x_2 \land \cdots \land x_n$.  Then the
above gives us a proof of $F_n$.  And this proof has size $O(n)$,
which is much better than the $n2^n$ we got with truth tables.

Let us look a slightly more serious example.  Let 
\[
  G = A \lor (B \land C) \implies (A \lor B) \land (A \lor C).
\]
This states that $\lor$ distributes over $\land$.  A proof of $G$
could go as follows:
\begin{formalproof}
\line $A \implies A$
\line $A \implies A \lor B$
\line $B \implies B$
\line $B \land C \implies A \lor B$
\line $A \lor (B \land C) \implies A \lor B$, from lines 2 and 4
\line $A \implies A$
\line $A \implies A \lor C$
\line $C \implies C$
\line $B \land C \implies A \lor C$
\line $A \lor (B \land C) \implies A \lor C$, from lines 7 and 9
\line $A \lor (B \land C) \implies (A \lor B) \land (A \lor C)$, from
      lines 5 and 10
\end{formalproof}

Now consider the generalized form of this ``Distributivity Law'': 
\[
  G_n = A \lor (B_1 \land \cdots \land B_n) \implies (A \lor B_1)
  \land \cdots \land (A \lor B_n).
\]
The following is a proof of $G_n$:
\begin{formalproof}
\line $A \lor (B_1 \land \cdots \land B_n) \implies (A \lor B_1) \land
(A \lor (B_2 \land \cdots \land B_n))$, by using the above proof with
$B = B_1$ and $C = B_2 \land \cdots \land B_n$

\line $A \lor (B_1 \land \cdots \land B_n) \implies (A \lor B_1)$,
from line 1

\line $A \lor (B_1 \land \cdots \land B_n) \implies A \lor (B_2 \land
\cdots \land B_n)$, from line 1

\line $A \lor (B_2 \land \cdots \land B_n) \implies (A \lor B_2) \land
\cdots \land (A \lor B_n)$ (see below for an explanation of this one)

\line $A \lor (B_1 \land \cdots \land B_n) \implies (A \lor B_2) \land
\cdots \land (A \lor B_n)$, by combining lines 3 and 4

\line $G_n$, from lines 2 and 5
\end{formalproof}
A proof for line 4 can be constructed recursively, by repeating the
process that proved $G_n$ given line 4.  At each step in this
recursion, line 4 gets simpler until it becomes $A \lor (B_{n-1} \land
B_n) \implies (A \lor B_{n-1}) \land (A \lor B_n)$, which we can prove
directly.  Or, in other words, the existence of a proof of the formula
on line 4 can be established by induction on the number of $B_i$'s.
Either way, we get a convincing proof of $G_n$ and its size is
$O(n^2)$, which once again is much better than the size of the
corresponding truth table.


\section{The Sequent Calculus}

What we have established in the previous two examples are the basic
elements of a proof system called the {\em Sequent Calculus}.  Lines
in a Sequent Calculus proof are called {\em sequents} and are formulas
of the form $(A_1 \land \cdots \land A_m) \implies (B_1 \lor \cdots
\lor B_n)$.  Since they always have this general form, sequents are
written as $A_1, \ldots, A_m \implies B_1, \ldots, B_n$.  By
convention, the {\em empty sequent} ``$\implies$'' is considered
false, ``$\implies F$'' is considered equivalent to $F$, and ``$F
\implies$'' is considered equivalent to $\neg F$.  We will allow
unbounded fan-in connectives in our formulas and these will be written
as follows: $\land(A_1, \ldots, A_n)$ and $\lor(A_1, \ldots, A_n)$.

In the Sequent Calculus, the {\em axioms}, or {\em initial sequents},
are ``$A \implies A$'', ``$\implies \land()$'' and ``$\lor() \implies$''.
The {\em inference rules} are divided into three groups: the {\em
structural} rules, the {\em logical} rules, and the {\em cut} rule.
The structural rules are:
\begin{enumerate}
\item[] Weakening: if $\Gamma_1, \Gamma_2 \implies \Delta$, then
$\Gamma_1, A, \Gamma_2 \implies \Delta$.  Similarly on the right.
\item[] Permutation: if $\Gamma_1, A, B, \Gamma_2 \implies \Delta$, then
$\Gamma_1, B, A, \Gamma_2 \implies \Delta$.   Similarly on the right.
\item[] Contraction: if $\Gamma_1, A, A, \Gamma_2 \implies \Delta$, then
$\Gamma_1, A, \Gamma_2 \implies \Delta$.  Similarly on the right.
\end{enumerate}
The logical rules are:
\begin{enumerate}
\item[] NOT-left: if $\Gamma \implies A, \Delta$, then $\neg A, \Gamma
\implies \Delta$.
\item[] NOT-right: if $A, \Gamma \implies \Delta$, then $\Gamma \implies
\neg A, \Delta$.
\item[] AND-left: if $A_1, \land(A_2, \ldots, A_n), \Gamma \implies
\Delta$, then $\land(A_1, \ldots, A_n), \Gamma \implies \Delta$.
\item[] AND-right: if $\Gamma \implies A_1, \Delta$ and $\Gamma \implies
\land(A_2, \ldots, A_n), \Delta$, then $\Gamma \implies \land(A_1,
\ldots, A_n), \Delta$.
\item[] OR-left: if $A_1, \Gamma \implies \Delta$ and $\lor(A_2, \ldots,
A_n), \Gamma \implies \Delta$, then $\lor(A_1, \ldots, A_n), \Gamma
\implies \Delta$.
\item[] OR-right: if $\Gamma \implies A_1, \lor(A_2, \ldots, A_n),
\Delta$, then $\Gamma \implies \lor(A_1, \ldots, A_n), \Delta$.
\end{enumerate}
Finally, the cut rule is:
\begin{itemize}
\item[] Cut rule: if $A, \Gamma \implies \Delta$ and $\Gamma \implies
A, \Delta$, then $\Gamma \implies \Delta$.
\end{itemize}
In each of these rules and axioms, $A$ and the $A_i$'s can be replaced
by arbitrary formulas (in a consistent way).  In the rules, the
$\Gamma$'s and $\Delta$'s stand for arbitrary sequences of formulas.

A Sequent Calculus {\em proof} is a finite sequence of lines, each
consisting of either an initial sequent or a sequent that can be
derived from previous ones by one of the inference rules.  A proof of
formula $F$ can be either
\begin{enumerate}

\item a proof whose last line is ``$\implies F$'',

\item a proof whose last line is the sequent $A_1, \ldots, A_m
\implies B_1, \ldots, B_n$, if $F$ is of the form $A_1 \land \cdots
\land A_m \implies B_1 \lor \cdots \lor B_n$, or

\item a proof that uses ``$F \implies$'' as an additional axiom and
whose last line is the empty sequent ``$\implies$''.

\end{enumerate}
This last form of proof is called a {\em refutation} of $\neg F$.  The
{\em size} of a proof is the total number of symbols in it.

It should be clear that Sequent Calculus proofs are convincing, in the
sense that they can only prove tautologies.  This property is called
{\em soundness} and follows from the fact that each of the axioms and
inference rules are themselves sound.

Now does every tautology have a Sequent Calculus proof?  The answer is
yes and can be illustrated by the proof we presented earlier for the
formula $G = A \lor (B \land C) \implies (A \lor B) \land (A \lor C)$.
If we read the proof backwards, starting with the bottom line and
following each application of the inference rules in the reverse
order, towards the axioms, we realize that each inference rule is
simplifying the current sequent by eliminating a connective.  For
example, $A \lor (B \land C) \implies A \lor B$ is broken down into $A
\implies A \lor B$ and $B \land C \implies A \lor B$.  Now, except for
weakening, each inference rule has a property we can call {\em reverse
soundness}: if its conclusion is a tautology, then each of its
hypotheses is a tautology.  We can use this to recursively break down
the formula we are trying to prove until we end up with sequents that
must be axioms.  (The details are left as an exercise.)  Therefore, we
say that the Sequent Calculus is {\em complete}.

From our first example, we get that the formula $F_n = A_1 \land A_2
\land \cdots \land A_n \implies A_1 \lor A_2 \lor \cdots \lor A_n$ has
a truth table of size at least $n2^n$ but a Sequent Calculus proof of
size $O(n)$.  More precisely, what we have here is a sequence of
formulas $F_1, F_2, F_3, \ldots$ and a sequence of proofs $P_1, P_2,
P_3, \ldots$ such that $P_i$ is a proof of $F_i$.  Equivalently, we
can view these sequences as a single (parametrized) formula $F_n$ and
a single (parametrized) proof $P_n$.  We measure the complexity of
$P_n$ as a function of $n$.  For example, in the case of Sequent
Calculus proofs, the {\em size} of $P_n$ is the total number of
symbols in $P_n$, expressed as a function of $n$.  

The proof of completeness of the Sequent Calculus outlined above
actually shows that every formula has a Sequent Calculus proof of size
$O(n2^n)$.  And we have two examples of tautologies, $F_n$ and $G_n$,
that have Sequent Calculus proofs that are much smaller than that:
$O(n)$ and $O(n^2)$, respectively.  So we can say that as proof
systems, the Sequent Calculus is more powerful than truth tables since
the truth table of a formula with $n$ variables is always of size
$\Theta(n2^n)$.

But does every tautology have a Sequent Calculus proof of polynomial
size?  The answer is not known and we will see in the next section
that a positive answer would have very interesting consequences.

In the meantime, let us say that the Sequent Calculus is closely
related to {\em Frege} proof systems.  As in the Sequent Calculus, a
proof in these systems is a sequence of formulas, each of which of is
either an axiom or a formula that can be derived from previous ones by
an inference rule.  A particular Frege proof system is defined by a
finite, sound and implicationally complete set of axioms and inference
rules.  A proof system is {\em implicationally complete} if whenever
$A \implies B$ is a tautology, there is a proof of $B$ that uses $A$
as an additional axiom.  Note that neither $A$ nor $B$ need to be
tautologies themselves.  Since for any axiom $A$, $A \implies B$ is a
tautology whenever $B$ is, implicational completeness implies
completeness.

As in the Sequent Calculus, the axioms and rules of a Frege system are
{\em schemas}, meaning that they can be used with arbitrary formulas.
Any two Frege systems are equivalent to each other in the sense that a
formula has a polynomial-size proof in one system if and only if it
has a polynomial-size proof in the other.  The idea is that the
logical implication corresponding to a rule in one system can be
proved in the other by using implicational completeness.  (The details
are left as an exercise.)  The Sequent Calculus is also equivalent to
any Frege system.  In contrast, we know from the examples of the
previous section that the Sequent Calculus and truth tables are not
equivalent.


\section{A Formal Notion of Proof System}

Now that we have defined the Sequent Calculus, and explored its
relationship to truth tables and other Frege systems, let us step back
and ask what a proof system is after all.  In a proof system, given any
string $x$ and a formula $F$, it should be possible to determine
whether $x$ is a proof of $F$.  So the predicate ``$x$ is a proof of
$F$'' should be computable.  But we normally also want proofs to be
easy to verify.  So we will also insist that this predicate be
computable in time polynomial in the length of $\langle x, F \rangle$.
For example, whether a table is the correct truth table of a given
formula can be determined in time linear in the size of the table.
And determining whether a list of sequents constitutes a Sequent
Calculus proof of a given formula can be done in time polynomial in
the size of the list.

So our formal, abstract notion of a {\em proof system} will be this: a
polynomial-time computable predicate $R(x,F)$, where $x$ is any string
and $F$ is any formula.  The meaning of $R$ is that $x$ is a proof of
$F$, but it is $R$ that defines what a proof is.  The proof system is
{\em complete} if for every tautology $F$, there is $x$ such that
$R(x,F) = 1$ (and then we say that $x$ is a proof of $F$).  The proof
system is {\em sound} if $F$ is tautology whenever there is $x$ such
that $R(x,F)$.

In the previous section, we said that any two Frege systems are
equivalent in the sense that a formula has a polynomial-size proof in
one system if and only if it has a polynomial-size proof in the other.
Now we add the condition that proofs in one system can be efficiently
translated into proofs in the other.  We say that a proof system $\cal
S$ {\em polynomially simulates} (P-simulates) a proof system $\cal T$
if there is a polynomial-time computable function $f$ such that for
every pair $\langle x, F \rangle$, $x$ is a proof of $F$ in $\cal T$
if and only if $f(x)$ is a proof of $F$ in $\cal S$.  (Notice the
similarity to the notion of polynomial-time reducibility.)  We then
say that two systems are polynomially equivalent (P-equivalent) if
they P-simulate each other.  The Sequent Calculus P-simulates truth
tables, but truth tables do not P-simulate the Sequent Calculus.  All
Frege systems are P-equivalent to each other.

Now let us go back to a question we asked earlier: does every
tautology have a Sequent Calculus proof of polynomial size?  We can
define the {\em complexity} of a proof system as the following
function of $n$: the maximum over all tautologies $F$ of size $n$ of
the minimum length of a proof of $F$, 
\[
   \max_{F: |F| = n} \; \min_{P: P \mbox{\scriptsize\rm\ is a
   proof of } F} |P| .
\]
For example, the truth tables
proof system has $\Theta(n2^n)$ complexity.  Proof systems of
complexity $n^{O(1)}$ are called {\em polynomially bounded}.  Asking
whether there is a polynomial-size Sequent Calculus proof of every
tautology is the same as asking whether the Sequent Calculus is
polynomially bounded.

There is currently no proof system known to be polynomially bounded.
And if one was found, then this would imply that $\NP$ is closed under
complement.  One way to show that $\P \neq \NP$ is to show that $\NP$
is {\em not} closed under complement.  This is because we know that
$\P$ {\em is} closed under complement, so $\P$ and $\NP$ could not
possibly be the same class.  Therefore, if a single polynomially
bounded proof system was found, then this approach to showing that $\P
\neq \NP$ would be invalidated.

\begin{theorem}
  There is a polynomially bounded (sound and complete) proof system if
  and only if $\NP$ is closed under complement.
\end{theorem}

\begin{proof}
Suppose that $\NP$ is closed under complement.  Consider the language
TAUT consisting of all tautologies: TAUT = $\{ F : F$ is a
tautology$\}$.  The complement of TAUT is in $\NP$.  Therefore, by
hypothesis, TAUT is in $\NP$.  This implies that TAUT has a
polynomial-time verifier $V$.  Consider the predicate $R$ computed by
$V$.  If $F \notin$ TAUT, then $R(P,F) = 0$ for every $P$.  If $F \in$
TAUT, then there is $P$ of length at most $|F|^{O(1)}$ such that
$R(P,F)= 1$.  Therefore, $R$ is a polynomially bounded proof system.

Now suppose that there is a polynomially bounded proof system $R$.
Then the polynomial-time machine that decides $R$ constitutes a
polynomial-time verifier for TAUT.  Therefore, TAUT is in $\NP$.  If a
formula $F$ is satisfiable, then its negation $\neg F$ is not a
tautology, and vice-versa.  Therefore, SAT $\Preduces
\overline{\mbox{\rm TAUT}}$, which implies that $\overline{\mbox{\rm
TAUT}}$ is $\NP$-complete.  Now suppose that $A \in \NP$.  Then $A
\Preduces \overline{\mbox{\rm TAUT}}$, which implies that
$\overline{A} \Preduces$ TAUT.  Therefore, $\overline{A} \in \NP$
since TAUT $\in \NP$.  This shows that $\NP$ is closed under
complement.
\end{proof}

In the previous proof, we showed that TAUT $\in \NP$ if and only if
there is a polynomially bounded proof system.  This leads to a more
general idea.  Say that $R$ is a {\em proof system for a language $L$}
if $R$ is computable in polynomial time and if $w \in L$ if and only
if there is a string $P$ such that $R(P,w) = 1$.  Then $\NP$ is
exactly the class of languages that have polynomially bounded proof
systems.

Why study the complexity proof systems?  Here are three reasons.
First, to answer some very natural and fundamental questions such as
``What is a proof?'', ``What is an efficient proof system?'' and ``Do
efficient proofs systems exist?''.

Second, because of the connections to the $\P$ versus $\NP$ question
we just explained.  If we could show that there are no polynomially
bounded proof systems, then we would have shown that $\P \neq \NP$.
This gives us the following approach for proving that $\P \neq \NP$:
start with proof systems known not to be polynomially bounded (like
truth tables) and prove that increasingly more powerful proof systems
are not polynomially bounded.

Third, consider the problem of finding a proof for a given tautology.
Typically, the weaker the proof system, the more efficient the proof
search algorithm.  So relatively weak proof systems are studied in the
context of automated theorem proving and understanding their exact
power (i.e., the class of tautologies that have polynomial-size proofs
in these systems) is therefore important.


\section{Exercises}

\begin{enumerate}

\item Consider a Frege system with only one rule ``if $P$ and $P
\implies Q$, then $Q$'' (modus ponens) and with the following ten
axioms:
\begin{enumerate}
\item $(P \land Q) \implies P$
\item $(P \land Q) \implies Q$
\item $P \implies (P \lor Q)$
\item $Q \implies (P \lor Q)$
\item $\neg\neg P \implies P$
\item $(P \implies Q) \implies [(P \implies \neg Q) \implies \neg P]$
\item $P \implies (Q \implies P \land Q)$
\item $(P \implies R) \implies [(Q \implies R) \implies (P \lor Q
\implies R)]$
\item $P \implies (Q \implies P)$
\item $(P \implies Q) \implies [(P \implies [Q \implies R]) \implies
(P \implies R)]$
\end{enumerate}
Give Sequent Calculus proofs of each of these axioms.  Then give a
sequent calculus proof that simulates the modus ponens rule, in the
sense that it proves the sequent $Q$ using the sequents $\implies P$
and $\implies (P \implies Q)$ as additional axioms.

\item Provide the details of a proof that the Sequent Calculus is
complete.

\item Show that the Sequent Calculus is implicationally complete.

\item In the version of the Sequent Calculus we defined, there are
three possible ways to prove a formula $F$.   Show that they are
interchangeable by proving the following:

  \begin{enumerate} 

  \item For any tautology $F$, a direct proof of $F$ can be transformed
  in polynomial time into a refutation of $\neg F$, and vice-versa.

  \item If $F$ is of the form $A_1 \land \cdots \land A_m \implies B_1
  \lor \cdots \lor B_n$, then a proof whose last line is the sequent
  $\implies F$ can be transformed in polynomial time into a proof
  whose last line is the sequent $A_1, \ldots, A_m \implies B_1,
  \ldots, B_n$.

  \end{enumerate}

\item Show that any two Frege systems are P-equivalent.

\item Show that the Sequent Calculus is P-equivalent to any Frege
system.

\item Show that a proof system for TAUT is also a proof system for the
complement of SAT.

\item Show that a language is in $\NP$ if and only if it has a
polynomially bounded proof system.

\item Suppose that $\cal S$ P-simulates $\cal T$.  Show that the class
of tautologies $\cal T$ can prove in polynomial size is contained in
the class of tautologies $\cal S$ can prove in polynomial size.  Show
that if $\cal T$ is polynomially bounded, then so is $\cal S$.

% \item Once again, suppose that $\cal S$ P-simulates $\cal T$.
% Consider the problem of finding a proof for a given a formula $F$ (or
% returning FALSE if $F$ is not a tautology).  This is called the proof
% search problem.  Measure the running time of a proof search algorithm
% as a function of the size of the proof produced.  Show that if there
% is a polynomial-time proof search algorithm for $\cal T$, then there
% is one for $\cal S$.

\end{enumerate}

\end{document}
