# tcs math – some mathematics of theoretical computer science

## September 21, 2008

### Lecture 1: Cheating with foams

Filed under: CSE 599S, lecture, Math — Tags: , , — James Lee @ 1:35 pm

This the first lecture for CSE 599S:  Analytical and geometric methods in the theory of computation.  Today we’ll consider the gap amplification problem for 2-prover games, and see how it’s intimately related to some high-dimensional isoperimetric problems about foams.  In the next lecture, we’ll use spectral techniques to find approximately optimal foams (which will then let us cheat at repeated games).

### The PCP Theorem, 2-prover games, and parallel repetition

For a 3-CNF formula $\varphi$, let $\mathsf{sat}(\varphi)$ denote the maximum fraction of clauses in $\varphi$ which are simultaneously satisfiable. For instance, $\varphi$ is satisfiable if and only if $\mathsf{sat}(\varphi)=1$. One equivalent formulation of the PCP Theorem is that the following problem is NP-complete:

Formulation 1.

Given a 3-CNF formula $\varphi$, answer YES if $\mathsf{sat}(\varphi)=1$ and NO if $\mathsf{sat}(\varphi) \leq 0.9$ (any answer is acceptable if neither condition holds).

We can restate this result in the language of 2-prover games. A 2-prover game $\mathcal G$ consists of four finite sets $Q,Q',A,A'$, where $Q$ and $Q'$ are sets of questions, while $A$ and $A'$ are sets of answers to the questions in $Q$ and $Q'$, respectively. There is also a verifier $V : Q \times Q' \times A \times A' \to \{0,1\}$ which checks the validity of answers. For a pair of questions $(q,q')$ and answers $(a,a')$, the verifier is satisfied if and only if $V(q,q',a,a')=1$. The final component of $\mathcal G$ is a probability distribution $\mu$ on $Q \times Q'$.

Now a strategy for the game consists of two provers $P : Q \to A$ and $P' : Q' \to A'$ who map questions to answers. The score of the two provers $(P,P')$ is precisely

$\displaystyle\mathsf{val}_{P,P'}(\mathcal G) = \Pr \left[V(q, q', P(q), P(q'))=1\vphantom{\bigoplus}\right],$

where $(q,q')$ is drawn from $\mu$. This is just the probability that the verifier is happy with the answers provided by the two provers. The value of the game is now defined as

$\displaystyle\mathsf{val}(\mathcal G) = \max_{P,P'} \mathsf{val}_{P,P'}(\mathcal G),$

i.e. the best-possible score achievable by any two provers.

Now we can again restate the PCP Theorem as saying that the following problem is NP-complete:

Formulation 2.

Given a 2-prover game with $\mathcal G$ with $|A|,|A'|=O(1)$, answer YES if $\mathsf{val}(\mathcal G)=1$ and answer NO if $\mathsf{val}(\mathcal G) \leq 0.99$.

To see that Formulation 1 implies Formulation 2, consider, for any 3-CNF formula $\varphi$, the game $\mathcal G_{\varphi}$ defined as follows. $Q$ is the set of clauses in $\varphi$, $Q'$ is the set of variables in $\varphi$, while $A = \{ TTT, TTF, TFT, FTT, TFF, FTF, FFT, FFF \}$ and $A' = \{T, F\}$. Here $A$ represents the set of eight possible truth assignments to a three-variable clause, and $A'$ represents the set of possible truth assignments to a variable.

The distribution $\mu$ is defined as follows: Choose first a uniformly random clause $C \in Q$, and then uniformly at random one of the three variables $x \in Q'$ which appears in $C$. An answer $(a_C, a_x) \in A \times A'$ is valid if the assignment $a_C$ makes $C$ true, and if $a_x$ and $a_C$ are consistent in the sense that they give the same truth value to the variable $x$. The following statement is an easy exercise:

For every 3-CNF formula $\varphi$, we have $\mathsf{val}(\mathcal G_{\varphi}) = \frac23 + \frac13 \mathsf{sat}(\varphi)$.

The best strategy is to choose an assignment $\mathcal A$ to the variables in $\varphi$$P'$ plays according to $\mathcal A$, while $P$ plays according to $\mathcal A$ unless he is about to answer with an assignment that doesn’t satisfy the clause $C$.   At that point, he flips one of the literals to make his assignment satisfying (in this case, the chance of catching $P$ cheating is only $1/3$, the probability that $P'$ is sent the variable that $P$ flipped).

This completes our argument that Formulation 1 implies Formulation 2.

Parallel repetition
A very natural question is whether the constant $0.9$ in Formulation 2 can be replaced by $0.001$ (or an arbitrarily small constant). A natural way of gap amplification is by “parallel repetition” of a given game. Starting with a game $\mathcal G = \langle Q,Q',A,A',V,\mu \rangle$, we can consider the game $\mathcal G^{\otimes k} = \langle Q^k, Q'^k, A^k, A'^k, V^{\otimes k}, \mu^{\otimes k} \rangle$, where $\mu^{\otimes k}$ is just the product distribution on $(Q \times Q')^k \cong Q^k \times Q'^k$. Here, we choose $k$ pairs of questions $(q_1, q'_1), \ldots, (q_k, q'_k)$ i.i.d. from $\mu$ and the two provers then respond with answers $(a_1, \ldots, a_k) \in A^k$ and $(a'_1, \ldots, a'_k) \in A'^k$. The verifier $V^{\otimes k}$ is satisfied if and only if $V(q_i, q'_i, a_i, a'_i)=1$ for every $i = 1, 2, \ldots, k$.

Clearly $\mathsf{val}(\mathcal G^{\otimes k}) \geq \mathsf{val}(G)^k$ because given a strategy $(P,P')$ for $\mathcal G$, we can play the same strategy in every coordinate, and then our probability to win is just the probability that we simultaneously win $k$ independent games. But is there a more clever strategy that can do better? Famously, early papers in this arena assumed it was obvious that $\mathsf{val}(\mathcal G^{\otimes k}) = \mathsf{val}(\mathcal G)^k$.

In fact, there are easy examples of games $\mathcal G$ where $\mathsf{val}(\mathcal G^{\otimes 2}) = \mathsf{val}(\mathcal G) = \frac12$.  (Exercise: Show that this is true for the following game devised by Uri Feige. The verifier chooses two independent random bits $b, b' \in \{0,1\}$, and sends $b$ to $P$ and $b'$ to $P'$. The answers of the two provers are from the set $\{1,2\} \times \{0,1\}$. The verifier accepts if both provers answer $(1,b)$ or both provers answer $(2,b')$.)

Nevertheless, in a seminal work, Ran Raz proved the Parallel Repetition Theorem, which states that the value of the repeated game does, in fact, drop exponentially.

Theorem 1.1: For every 2-prover game $\mathcal G$, there exists a constant $c = O(\log(|A|+|A'|))$ such that if $\mathsf{val}(\mathcal G) \leq 1-\epsilon$, then for every $k \in \mathbb N$,

$\displaystyle\mathsf{val}(\mathcal G^{\otimes k}) \leq (1-\epsilon^{3})^{k/c}.$

The exponent 3 above is actually due to an improvement of Holenstein (Raz’s original paper can be mined for an exponent of 32).

### Special games and the unique games conjecture

There are two special times of games which should be emphasized, depending on the structure of the mapping $V : Q \times Q' \times A \times A' \to \{0,1\}$. A projection game is one in which, for every $(q,q') \in Q \times Q'$ and $a \in A$, there is at most one value $a'$ for which $V(q,q',a,a')=1$. In other words, after fixing an answer to the first question, there is at most a single answer to the second question which the verifier accepts. The 3-CNF game $\mathcal G_{\varphi}$ above is an example (given an assignment to the variables in the clause $C$, the second prover has to give the consistent assignment to the variable $x$). Recently, Anup Rao gave an improved parallel repetition theorem for this special case.

Theorem 1.2: For every 2-prover projection game $\mathcal G$, with $\mathsf{val}(\mathcal G) \leq 1-\epsilon$, and for every $k \in \mathbb N$,

$\displaystyle\mathsf{val}(\mathcal G^{\otimes k}) \leq (1-\epsilon^{2})^{k/100}.$

Notice that in addition to the improved exponent of $2$, there is no dependence on $|A|,|A'|$. (This dependence is known to be necessary for general games.)

Why do we care about the exponent?
Our main focus in this lecture will actually be on the exponent of $\epsilon$ in the preceding theorems, so it seems like a good time to discuss its importance. For that, we need to introduce unique games.  This is a game where $Q=Q'$, $A=A'$, and the verifier behaves as follows. Given a pair of questions $q,q' \in Q$, there exist a bijection $\pi_{q,q'} : A \to A$ such that $V(q,q',a,a')=1$ if and only if $\pi_{q,q'}(a)=a'$. In other words, after fixing a pair of questions, for every answer of one prover there exists a unique answer of the other prover that satisfies the verifier.  (This is almost the same as satisfying the projection property from both sides, except that here we have enforced that after fixing $(q,q')$ and $a$, there exists exactly one satisfying $a'$ instead
of at most one.)

Now we state Khot’s unique games conjecture (UGC), which has far-reaching consequences in hardness of approximation (and is the central open question in the field).

Conjecture (UGC): For every $\delta > 0$, there exists a $C=C(\delta)$ such that the following problem is NP-complete: Given a unique game $\mathcal G$ with $|A| \leq C$, answer YES if $\mathsf{val}(\mathcal G) \geq 1-\delta$ and answer NO if $\mathsf{val}(\mathcal G) \leq \delta$.

At present, there are very few good ideas on how to attack this conjecture, and there are differing beliefs about its truth. Observe that, unlike Formulations 1 and 2 above, in the YES instance, we now only require $\mathsf{val}(\mathcal G) \geq 1-\delta$.  Of course, this problem is harder than distinguishing $\mathsf{val}(\mathcal G)=1$ from $\mathsf{val}(\mathcal G) \leq \delta$. In fact, this is fundamental: It is easy to design an efficient algorithm which checks whether $\mathsf{val}(\mathcal G)=1$ for a unique game.  (Exercise: Verify this.)

We now outline one possible approach to proving the UGC, in which the exponent of $\epsilon$ becomes crucial. For an undirected graph $G=(V,E)$, define

$\displaystyle\textsf{max-cut}(G) = \max_{S \subseteq V} \frac{|E(S,\bar S)|}{|E|},$

where $E(S,\bar S)$ denotes the set of edges from $S$ to its complement. Consider the following conjecture.

Conjecture (MAX-CUT conjecture): There exists a constant $\beta$ such that for every $\delta > 0$, the following problem is NP-complete: Given a graph $G$, distinguish between the two cases

$\textsf{max-cut}(G) \geq 1-\delta$
$\textsf{max-cut}(G) \leq 1-\beta\sqrt{\delta}$

Note that this conjectured hardness ($1-\delta$ vs. $1 - \sqrt{\delta}$) is tight, as shown by the Goemans-Williamson algorithm.  See also a recent spectral algorithm of Luca Trevisan that achieves the same bound.  From work of Khot, Kindler, Mossel, and O’Donnell (and the subsequent Majority is Stablest theorem, which we’ll get to later in the course), we know that the UGC implies the MAX-CUT conjecture. Might we prove that the two conjectures are actually equivalent?

To address this, let’s first observe that we can turn the MAX-CUT problem into a 2-prover unique game in a straightforward way. Given a graph $G=(V,E)$, consider the game $\mathcal G_G$ where $Q=Q'=V$ and $A=A'=\{0,1\}$.

The distribution $\mu$ on questions is as follows. With probability $1/2$ each: (1) Choose a uniformly random vertex $v \in V$ and ask the questions $(v,v) \in Q \times Q'$, or (2) choose a uniformly random edge $(u,v) \in E$ and ask the questions $(u,v) \in Q \times Q'$. The verifier is satisfied in case (1) only if the two provers give the same answer for $v$. The verifier is satisfied in case (2) only if the two provers give different answers. It only takes a moment’s though to see that the two provers should fix a maximum cut in the graph, and play according to it, yielding $\mathsf{val}(\mathcal G_G) = \frac12 + \frac12 \textsf{max-cut}(G).$

Now we are ready to see that the two conjectures are equivalent if we can obtain a good-enough exponent in parallel repetition of unique games. To this end, let $\alpha^*$ be the infimal value of $\alpha$ such that there exists a constant $c$ so that for every unique game $\mathcal G$, we have

$\displaystyle\mathsf{val}(\mathcal G) \leq 1-\epsilon \implies \mathsf{val}(\mathcal G^{\otimes k}) \leq (1-\epsilon^{\alpha})^{k/c}$

for every $k \in \mathbb N$. From Theorem 1.2, we know that $\alpha^* \leq 2$. A little better would show the equivalence of these two conjectures.

Claim 1.3: If $\alpha^* < 2$, then the the MAX-CUT conjecture implies the UGC.

Proof: The proof is straightforward: If $\textsf{max-cut}(G) \geq 1-\delta$, then $\mathsf{val}(\mathcal G_G) \geq 1-\delta/2$, which implies that $\mathsf{val}(\mathcal G_G^{\otimes k}) \geq (1-\delta/2)^k \geq 1-k \delta/2$. On the other hand, if $\textsf{max-cut}(G) \leq 1-\beta\sqrt{\delta}$, then $\mathsf{val}(\mathcal G_G) \leq 1-\beta\sqrt{\delta}/2$, and for some $\alpha < 2$,

$\displaystyle\mathsf{val}(\mathcal G_G^{\otimes k}) \leq (1-\Omega(\delta^{\alpha/2}))^{k/c} \leq \delta$

for $k \approx \delta^{-\alpha/2} \log(1/\delta)$. Taking $\delta \to 0$, we get $\mathsf{val}(\mathcal G_G^{\otimes k}) \to 1$ in the first case, and $\mathsf{val}(\mathcal G_G^{\otimes k}) \to 0$ in the other. Since $\mathcal G^{\otimes k}$ is a unique game whenever $\mathcal G$ is unique, this completes the proof.

### MAX-CUT on a cycle

To this end, a number of authors have asked whether a strong parallel repetition theorem holds, i.e. whether $\alpha^* = 1$ (even for general games).

The rest of the lecture follows Feige, Kindler, and O’Donnell.  We’ll consider a very simple unique game: MAX-CUT on an odd cycle, though we’ll slightly change the probabilities of our earlier verifier for the sake of convenience. We define the game $\mathcal G_m$ as follows: $Q=Q'=\mathbb Z_m$, $A=A'=\{0,1\}$, and $\mu$ is specified by first choosing $q \in \mathbb Z_m$ uniformly at random, and then choosing $q' = q+x$ where $x \in \{-1,0,1\}$ is chosen uniformly at random. As before, when $q=q'$, $V$ accepts only if the answers satisfy $a=a'$, and otherwise $V$ accepts only if the answers satisfy $a \neq a'$.

It is easy to check that for $m$ odd, we have

$\displaystyle\mathsf{val}(\mathcal G_m) = \frac13 + \frac23 \textsf{max-cut}(m\textrm{-cycle)} = 1 - \frac{2}{3m},$

since any cut in an odd cycle must have some some edge which doesn’t cross it (e.g. the edge with two green endpoints above). The point now is to understand the behavior of $\mathsf{val}(\mathcal G_m^{\otimes k})$.

Toward this end, consider the graph $(\mathbb Z_m^k)_{\infty}$ whose vertex set is $\mathbb Z_m^k$ and which has an edge $(u,v)$ whenever $|(u_i - v_i) \textrm{ mod } m| \leq 1$ for $i=1, 2, \ldots, k$ (this is also known as the $k$-fold AND-product of an $m$-cycle with itself).  For instance, here is $(\mathbb Z_5^2)_{\infty}$, where the dashed edges are meant to wrap around.

There is a natural embedding of $(\mathbb Z_m^k)_{\infty}$ into the $k$-dimensional torus $\mathcal T^k = \mathbb R^k/\mathbb Z^k$, which endows oriented cycles in $(\mathbb Z_m^k)_\infty$ with a homotopy class from $\mathcal T^k$.

Given such a cycle $\gamma$, we use $\lbrack\gamma\rbrack \in \mathbb Z^k$ to denote the corresponding element of the fundamental group. $\lbrack\gamma\rbrack = (c_1, c_2, \ldots, c_k)$ if $\gamma$ wraps around the torus $c_i$ times in the $i$th direction.

A cycle $\gamma$ is said to be topologically non-trivial if $\lbrack\gamma\rbrack \neq (0,\ldots,0)$.  This is the same as saying that $\gamma$ cannot be continuously contracted to a point in $\mathcal T^k$.  The cycle is topologically odd if $\lbrack\gamma\rbrack_i$ is odd for some $i = 1, 2, \ldots, k$.  Here are two topologically odd cycles on the 2-torus, corresponding to the classes $(0,1)$ and $(1,0)$.

In the following pictures, the red cycle is topologically trivial, the blue cycle is simply a shift of the red cycle, and the purple cycle is topologically non-trivial (indeed, it is topologically odd).

Finally, consider the double cover $\{0,1\} \times (\mathbb Z_m^k)_\infty$, which is a bipartite graph with vertex set $\mathbb Z_m^k \times \mathbb Z_m^k$, and with an edge $(0, x) \sim (1,y)$ whenever $(x,y)$ is an edge in $(\mathbb Z_m^k)_\infty$ or $x=y$. Oriented cycles in $\{0,1\} \times (\mathbb Z_m^k)_\infty$ inherit the notions of topological non-triviality and oddness from the projection $\{0,1\} \times (\mathbb Z_m^k)_\infty \to (\mathbb Z_m^k)_\infty$.  A curve in $\{0,1\} \times (\mathbb Z_m^k)_\infty$ is said to be non-trivial or odd if its projection is.  Now consider the following problem.

Odd-cycle elimination problem in $\{0,1\} \times (\mathbb Z_m^k)_\infty$:

Remove the minimum-possible fraction of edges $\delta'(k,m)$ from $\{0,1\} \times (\mathbb Z_m^k)_\infty$ so as to delete all topologically odd cycles.

For example, here is a natural set of blocking edges (projected from $\{0,1\} \times (\mathbb Z_m^k)_\infty$ onto $(\mathbb Z_m^k)_\infty$).

It turns out that this is just $\mathsf{val}(\mathcal G^{\otimes k})$ in disguise.  For example, the above set of blocked edges corresponds to a strategy where $P$ and $P'$ play each coordinate independently.  More generally, we have the following:

Claim 1.4: $\mathsf{val}(\mathcal G_m^{\otimes k}) = 1 - \delta'(k,m)$.

Proof:

Consider two provers $P, P' : \mathbb Z_m^k \to \{0,1\}^k$ for the repeated game $\mathcal G^{\otimes k}$.  Edges in $\{0,1\} \times (\mathbb Z_m^k)_{\infty}$ correspond to pairs of questions $(q,q') \in (\mathbb Z_m^k)^2$.  Let $\mathcal E(P,P')$ be the set of edges corresponding to questions on which $P,P'$ answer incorrectly, i.e. edges $((0,q),(1,q'))$ for which $V^{\otimes k}(q,q',P(q),P(q')) = 0$.  Clearly

$\displaystyle \mathsf{val}_{P,P'}(\mathcal G^{\otimes k}) = 1 - \frac{|\mathcal E(P,P')|}{3^k m^k},$

where we recall that $3^k m^k$ is the number of edges in $\{0,1\} \times (\mathbb Z_m^k)_{\infty}$.

Say that an arbitrary set of edges $\mathcal E$ is blocking if removing $\mathcal E$ leaves no topologically odd cycles.  We need to show that every set of the form $\mathcal E(P,P')$ is blocking, and that from every blocking set $\mathcal E$, we can recover two provers $P,P'$ with $|\mathcal E(P,P')| \leq |\mathcal E|$.

First, assume that $\mathcal E(P,P')$ is not blocking.  Then there exists a cycle $\gamma$ in $\{0,1\} \times (\mathbb Z_m^k)_{\infty} \setminus \mathcal E(P,P')$ and a coordinate $1 \leq i \leq k$ such that $\lbrack\gamma\rbrack_i$ is odd.  But if we consider the projection of $\gamma$ onto coordinate $i$, then we get an odd cycle on which $P,P'$ play perfectly, which is clearly impossible.

On the other hand, consider an arbitrary blocking solution $\mathcal E$.  From any connected component $C$ of $\{0,1\} \times (\mathbb Z_m^k)_{\infty} \setminus \mathcal E$, choose an arbitrary vertex, say $(0,q) \in C$, and put $P(q)=(0,0,\ldots,0)$.  Once this is set, since we cannot violate any edges in $C$, there is a unique greedy extension of $P,P'$ to the rest of $C$.  For instance, if $(1,q) \in C$, then we must put $P'(q)=(0,0,\ldots,0)$ as well.  If $(1,q') \in C$ and $q$ and $q'$ differ by 1 in the $i$th coordinate, then $P'(q')=(0,\ldots, 1, \ldots, 0)$, where the 1 occurs in the $i$th coordinate.  It is easy to see that this greedy assignment cannot fail precisely because there are no topologically odd cycles remaining in $C$ (every bipartite graph has a cut which contains all edges).  Note that topologically trivial cycles are inconsequential since the coordinate projection of such a cycle results in a path.  This completes the proof.

Now define $\delta(k,m)$ to be the minimum be number of edges which need to be removed from $(\mathbb Z_m^k)_\infty$ so as to eliminate all topologically non-trivial cycles. It is easy to see that $\mathsf{val}(\mathcal G_m^{\otimes k}) = 1 - \delta'(k,m) \geq 1 - 2 \delta(k,m),$ because we can remove all topologically non-trivial cycles in $\{0,1\} \times (\mathbb Z_m^k)_\infty$ (and, in particular, all topologically odd cycles) by taking a set $F$ of edges which do this for $(\mathbb Z_m^k)_\infty$ and using the edges $\left\{\langle(0,x),(1,y)\rangle, \langle(1,x),(0,y)\rangle : (x,y) \in F\vphantom{\bigoplus}\right\}$ in $\{0,1\} \times (\mathbb Z_m^k)_\infty$.

Thus in order to “cheat” at the odd-cycle game, we only need to find small sets of edges whose removal blocks all non-trivial cycles.  In this next lecture, we will see how this is intimately related to foams which tile $\mathbb R^k$, and how Dirichlet eigenfunctions help us find good foams.

[Credits:  Some pictures taken from this talk of Ryan O'Donnell.]

1. [...] and near-optimal foams Filed under: Uncategorized — jrluw @ 2:09 am In the last lecture, we reduced the problem of cheating in (the k-times repeated m-cycle game) to finding a small set [...]

Pingback by Lecture 2: Spectral partitioning and near-optimal foams « tcs math - some mathematics of theoretical computer science — September 26, 2008 @ 2:09 am

2. Some typos:

In the proof of Claim 1.4, the equation for the value of the game played multiple times is missing a factor of m^k in the denominator (and in the description of the number of edges in the double cover graph listed below the equation.)

Also you say at the end of the proof “Note that topologically trivial odd cycles…” Do you really mean to add “odd” there?

Comment by Dave Bacon — September 29, 2008 @ 5:46 pm

3. thanks dave! at least I got this right in the actual lecture :)

Comment by James Lee — September 29, 2008 @ 11:19 pm

4. Hi James,

I liked your lecture and it was very well scribed.
I had a comment. You mention about the implication of a strong parallel repetition theorem (One with an exponent of < 2 for $\epsilon$.

You don’t mention a recent paper by Ran Raz,

A counter example to strong parallel repetition theorem

where he rules out the existence of such a theorem even for projection/unique/XOR games.

In essence he shows a protocol for the n-repeated odd cycle game that achieves $(1-(\frac{1}{m})O(\sqrt{n}))$

Comment by sachdevasushant — October 2, 2008 @ 5:47 pm

5. You might want to check out Lecture 2.

Comment by James Lee — October 2, 2008 @ 5:51 pm

6. Profitez d’une voyance totalment gratuite sur notre site de tarologie et d’arts divinatoires.

Une séance de voyance gratuite va certainement changer votre existence, par ce que toute personne
qui a connaissance de son futur profite mieux des opportunités au
quotidien.
Tous les médiums de notre équipe de voyance ont des années d’expérience dans la prédiction de
la vie amoureuse et dans la divination de la vie professionnelle, et
ils vous donneront avec plaisir des conseils précieux.

Ne vous posez plus de questions, faites le pas et parlez à un des médiums qui fait partie de
notre équipe d’astrologie pour une prédiction détaillée.

Afin de vous donner un sentiment de confiance et prouver le
sérieux de nos médiums, nous offrons la première consultation en privé
selon les conditions stipulées.

Comment by voyance gratuite — November 27, 2013 @ 9:41 pm