\documentclass[11pt]{article}
\usepackage{amsmath,amssymb,amsthm}
\usepackage{xcolor}
\DeclareMathOperator*{\E}{\mathbb{E}}
\let\Pr\relax
\DeclareMathOperator*{\Pr}{\mathbb{P}}
\newcommand{\eps}{\epsilon}
\newcommand{\inprod}[1]{\left\langle #1 \right\rangle}
\newcommand{\R}{\mathbb{R}}
\newcommand{\handout}[5]{
\noindent
\begin{center}
\framebox{
\vbox{
\hbox to 5.78in { {\bf CS 388R: Randomized Algorithms } \hfill #2 }
\vspace{4mm}
\hbox to 5.78in { {\Large \hfill #5 \hfill} }
\vspace{2mm}
\hbox to 5.78in { {\em #3 \hfill #4} }
\textcolor{red}{\textbf{NOTE:} THESE NOTES HAVE NOT BEEN EDITED OR CHECKED FOR CORRECTNESS}
}
}
\end{center}
\vspace*{4mm}
}
\newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{Scribe: #4}{Lecture #1}}
\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}
% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in
\parindent 0in
\parskip 1.5ex
\begin{document}
\lecture{10 --- Oct. 3, 2017}{Fall 2017}{Prof.\ Eric Price}{Jianwei Chen, Josh Vekhter}
\section{Overview}
In the last lecture we talked about routing algorithms.
In this lecture we study fingerprinting. It is used when we want to check if two large files/strings/ matrices/polynomials are the same or not. One application of these techniques is verifying that a file you downloaded hasn't been tampered with.
\section{Matrix Multiplication}
To begin, let's consider the following problem: Given matrices $A,B,C\in\mathbb{R}^{n\times n}$, how do we know if $AB=C$?
The simplest approach is to compute the value of $AB$ using a deterministic algorithm and check if it is equal to $C$. There have been numerous algorithmic attempts to improve the efficiency of this computation, bounding it as $O(n^w)$ for various exponents:
\begin{itemize}
\item For naive approach, $w=3$
\item For Strassen's Algorithm (1969), $w=2.8074$
\item For Coppersmith--Winograd's algorithm (1990), $w=2.3755$
\item For Andrew Strothers' (2010), $w= 2.374$
\item For Virginia Williams' (2011), $w= 2.37286...$
\end{itemize}
It is a long standing open conjecture that the $O(n^2)$ computations are possible. It is also known that this lower bound is sharp, because it is necessary to read all of the entries of both matrices to be confident that they are the same.
However, there also exists a randomized algorithm that can easily solve this problem in $O(n^2)$ time. Simply choose a random vector $r\in\{0, 1\}^n$, and check whether $ABr=Cr$.
Now we need the analyze the possibility of that $AB\neq C, ABr=Cr$. We are going to show that $\Pr[ABr\neq Cr]\leq\frac{1}{2}$.
As $AB\neq C \iff AB-C\neq0$, there exists a nonzero row $v=\{v_1, v_2,\ldots,v_n\}$ of $AB-C$. Suppose $v_i\neq0$. To do the analysis we assume that we first picked all $r_j(j\neq i)$, then
\begin{align*}
\Pr\left[\sum v_{k}r_{k}=0\right] & =\Pr\left[v_{i}r_{i}+\sum_{j\neq i}v_{j}r_{j}=0\right]\\
& =\Pr\left[r_{i}=\frac{\sum_{j\neq i}v_{j}r_{j}}{v_{i}}\right]\\
& \le\frac{1}{2}
\end{align*}
Here we use the fact that $\sum_{j\neq i}v_{j}r_{j}/v_{i}$ can only be 0 or 1 (or neither) and $r_i$ is chosen randomly among 0 and 1, this gives us the probability of at most half.
\section{Polynomial Matching}
Now we consider another class of finger printing problems also in the spirit of partial evaluation: Checking if $P(x) * Q(x) = R(x)$.\footnote{A more general version of this formulation is verifying if $\prod P_i(x) = Q(x)$}
To be a bit more concrete, let's suppose we know (or can quickly estimate) the degree of each polynomial. For convenience, let's say that both $P$ and $Q$ are of degree $d$ and are reduced (i.e. they have the form $a_1x^d + a_2x^{d-1} + \dots + a_dx + a_{d_1}$).
In this case, the na\"ive approach yields a $O(d^2)$ algorithm to reduce this product (wuddup FOIL method).
A more sophisticated deterministic approach can reduce this product in $O(d log d)$ time (looking at you FFT algorithms)!
Another natural question to ask about polynomials is if they are equal to the zero polynomial (i.e. $p(x) = 0$ for all $x$). Note that this is a more general problem than polynomial comparison (because given polynomials $p(x)$ and $q(x)$, one natural way to test equality is to check if $p(x) - q(x) = 0$ for all $x$). Here the na\"ive (deterministic) approach is to apply the test the polynomial at any $d+1$ points. If they are all zero then the polynomial must be zero everywhere by the fundamental theorem of algebra (recall that this theorem says a mono-variate polynomial over $\mathbb {C}$ has exactly $d$ roots, and note that it is possible to achieve an analogue on finite fields via the euclidean algorithm).
But there's a practical issue here because often polynomials come in unreduced forms. As a back of the envelope argument, note that a $d$ degree polynomial can be represented as the product of $O(2^{\sqrt{d}})$ strings with distinct terms (because each power of $x^k$ may or may not appear in each string). It may take up to $O(d)$ time to evaluate each pairwise product, and there are potentially exponentially many of these. As a more concrete example, note that the na\"ive algorithm for computing the determinant of a matrix generates a multi-variate polynomial with $O(d!)$ terms to reduce. More generally, it can be shown that it's possible to construct a correspondence between evaluating zeros in multi-variate polynomials over $\mathbb{F}^2$ and 3SAT (see \cite{circuit}).
Thus a natural question to ask is: can we determine if a polynomial $p(x)$ is the zero polynomial with less than $d + 1$ evaluations w.h.p? Suppose we restrict ourselves to the case where $p(x)$ is evaluated over the finite field of size $B$. In this case, our chance of selecting a root of the polynomial is $\leq \frac{d}{B}$. Thus if we set $B = 2d$ then at each trial, we have a $\frac{1}{2}$ chance of selecting a root which implies that after $k$ trials, if each time the polynomial evaluated to zero, then the probability that it is not the zero polynomial is $\frac{1}{2^k}$, which allows us to determine if $p(x)$ is the zero polynomial in $O(1)$ evaluations w.h.p.
One detail worth noting here is that in order for the above argument to work, $B$ actually needs to be relatively prime to all factors of $p(x)$, i.e. we need to find a prime $L$ such that $L$ is prime and $L > 2d$. By \cite{bert}'s Postulate, such a prime must exist between $n < p < 2n$, and so we know that $L < 4d$. It's also worth noting that a very similar argument gives rise to the Schwartz--Zippel lemma for testing zeros in multi-variate polynomials (which is more surprising because non-zero multivariate polynomials can have infinitely many roots)\cite{schwartz}\cite{zipple}.
\section{String Fingerprinting}
Let's consider an application of the probabilistic polynomial matching algorithm described above. Suppose Alice has a string $a=a_1a_2\ldots a_n\in\{0, 1\}^n$, Bob has has a string $b=b_1b_2\ldots b_n\in\{0, 1\}^n$, and we want to compute randomized fingerprints to check if $a=b$.
A natural idea is that if Alice and Bob have some shared seed of randomness, they can both compute a hash of their strings and send the hashed value as the fingerprint. However, if they don't share such randomness, we might need to send the hash function along with the fingerprints.
\subsection{Rabin-Karp Hashing}
We can use the idea in fingerprinting polynomials and treat the strings as polynomials. This is Rabin-Karp Hashing Algorithm. Namely, Alice use a random $x$ computes $\sum a_ix^i$ and send $x$ and the $\sum a_ix^i$, Bob computes $\sum b_ix^i$ and check if they are the same. This can be done using $O(\log n)$ bits using the previous method learned in polynomial matching.
\subsection{Another Method}
When we express those strings as binary vectors, we can consider another equivalent setting of the problem: We have $a\in\left[2^{n}\right]$ and $b\in\left[2^{n}\right]$, we want to know if $a=b$.
One way is to check if $\left(a-b\right)\equiv0\text{ (mod }p_{i}\text{)}$, $p_i$ is a chosen prime in some way. Then we need to ask how many primes can divide a n-bit number $a-b$? We know that $\pi(n)$ (the number of primes below n) has such relationship: $\pi(n)\sim \frac{n}{\log n}