\documentclass[11pt]{article}
\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{xcolor}
\usepackage{epsfig}
\usepackage[usenames,dvipsnames]{pstricks}

\DeclareMathOperator*{\E}{\mathbb{E}}
\let\Pr\relax
\DeclareMathOperator*{\Pr}{\mathbb{P}}

\newcommand{\eps}{\epsilon}
\newcommand{\inprod}[1]{\left\langle #1 \right\rangle}
\newcommand{\R}{\mathbb{R}}

\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {\bf CS 388R: Randomized Algorithms } \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
      \textcolor{red}{\textbf{NOTE:} THESE NOTES HAVE NOT BEEN EDITED OR CHECKED FOR CORRECTNESS}
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[4]{\handout{#1}{#2}{#3}{Scribe: #4}{Lecture #1}}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}

% 1-inch margins, from fullpage.sty by H.Partl, Version 2, Dec. 15, 1988.
\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in

\parindent 0in
\parskip 1.5ex

\begin{document}

\lecture{8 --- September 26, 2017}{Fall 2017}{Prof.\ Eric Price}{Sushrut Karmalkar \& Ziyang Tang}

\section{Perfect Hashing}
\subsection{Overview}
In the previous lecture, we analyzed Cuckoo Hashing, which still has a very low chance of collision. Also, Cuckoo Hashing requires fully randomize functions. Today we will try to find a perfect hashing scheme with no collisions, and which only requires pairwise randomized.

\subsection{Goal}
\begin{definition}
A hash function h is \textbf{perfect} for $S\in [U]$ if it has no collisions, i.e. $h(x)\neq h(y)$ for all $x\neq y$ and $x,y\in S$.
\end{definition}

\paragraph{Goal:} 
For a given set $S$ of size $k$, find a perfect hash function $h:[U]\to [m]$, we want $m$ as small as possible.

\subsection{Intuition and Easy solutions}
\begin{itemize}
\item Identity function: $h(x) = x$, but with $m=U$.
\item Giant lookup table via hash table, actually depends on which hash table you pick.
\item Pairwise independent hash function with $m=O(k)$, but not perfect with $O(\frac{\log k}{\log\log k})$ worst case lookup.
\end{itemize}

Now we still use a pairwise independent hash function $h$ but with more space than $O(k)$, we want to find an upper bound for $m$ such that no collisions will occur.

\begin{lemma}
With probability more than $\frac{1}{2}$ we can find a perfect random pairwise independent hash function $h$ with $m = k^2$ for $S$.
\end{lemma}
\begin{proof}
Using Markov, we have,

Pr[$h$ is not perfect for $S$] = Pr[$h$ has at least 1 collision for $D$] $\leq$ E[number of collisions for $h$].

By expanding E[number of collisions for $h$] as pairs of collision indicators we get:
\begin{equation}
E = \sum_{x_1<x_2\ x_1,x_2\in S}Pr[h(x_1)=h(x_2)] \leq {{k}\choose {2}}\max_{x_1,x_2} Pr[h(x_1)=h(x_2)] \leq \frac{k^2}{2m}
\end{equation}

Therefore we know if we let $m=k^2$ and we randomly choose a hash function that is pairwise independent, we will have failure probability at most a half.
\end{proof}

If we design a Las Vegas algorithm to repeatedly find a pairwise independent hash function, after a constant number of tries, we will get a perfect hash function with high probability.

\subsection{Perfect Hashing}
Now we have a way to find a perfect hash function with $m=k^2$. However, we want $m=O(k)$.

Suppose we first get a random hash function $h^*[U]\to [m]$, with $m=O(k)$. This hash function may have collisions. Create a linked list for each collision.

Now suppose we map each linked list with size $k'$ with a perfect hash function  with size $k'^2$, we can then flatten that link list out and store with some extra space to make it a perfect hash function.


\psscalebox{1.0 1.0} % Change this value to rescale the drawing.
{
\begin{pspicture}(0,-3.2)(9.2,3.2)
\rput{-90.0}(4.22,8.18){\pstriangle[linecolor=black, linewidth=0.04, dimen=outer](6.2,1.78)(0.4,0.4)}
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,2.4)(3.2,-3.2)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,2.4)(3.2,1.6)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,1.6)(3.2,0.8)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,0.8)(3.2,0.0)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,0.0)(3.2,-0.8)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,-2.4)(3.2,-3.2)
\rput(2.8,1.2){$h^*(z)$}
\rput(4.8,2.0){$x_1$}
\rput(4.8,1.2){$x_2$}
\rput(4.8,0.4){$x_3$}
\rput(4.8,-2.8){$x_n$}
\psline[linecolor=black, linewidth=0.04](5.2,2.0)(6.0,2.0)(6.0,2.0)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.4,2.4)(6.8,1.6)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](7.6,2.4)(7.2,1.6)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.4,2.4)(8.0,1.6)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](7.2,2.4)(6.8,1.6)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.0,2.4)(7.6,1.6)
\rput{-90.0}(5.02,7.38){\pstriangle[linecolor=black, linewidth=0.04, dimen=outer](6.2,0.98)(0.4,0.4)}
\psline[linecolor=black, linewidth=0.04](5.2,1.2)(6.0,1.2)(6.0,1.2)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](9.2,1.6)(6.8,0.8)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](7.2,1.6)(6.8,0.8)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](7.6,1.6)(7.2,0.8)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.0,1.6)(7.6,0.8)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.02,1.6)(8.0,1.58)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.4,1.6)(8.0,0.8)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.8,1.6)(8.4,0.8)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](9.2,1.6)(8.8,0.8)
\end{pspicture}
}


More formally, we have $h^*:[U]\to [m]$, and $h_i:[U]\to [Z_i^2]$ to be a perfect hashing, where $Z_i$ is the number of elements that hash to cell i, or mathematically denoted as $|(h^*)^{-1}\cap S|$.

Record $Y_i = \sum_{j<i}Z_i^2$. We set our final perfect mapping as 
$$h(u) = Y_i+h_i(u), where\ i=h^*(u)$$.

It is then easy to see that $h$ is perfect with range $\sum_{i=1}^m Z_i^2,$ we need to estimate $\sum_{i=1}^m Z_i^2$.

Since total number of collisions equals to
$$\sum_{i=1}^m {{Z_i}\choose {2}} = \frac{1}{2}(\sum_{i=1}^m Z_i^2)-\frac{k}{2},$$

By taking expected value of both sides we have:
\begin{equation}
E[\sum_{i=1}^m Z_i^2] = 2E[\mbox{total number of collisions}] + k \leq \frac{2k^2}{2m}+k
\end{equation}

If we let $m=k$, we will get $2k$ to be the expected size of our hash function $h$.

Now we get a Las Vegas algorithm to rebuild $h$ until size of h is less or equal to $4k$. By Markov, each round our success probability is greater than $\frac{2k}{4k} = \frac{1}{2}$, thus with $O(1)$ rounds or $O(k)$ total times we will success with high probability.

If we want to further achieve $m=k$, we can use a lookup table for the perfect hashing, where we index each non-empty element in the mapping of perfect hashing $h$ as $I$, then we take $h'(x) = I_{h(x)}$ which still runs in $O(k)$ times.


\psscalebox{1.0 1.0} % Change this value to rescale the drawing.
{
\begin{pspicture}(0,-3.4875)(8.0,3.4875)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,2.6875)(3.2,-2.9125)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,2.6875)(3.2,1.8875)
\psframe[linecolor=black, linewidth=0.04, fillstyle=crosshatch, hatchwidth=0.028222222, hatchangle=0.0, hatchsep=0.1411111, dimen=outer](4.4,1.8875)(3.2,1.0875)
\psframe[linecolor=black, linewidth=0.04, fillstyle=crosshatch, hatchwidth=0.028222222, hatchangle=0.0, hatchsep=0.1411111, dimen=outer](4.4,1.0875)(3.2,0.2875)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,0.2875)(3.2,-0.5125)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,-2.1125)(3.2,-2.9125)
\rput(2.8,1.4875){$h(x)$}

\psframe[linecolor=black, linewidth=0.04, dimen=outer](4.4,-0.5125)(3.2,-1.3125)
\psframe[linecolor=black, linewidth=0.04, fillstyle=crosshatch, hatchwidth=0.028222222, hatchangle=0.0, hatchsep=0.1411111, dimen=outer](4.4,-1.3125)(3.2,-2.1125)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.0,2.6875)(6.8,1.8875)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.0,1.8875)(6.8,1.0875)
\psframe[linecolor=black, linewidth=0.04, dimen=outer](8.0,1.0875)(6.8,0.2875)
\psline[linecolor=black, linewidth=0.04](4.4,1.4875)(6.8,2.2875)(6.8,2.2875)
\psline[linecolor=black, linewidth=0.04](4.4,0.6875)(6.8,1.4875)
\psline[linecolor=black, linewidth=0.04](4.4,-1.7125)(6.8,0.6875)
\rput(6.0,1.4875){$I$}
\rput(3.6,-3.3125){$[4n]$}
\rput(7.6,-0.1125){$[n]$}
\end{pspicture}
}

\section{Lower bound on hashing}
To hash a set $S$ of size k in $[U]$, lots of scheme give $O(k)$ word of space, where 1 word = $\log U$ bits. A natural question is to ask, can we do better?

Suppose the hash table was stored using $b$ bits, then the total number of possible representations you can have is at most $2^b$. Since your representations must include all possible subsets of size $k$ of $U$ we see that $2^b \geq \left(|U|_k\right) \geq \left(\frac{|U|}{k}\right)^k = 2^{k\log(|U|/k)}$. If $k < \sqrt{|U|}$ then we see $b \geq \frac{1}{2}k \log(|U|)$. However, $\log(|U|)$ is the size of any word, and so we need $\Omega(k)$ words. This means if we need to be able to has ALL POSSIBLE sets, then we cannot do better than a regular hash function. 

\section{Bloom Filters}
This is a set membership data structure with some chance of false positives. In particular, for a particular set $S$ you can get queries of the kind $x \in S?$, if the answer is `yes' you would like to be always right, however if the answer is `no', then you are allowed to fail with probability $1-\delta$. It is possible to do this with $O(k\log(\frac{1}{\delta}))$ bits. 

Applications of this structure: 
\begin{itemize}
\item Use the filter before a slow operation (for example, chrome uses this to maintain a list of malicious urls).
\item Database joins (`Does this key have a different entry in the corresponding table?')
\item Bitcoin (to speed up wallet synchronization). 
\end{itemize}

Let $n$ be the number of items, $m$ be the number of buckets. The datastructure picks up $k$ uniform random hash functions $h_1, \dots, h_k$ where $k$ is a parameter to be decided later. You then store $\vec{y} \in \{0,1\}^m
$ where $y_j = 1$ iff $\exists x \in S, i \in [k]. h_i(x)=j$. Respond with `yes' to a query on $x$ iff $x \in \cap_{i \in [k]} Y_{h_i(x)}$.

We now analyze the failure probability of this. Let $p = $ the fraction of $0$'s in an array. $E[p] = Pr[\text{any single entry is $0$}] = \left(1-\frac{1}{m}\right)^{nk} \approx e^{-nk/m}$. The variables negatively associate and hence concentrate, which means $p$ is most probably going to be the expectation, upto a constant. The probability that one of these was $1$ is $(1-p)$ and so we have $\delta = (1-p)^k \approx (1-e^{-nk/m})^k$. We will try to find the $k$ that minimizes this value. To do this, observe that $(1-e^{-nk/m})^k = [(1-e^{-z})^z]^{n/m}$ where $z = \frac{nk}{m}$. It sufficies to minimize with respect to $z$ which can be done by differentiating the log and setting it to $0$. It turns out that at the minimum $k= \frac{m}{n}\ln(2)$ and $\delta < \frac{1}{2^k} = 0.618^{\frac{m}{n}}$. Setting $m = O(n\log(1/\delta))$ does the job.  
\end{document}
