\documentclass[11pt]{article}
\usepackage{amsmath,amssymb,amsthm}
\usepackage{xcolor}
\usepackage{natbib}
\DeclareMathOperator*{\E}{\mathbb{E}}
\let\Pr\relax
\DeclareMathOperator*{\Pr}{\text{Pr}}
\DeclareMathOperator*{\p}{\text{Pr}}

\newcommand{\eps}{\epsilon}
\newcommand{\inprod}[1]{\left\langle #1 \right\rangle}
\newcommand{\R}{\mathbb{R}}

\newcommand{\handout}[5]{
  \noindent
  \begin{center}
  \framebox{
    \vbox{
      \hbox to 5.78in { {\bf CS 388R: Randomized Algorithms, Fall 2019 } \hfill #2 }
      \vspace{4mm}
      \hbox to 5.78in { {\Large \hfill #5  \hfill} }
      \vspace{2mm}
      \hbox to 5.78in { {\em #3 \hfill #4} }
      \textcolor{red}{\textbf{NOTE:} THESE NOTES HAVE NOT BEEN EDITED OR CHEKED FOR CORECTNES}
    }
  }
  \end{center}
  \vspace*{4mm}
}

\newcommand{\lecture}[5]{\handout{XX}{#3}{#4}{Scribe: #5}{Lecture #1: #2}}

\newtheorem{theorem}{Theorem}
\newtheorem{corollary}[theorem]{Corollary}
\newtheorem{lemma}[theorem]{Lemma}
\newtheorem{observation}[theorem]{Observation}
\newtheorem{proposition}[theorem]{Proposition}
\newtheorem{definition}[theorem]{Definition}
\newtheorem{claim}[theorem]{Claim}
\newtheorem{fact}[theorem]{Fact}
\newtheorem{assumption}[theorem]{Assumption}
\usepackage[utf8]{inputenc}

\usepackage{amsmath}
\usepackage{amssymb}
\usepackage{amsthm}
\usepackage{wasysym}
\usepackage{geometry}
\usepackage{color}
\usepackage{hyperref}
\usepackage{bm}
\usepackage{algorithm}
\usepackage{algorithmic}
\usepackage{enumitem}
\hypersetup{
    colorlinks=true,
    linkcolor=blue,
    filecolor=magenta,      
    urlcolor=cyan,
}

\topmargin 0pt
\advance \topmargin by -\headheight
\advance \topmargin by -\headsep
\textheight 8.9in
\oddsidemargin 0pt
\evensidemargin \oddsidemargin
\marginparwidth 0.5in
\textwidth 6.5in

\parindent 0in
\parskip 1.5ex

\newcommand*{\dif}{\mathop{}\!\mathrm{d}}
% \geometry{a4paper,scale=0.8}
\title{Randomized Algorithms Lecture 2}
\author{Jack Youstra, Feifei Zhang}
\date{September 17, 2019}

\begin{document}

\lecture{6}{Power of Two Choices}{September 17}{Prof.\ Eric Price}{Mofei Zhang, Jack Youstra}
% \maketitle

\section{Power of Two Choices}

Take the balls in bins problem, and modify it so that there's two choices (instead of one choice) for each ball to go into each bin (with each ball opting for the less-full bin). This should reduce the number of collisions.

\subsection{Expected number of balls in any bin}

Is one, same as with only one choice. A more interesting question arises about the maximum number of balls.

\subsection{Expected maximum number of balls in any given bin}

Assisted with a claim: Let 
$$V_{i}(t) = \text{number of bins after } t \text{ balls with more than } i \text{ balls inside}$$
$$h_t = \text{height at which the } t_{th} \text{ ball is placed}$$

An example. Suppose $h_{i}$ is the new height of the bin that the $i^{th}$ ball lands in, and suppose $t = 7$.

\begin{flalign*}
&&h = (h_{1}, h_{2}, ..., h_{7}) &= (1, 1, 1, 2, 1, 2, 3)&&\\
&&V_{1}(t) &= 4 &&\text{(count the number of ones)}\\
&&V_{2}(t) &= 2 &&\\
&&V_{3}(t) &= 1 &&\\
&&V_{i}(t) &= \text{number total where } h_{t} = i&&\\
\end{flalign*}
An aside - whenever ``with high probability'' is mentioned, it refers to $P = 1-n^{-c}$ for some arbitrary constant $c$.

We have another claim - increasingly higher heights are associated with an increasingly smaller number of bins with that height. Specifically, we have (with high probability) $\forall i \geq 4,\ 1\leq t\leq n$

$$V_{i}(t) \leq \beta_{i} * n$$

We can see a base case $$\beta_{4} = \frac{1}{4}$$

This sequence decays faster than exponentially:
$$\beta_{i+1} = 2\beta_{i}^{2}$$

And with high probability, there exists an $h = O(\log(\log(n)))$ such that the following statement holds.

$$V_{h}(t) < \frac{O(\log(n))}{n}$$

We prove this by inducting on $i$.

We choose a base case of $i = 4$ because one and two are too small to make the doubling every step irrelevant (relative to the exponential), and three could be inconvenient. The proof for the base case is trivial ($n$ balls can lead to max $\frac{n}{4}$ bins of height $\geq 4$).

For now, let's suppose something a bit more strong than the inductive hypothesis: suppose that the hypothesis is deterministically true for tree with height $i$. 

%< some garbage >

However, we can't suppose that it always happens - it may fail at a higher step, and if that happens, it'll steadily get worse. So, at each height $i$, define $Y_{t} = 1$ if $h_{t} = i+1$ and $V_{i}(t-1) \leq \beta_{i}n$. $Y_t=0$ otherwise. 

This implies that
$$
\E[Y_t]=\Pr[Y_t=1]\leq \Pr[h_t\geq i+1\ |\ V_i(t-1)\leq \beta_i n]\leq \left(\frac{\beta_i n}{n}\right)^2=\beta_i^2.
$$
Thus $\E[\sum_tY_{t}] \leq \beta_{i}^{2}n = \frac{1}{2} \beta_{i+1} n$.

The next step would be to analyze $\p\left[\sum Y_{t} \geq \beta_{i+1}n\right]$, but $Y_t$ aren't independent. However, there's a way out of this! If we're able to show that no matter what happens over the preceding steps $1, ..., t-1$, that $\E[Y_{t}\ |\ Y_1,\dots,Y_{t-1}] \leq \beta_{i}^2$, we could say that there exists set of independent random variables $Z = \{Z_{1}, ..., Z_{t}\}$ that \textbf{stochastically dominate} $y$. We say that $Z$ stochastically dominates $Y$ if there exists correspondents distribution over $Z \times Y$ with marginals $Z, Y$ such that $Z_{i} \geq Y_{i}$ always. This is true since if $V_i(t-1)>\beta_i n$ then all $Y_t,Y_{t+1},\dots,Y_n$ would be equal to 0, otherwise $\Pr[Y_t=1]\leq \beta_i^2$. Thus we have
\begin{align*}
\p\left[\sum Y_{t} \geq \beta_{i+1}n\right] &\leq \p\left[\sum Z_{t} \geq \beta_{i+1}n\right]\\
&\leq e^{-\frac{1}{6}\beta_{i+1}n}
\end{align*}
where the last inequality comes from Bernstein's inequality applied to independent variables $Z_t$.
Hence the $Y_t$'s are unlikely to collide, which we cared about because the $Y_t$'s are condition on the previous value. For large enough $\beta_{i+1}$ we can assume that
$$
\Pr\left[\sum Y_t\geq \beta_{i+1}n\right]< \frac{1}{n^c}
$$
for fixed constant $c>0$. Precisely speaking, $\beta_{i+1}$ needs to be $\Omega(\log n/n)$ with appropriate constant.

Let $Q_{i} = \mathbb{I}[V_{i+1}(t) \geq \beta_{i+1}n]$, then for $\beta_{i+1}=\Omega(\log n/n)$ we have that
\begin{align*}
  \p[Q_{i+1}]&\leq \p\left[Q_{i+1} \wedge \overline{Q_{i}}\right] + \p[Q_{i}]\\
&\leq \p\left[\sum Y_{t} \geq \beta_{i+1}n\right]+\p[Q_i]\\
&\leq \p[Q_{i}] + \frac{1}{n^{c}}.\\
\end{align*}

Letting $h^*$ be the largest $i$ such that $\beta_{i}=\Omega(\log n/n)$, by union bound with probability at least $1-n^{1-c}$, the condition $V_{i}(t)\leq \beta_i n$ holds for $i=1,\dots, h^*$.

Till now, we proved that with high probability there exists $h^*$ of $O(\log\log n)$ such that 
\begin{equation}
  V_{h^{*}}(t) < O\left(\frac{\log n}{n}\right).
  \label{Eq1}
\end{equation}

Then, we have to determine $\p[\text{any ball gets height larger than } h^{*}+1]$.
Observe that given (\ref{Eq1}) the probability of ball put higher than $h^*$ would be bounded by $O\big(\big(\frac{\log(n)}{n}\big)^2\big)$. Using this result we have
$$\p\left[\sum Y_{t} \geq c\right] \leq {n \choose c} \left(\frac{\log n}{n}\right)^{2c} \leq \left(\frac{ne}{c} \cdot \frac{\log^{2}n}{n^{2}}\right)^c < \frac{1}{n^{c/2}}$$
for large enough $n$.

Hence the max height is $\log(\log(n)) + O(1)$ with high probability.
\qed

What's $\E[\text{max height}]$?
\begin{align*}
  \E[\text{max height}]&\leq h^{*} + c + \p[\max h \geq h^{*} + c]\cdot(\text{max possible $h$})\\
  &\leq h^{*} + c + \frac{n}{n^{c/2}}\\
  &\leq h^{*} + O(1)
\end{align*}

\end{document}