\documentclass[11pt,amsfonts,amsmath]{article}
\usepackage{times}
\usepackage[latin1]{inputenc}
\usepackage{amsfonts,amsmath,amsthm,amssymb,stmaryrd}


\title{Scalable Enumerations}
\author{}
\date{\today}

\begin{document}

\maketitle

\section{Motivation}

We propose an algorithm to count a very large set on the cloud.
Cloud API are really young, and they do not provide yet a scalable technique to enumerate some very large set.

The algorithm we propose is very simple, but has not been proposed sofar because it would have no sense on another hardware. Indeed, counting a set could be done in n operations where n is the number of points, where as the technique we propose use a sort on the set, with a cost of $O(nlog(n))$, which is greater, even if the constant is small. However, the proposed algorithm performs better on the cloud to count a very large set. We first give a very short introduction to Azure API and storage, before we explain why our algorithm behaves better than raw count.

\section{Description of Azure Framework}

INCLUDE A DESCRIPTION OF THE BLOB STORAGE WITH THE STRINGS

When adding an item to the blobStorage, we use API giving the item and an address expressed as a concatenated string. Let's take an example. Suppose we are defining address as follows : container/purpose/itemName. An item stored in the container Analytics, whose purpose is Regression, and name is item3 will be set in :\\

\textbf{Analytics/Regression/item3}\\

We are now changing the way we store items in the blobStorage.
Before ordering them by purpose, we first add a random number between $1 ~and ~M$ as an index. Therefore, for the same item, the new address could be :\\

\textbf{Analytics/21795/Regression/item3}\\

The Azure API provides a primitive to get the k first elements of a container when ordering them by their blobName. This primitive is supposed to be run in very low and constant time, because the sort is done when adding items in blobStorage, not when asking.

Therefore, calling this primitive with the new naming convention on the blobStorage will result in getting first all the items stored in \textbf{Analytics/1/...} , then if we have less than k items in this directory, the primitive returns the items stored in \textbf{Analytics/2/...} and so on...\\

Let $~X_1, X_2, ..., X_N ~be ~N ~random ~variables$\\
We $~define ~X_{(i)} ~as ~the ~ith ~shortest ~value ~in ~the ~set ~{X_1,..., X_N}$\\
Then, $X_{(1)} = \min{ \{X_1,..., X_N\}} ~and ~X_{(N)} = \max{ \{X_1,..., X_N\}} $

What the primitives gives us is a way to get $X_{(k)}$ in a very quick way, because the ordering has been pre-computed.

\section{Estimation of the set count}

We propose in this section a new way to compute an approximation of the set by using the order statistics.\\
Suppose we have a set of N items. For each item, we are giving a random value index from $1$ to $M$.\\
As explained in previous section, for any $k$ we can get $X_{(k)}$ in a very short time.
We could approximate the ratio $\frac{N}{M}$ which corresponds to the average number of points that take a fixed value by the corresponding quantity : $\frac{k}{X_{(k)}}$. Therefore, we propose to approximate N by :

\begin{equation}
\label{orderingStatisticsAsEstimatorOfSetCount}
\hat{N} = \frac{kM}{X_{(k)}}
\end{equation}

\end{document} 