\graphicspath{{../../Pictures/Logos/ }{../../Pictures/MapReduce/}{././Pictures/}}
\newcommand{\Rb}[1]{\ensuremath{\mathbb{R}^{#1}}}

%\section{Microsoft Windows Azure}
%Windows Azure Platform is Microsoft's cloud computing solution, in the form of
%Platform as a Service (PaaS). The underlying cloud operating system (Windows
%Azure) provides services hosting and scalability. It is composed of a storage
%system (that includes Blob Storage and Queue Storage) and of a
%processing system (that includes \emph{web roles} and \emph{worker
%  roles}). The Azure components we used are described below.
%
%The architecture of an Azure hosted application is based on two components:
%\textbf{web roles} and \textbf{worker roles}. Web roles are designed for web
%application programming and do not concern the present article. Worker roles
%are designed to run general background processing. Each worker role typically
%gathers several \emph{cloud services} and uses many \emph{workers} (Azure's
%processing units) to execute them. Our prototype uses only one worker role,
%several services and tens of PU. The computing power is provided by the
%workers, while Azure storage system is used to implement synchronization and
%communication between workers. It must be noted indeed that Azure does not
%offer currently any standard API for distributed computation, neither a low
%level one such as MPI \cite{MPI1996}, nor a more high level one such as Map
%Reduce \cite{MapReduce2004} or Dryad \cite{Dryad2007}. Map reduce could be
%implemented using Azure components (following the strategy of
%\cite{CloudMapReduce2010}), yet, as pointed out in e.g. \cite{Graphlab2010},
%those high level API might be inappropriate for iterative machine learning
%algorithms such as the k-means. We rely therefore directly on Azure queues and
%blob storage.
%
%\textbf{Azure Queues} provide a message delivery mechanism through distributed
%queues. Queues are designed to store a large amount of small messages (with
%maximal individual size of 8 KB). Using queues to communicate helps building
%loosely coupled components and mitigates the impact of individual component
%failure. Here, queues are used to store job messages. Messages stored in a
%queue are guaranteed to be returned at least once, but possibly several times:
%this requires one to design idempotent jobs.  When a message is unqueued by a
%worker, this message is not deleted but it becomes invisible for other
%workers. If a worker fails to complete the corresponding job (because it
%throws some exception or because the worker dies), the message becomes
%available after a certain period of time. Through this process, one can make
%sure no job is lost because of e.g., a hardware failure.
%
%\textbf{Azure Blob Storage} enables applications to store large objects, up to
%50 GB each.  It supports a massively scalable blob system, where hot blobs will
%be served from many servers to scale read access. Blobs are composed of a
%string (that is used as a key to store the value), a value composed of a
%binary object, and a timestamp (etag) (that indicates the last write on this
%blob). In addition of Get and Put methods, blobs whose key shares a given
%prefix can be listed. Optimistic non locking atomic read-modify-write
%operations can be implemented using a timestamp matching condition: a write
%succeed if and only if the timestamp of the storage matches the one provided
%by the write operation.
%
%Our prototype uses
%\textit{Lokad-Cloud}\footnote{\url{http://code.google.com/p/lokad-cloud/}}, an
%open-source framework that adds a small abstraction layer to ease Azure
%workers startup and life cycle management, and storage access.

%\section{Proposed implementation}
%Our prototype consists in three cloud services (setup, map and reduce
%services), each one matching a specific need. A queue is associated to each
%service: it contains messages specifying the storage location of the data
%needed for the jobs. Workers are stored in a pool and regularly ping the queues
%to acquire a message. Once it has acquired a message, a worker starts running
%the service related to the queue where the message was stored, and the message
%becomes invisible till the job is completed or timeouts. Overall, we use
%$P+\sqrt{P}+1$ processing units in the services described below.
%
%The \textbf{SetUp Service} generates P split datasets of N/P points in each
%and put them into the BlobStorage. It is also generating the original shared
%prototypes which are also stored in the BlobStorage. Once completed, it pushes
%P messages in the queue corresponding to the "Map Service". Each message
%contains a jobId related to a split dataset to be processed and a groupId used
%in the "Reduce Service". It also pushes $\sqrt{P}$ messages in the queue
%corresponding to the "Reduce Service". In the current implementation, the
%SetUp service is executed by P workers (PU) as the dataset is randomly
%generated. For a real application, the SetUp service could be executed by one
%worker or by several workers, depending on the availability and storage format of
%the data in the BlobStorage.
%
%When executing a \textbf{Map Service} job, a worker first downloads the
%dataset it is in charged of (once for all). Then the worker loads the initial
%shared prototypes and starts the computation step of attributing each point in
%the dataset to the closest prototype. New local prototypes version are build
%on the fly as the points are being processed. When all the map jobs have been
%completed, reducers must be notified to aggregate the results. We considered
%two designs for this synchronization process.
%
%A first solution consists in using the atomic read-modify-write operation to
%store a shared counter in the BlobStorage. Each worker increases the counter
%once it has completed its iteration. When the counter reaches P, reduction is
%started. A second possibility uses directly the reduce service. It queries on a
%regular basis the BlobStorage for the results of the Map Service. When it
%finds the required results, the reduction is started.
%
%We chose to use the second solution for several reasons. Firstly, this
%solution is compatible with multiple execution of the same map service job: if
%a map job is run twice, then the result is written twice at the same place and
%it does not affect synchronization. Secondly, it could help reducing
%straggler issues by overlapping latest computations with retrieval of first
%available results. Thirdly, it handles both synchronization and communication
%between mappers and reducers. Finally, it proved to be much faster when the
%number of workers reaches between 50 and 100. Indeed, optimistic non locking
%read-modify-write schemes are not adapted to high contention situations: for
%instance, we observed counter updating times up to 20 seconds with 70 workers.
%
%In practice, when all the points of a map service job have been processed
%once, the local prototypes version and their weights are pushed into the
%storage according to the following addressing rule: iteration/groupId/jobId.
%The worker then starts waiting, pinging the shared prototypes of the current
%iteration until it becomes available.
%
%Each \textbf{Reduce Service} job has a groupID, obtained from one of the
%$\sqrt{P}$ messages pushed into the reduce queue. As explained above, each
%reducer is listing blobnames with prefix: iteration/groupId in the BlobStorage
%until it gets all the results expected within this groupId. Map results
%are downloaded as soon as available. Once all the map results expected have
%been retrieved and loaded locally, the reducer merges the results into a
%partial merged result, and pushed this in a partial reduce results
%directory. One last reducer runs the same process on the partial reduce results
%directory. Once the $\sqrt{P}$ partial reduce results are retrieved, the last
%reducer builds a new shared version out of the partial reduce results and pushes the
%shared prototypes into the storage. After several seconds, every map worker
%has been pinging the now available shared result blob, and the map step can be
%run again.

%\section{Expected performances}
%The assignment phase of the k-means is perfectly parallelized in
%\cite{DhillonModha2000} and in our implementation; its speedup is $P$, the
%maximal value. Global performances are only hindered by the reduction
%phase. On the high performance distributed memory system used in
%\cite{DhillonModha2000}, MPI's reduce is in $O(log(P))$. Neglecting this cost
%leads to a linear speedup confirmed in the large dataset case by
%\cite{DhillonModha2000}.
%
%In the proposed algorithm, the two phases reduce has not a $O(log(P))$ cost
%but rather a $2\sqrt{P}KDB$ cost (per iteration), where $B$ is the time needed
%to read a double from the BlobStorage (this cost neglects both the latency of
%the synchronization process and the issue of aggregated bandwidth, see details
%below).  This adds to the time needed to perform the $P$ parallel map jobs,
%given by $3NKDF/P$ (per iteration), where $F$ is the time needed to perform
%one floating point operation. As in \cite{DhillonModha2000}, increasing $N$
%brings the speedup closer to its $P$ limit. However, the main advantage of
%cloud computing over traditional clusters is the possibility to adapt the
%processing power to the data. In our case, given a dataset with fixed values
%of $N$, $D$ and $K$, the optimal number of mapper PU is given by
%$P^*=(3NF/B)^{2/3}$.
%For this value of $P^{*}$, which depends neither on $K$ nor on $D$, the total
%processing time consists in one third of mapping and two third of reducing.


%\section{Experimental results}
%We tested the performances of the proposed implementation, as well as the
%limits of the theoretical cost model on synthetic data generated uniformly in
%the unit hypercube (as pointed out above, timings are independent of the
%actual data values). We report the results of one out of several benchmarks
%that show the same general trends. The dataset consists of $N=500 000$
%observations in dimension $D=1000$ (4 GB). We look for $K=1000$ clusters. The
%algorithm is run for 10 iterations to get stable timing estimates. The
%observed floating point performances of our code on a small instance worker in
%Azure is 669 Mflop/s. The bandwidth of a single threaded worker is about
%8 MB/s. Neglecting loading time and memory issues (a small instance has only 1
%GB of memory), a sequential implementation would use approximately 6 hours and
%13 minutes to run the 10 iterations.
%
%The following table reports the total running time in seconds (including data
%loading) of the proposed implementation for different numbers of mapping PU
%($P$). We report the speedup over the theoretical total running time, the
%efficiency (speedup divided by $P+\sqrt{P}+1$, the total number of PU) and
%the theoretical efficiency predicted by the model.
%\begin{center}\scriptsize
%\begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|c|c|c|c}
%P &10&50&60&70&80&90&95&100&110&120&130&140&150&160\\
%\hline
%Time & 2223 & 657 & 574 & 551 & 560 & 525& 521 & 539 & 544 & 544 & 574 & 603 & 605 & 674\\
%\hline
%SpeedUp & 10.0 & 34.1 & 39.0 & 40.6 & 40.0 & 42.6 & 43.0 & 41.5 & 41.2 & 41.2 & 39.0 & 37.1 & 37.0 & 33.2\\
%\hline
%Efficiency & 0.67 & 0.58 & 0.57 & 0.51 & 0.44 & 0.42 & 0.41 & 0.37 & 0.34 & 0.31 & 0.27 & 0.24 & 0.23 & 0.19\\
%\hline
%Theo. Eff. & 0.63 & 0.61 & 0.60 & 0.55 & 0.53 & 0.49 & 0.48 & 0.47 & 0.43 & 0.41 & 0.37 & 0.36 & 0.33 & 0.32\\
%\end{tabular}
%\end{center}
%As expected, the total processing time is minimal for a specific value of $P$
%(here 95), for which one third of the total time is spent in the map phase (as
%predicted by the model). This is lower than the predicted value (170 for the
%chosen constants) as a consequence of the overestimation of the storage efficiency by
%the theoretical model. Further investigations are needed to get a more
%accurate cost model for large $P$. In particular, we need both to take into
%account latency and bandwidth aggregation issues. Indeed, Microsoft reports
%that the total aggregated bandwidth of a PU group tops at 800 MB/s: in our
%model, $B$ is independent of $P$, which underestimates the reduce cost for
%large values of $P$. In addition, we have observed a large latency between a
%write completion to the BlobStorage by a worker and the availability of the
%blob to other workers (up to two minutes in some cases). This latency should
%be modeled and taken into account to predict the optimal $P^*$.
%
%Nevertheless, the obtained performances are very satisfactory for a reasonable
%number of mapping processing units (around 60 for this data size). While there
%is room for improving our implementation, the latency issues might prevent
%resorting on a tree like $O(log(P))$ reducer as available in MPI. Without
%native high performances API, communication aspects will probably remain a
%major concern in cloud implementations.

%\section{Azure base performances}
%
%One can find some Azure performance benchmarks on \textit{AzureScope}\footnote{\url{http://azurescope.cloudapp.net/}}.
%Azure performance largely depends on our algorithm, our implementation and the framework (Lokad-cloud) we use.
%In order to use our model as a predictive tool to determine the optimal number of workers $P^{*}$ and the expected
%speedup, we estimated Azure performances by running some isolated jobs and recording time.
%
%\subsection{Read bandwidth}
%
%We pushed 8 blobs of 8 MB into the storage (this is the size of map results we are going to load back in reducers), and
%we measured the time spent to retrieve them. For a single worker using a single thread, we retrieved the blobs in
%7.79 seconds, implying a 8.21 MB/sec read bandwidth. We tried to use multiple threads on a single worker as
%advised by AzureScope to speedup the download process, and we found the best read bandwidth was obtained using
%5 threads: we retrieved the 8 blobs in 6,13 seconds, implying a 10.44 MB/sec read bandwidth. The multithreads
%read bandwidth we observed differs from what is achieved in AzureScope. This may be explained by two potential factors:
%we observed average bandwidth whereas
%AzureScope observed peak performances and we did not have the same experiment environment. Especially, all the parallel
%read requests were run on a single container instead of being distributed on several BlobStorage containers.
%
%This benchmark is very optimistic as opposed to our clustering experiments: the bandwidth we estimated was recorded while
%no other workers were reading or writing into the storage. Other parallel reads and writes can have two effects
%on the read bandwidth performance. Firstly, they can limit the bandwidth because of aggregated bandwidth boundaries
%(AzureScope is reporting 800 MB/sec bandwidth for reading operations). Secondly, they can increase the latency between
%a write completion in the blob storage and the availability of the blob to other workers. When running experiments
%with a large number of workers reading and writing in parallel, we sometimes experienced blobs already pushed into the storage
%becoming available only after 1 or 2 minutes. To use our model as a predictive tool to determine $P^{*}$, we used a value
%of bandwidth equals to 8 MB/sec, which is a very conservative assumption that leads us to underestimate communication costs.
%
%\subsection{Workers Flops performances}
%
%All our experiments were run on Azure small VM that are guaranteed to run on 1.6 GHz CPU. Yet because
%of virtualization we do not have any warranty in term of floating point operations speed. Therefore, to fit
%our predictive speedup model, we ran some map jobs (where the number of floating point operations is known)
%to determine how fast could our algorithm be run on these VM. The code was first run on an Intel Core
%2 Duo T7250 2*2GHz using only one core. We estimated our map jobs to be run at approximatively 750 MFlops
%on average. We then ran some of our map jobs in Azure and we get for the same code performance of 669 MFlops on
%the small VM.\\
%
%It is worth mentioning that with the number of workers we used (at max 160 mappers), we observed very little differences
%in processing time between workers. Flops performance may slightly move with time, but at a given time,
%workers performed at very similar speed.
%
%%\section{Benchmarks}
%
%%\subsection{Speedup}
%
%%The next table shows speedups with varying values of P. All the runs listed below were using
%%N = 500000, K = 1000, D = 1000 and I = 10. Theoretically, K and D should have the same impact on map
%%and reduce phases since map and reduce costs are supposed to be proportional of KD. Yet, since our
%%model does not take into account latency, having very small values of K and D would lead to underestimate
%%communication cost by neglecting latency. We have set K and D to be much smaller than N, but also big
%%enough to neglect latency.
%
%%The following table reports the total running time in seconds (including data loading) of the proposed
%%implementation for different numbers of mapping PU (P). We report the speedup over the
%%theoretical total running time, the efficiency (speedup divided by the total number of
%%PU : $P$ mappers and $\sqrt{P} + 1$ reducers) and the theoretical efficiency predicted by the model.
%%
%%\begin{center}\scriptsize
%%\begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|c|c|c|c}
%%P &10&50&60&70&80&90&95&100&110&120&130&140&150&160\\
%%\hline
%%Wall Time & 2223 & 657 & 574 & 551 & 560 & 525& 521 & 539 & 544 & 544 & 574 & 603 & 605 & 674\\
%%\hline
%%SpeedUp & 10.0 & 34.1 & 39.0 & 40.6 & 40.0 & 42.6 & 43.0 & 41.5 & 41.2 & 41.2 & 39.0 & 37.1 & 37.0 & 33.2\\
%%\hline
%%Efficiency & 0.67 & 0.58 & 0.57 & 0.51 & 0.44 & 0.42 & 0.41 & 0.37 & 0.34 & 0.31 & 0.27 & 0.24 & 0.23 & 0.19\\
%%\hline
%%Theo Efficiency & 0.63 & 0.61 & 0.60 & 0.55 & 0.53 & 0.49 & 0.48 & 0.47 & 0.43 & 0.41 & 0.37 & 0.36 & 0.33 & 0.32\\
%%
%%\end{tabular}
%%\end{center}
%%
%%As expected, the total processing time is minimal for a specific value of P (here 95), for which
%%one third of the total time is spent in the map phase (as predicted by the model). One can notice that
%%the speedup curve is relatively flat in a neighborhood of $P^{*}$. This value of $P^{*}$ is lower
%%than the predicted value (170 for the chosen constants) as a consequence of the overestimation of
%%the storage efficiency by the theoretical model. As explained above, our model made the assumption
%%that bandwidth is scalable with the number of workers, and this is no more true when the number of workers
%%is too high. For example, when running 160 workers, some workers spend one minute to load the dataset from
%%the storage, and for 2 experiments we ran with more than 100 workers, one blob took more than 2 minutes to
%%be retrieved by a reducer after being pushed by a mapper.
%
%%\subsection{Scaleup}
%
%%Since in our model $P^{*}$ is not proportional to $N$ anymore (as this was the case using MPI's broadcast), we cannot hope to
%%achieve linear scaleup. Therefore, the scaleup challenge turns into minimizing growth of wall time as N grows, using
%%$P^{*}(N)$ mappers. Theoretically, as our model gives a processing cost proportional to $\frac{N}{P}$ and a communication
%%cost proportional to $\sqrt{P}$, for $ P = P^{*} =(3NF/B)^{2/3}$ (where $B$ is the time needed
%%to read a double from the BlobStorage and $F$ is the time needed to perform
%%one floating point operation), we should get a global cost proportional to $N^{\frac{1}{3}}$.
%%Therefore, if N is expected to be multiplied by 8, wall time of the algorithm running $P^{*}(N)$ mappers should be multiplied by 2.
%%We estimated by running experiments for some given values of $N$ the best value $P^{*}(N)$ and we observed the corresponding amount
%%of wall time.
%%Values of K and D (K=1000 and D=1000) were kept constant for all the experiments to ease comparisons.
%%
%%\begin{center}\scriptsize
%%\begin{tabular}{c|c|c|c|c|c|c}
%% & N & $P^{*}$ & Wall Time & Sequential theoretic time & Effective Speedup & Estimated Speedup ($=\frac{P^{*}}{3}$)\\
%%\hline
%%Run 1 & 62500 & 27 & 264 & 2798 & 10.6 & 9\\
%%\hline
%%Run 2 & 125000 & 45 & 306 & 5597 & 18.29 & 15\\
%%\hline
%%Run 3 & 250000 & 78 & 384 & 11194 & 29.15 & 26\\
%%\hline
%%Run 4 & 500000 & 95 & 521 & 22388 & 43.0 & 31.6\\
%%
%%\end{tabular}
%%\end{center}
%%
%%One can see that $P^{*}(N)$ does not grow as fast as $N$. For all the experiments, speedup achieved
%%with $P^{*}$ is approximatively  $\frac{P^{*}}{3}$, and reducing step took about twice the
%%time spent in map step for this number of mappers, as predicted by our cost model. While the size
%%of the dataset is multiplied by 8, wall time is exactly multiplied by 2, as predicted by the model.
%
%%\section{Straggler issues and impact on performance}
%%\label{sec_StragglerIssues}
%%
%%We first set an experiment with 100 workers, composed of 85 mappers, 11 reducers
%%(=$\lceil\sqrt{85}\rceil+1$), and 4 backup workers (in case some workers would
%%be unavailable). Each mapper performed the same task 10 times, taking about 7 minutes.
%%No communication between workers were used here, we only record the empirical distribution
%%of the computation time of the 850 tasks run, as reported in the following graphic :
%%
%% \begin{tabular}{cc}
%%       \includegraphics[height=0.4\linewidth]{../../Pictures/Straggler.png} &
%%       \includegraphics[height=0.4\linewidth]{../../Pictures/DistributionSingleWorker.png}
%%      \end{tabular}\\
%%%\includegraphics[height=12em]{Straggler.png}
%%
%%The same experiment has been run on 2 different days, on 2 different hours,
%%on different workers, and has returned the same pattern.
%%Therefore, we can deduce 2 points :
%%
%%\begin{itemize}
%%\item \textit{The 3 modes distribution} : 90\% of the runs are completed between
%%390 and 500 seconds. For the runs completed in this interval, we observe 3 different
%%modes of our empirical distribution. To investigate whether this is due to heterogeneity
%%in the hardware or to software reasons, we launch a long run on a single machine : 100 iterations
%%of 7 minutes each, to determine what was the behaviour of a single machine. Therefore, the 3 modes
%%may be due to hardware heterogeneity or multiple VM hosted on the same physical machine.
%%\item \textit{A worker can be affected by temporary slowness} : The three
%%longest runs (823 seconds, 632 seconds, 778 seconds) have been performed by the
%%same VM, which has also performed very well on other iterations :
%%(360 seconds, 364 seconds, ...). This could be explained by very different reasons,
%%such as the fact the physical machine hosting our VM has been hosting temporarily
%%another VM, a temporary downsizing of the size
%%of cache memory for our VM, ...
%%\end{itemize}
%%
%%Straggler issues have already been pointed out, for example in the original MapReduce
%%article \cite{MapReduce2004} by Google, yet they were observed while running thousands of
%%machines. We show that straggler issues are also observed on
%%as such a small pool of workers as 100 VM. \cite{MapReduce2004} describes a monitoring framework to detect tasks taking too much
%%time, and use backup workers to relaunch tasks
%%that has been detected to be too long. Yet, this approach leads to wait for the standard
%%duration of the task before detecting straggler tasks and launching again the corresponding
%%jobs. Therefore, for a given number of PU P, we cannot achieve better speed-up than P/2 using
%%\cite{MapReduce2004} approach on straggler issues.
%%
%%As pointed out by Graphlab \cite{Graphlab2010}, the ease of access of MapReduce frameworks
%%and the great help it provides to design scalable applications has driven part of the
%%Machine Learning community to think their algorithms to fit in a MapReduce framework. GraphLab
%%shows MapReduce is sometimes not a suitable framework for some specific Machine Learning
%%algorithm because of its lack of expressivity. The way the original MapReduce framework deals with
%%staggler issues is very well suited for very large batch jobs, as crawling the web, yet it
%%might not be the most efficient framework for machine learning on 100 PU.
%
%\section{Communication benchmark}
%
%We fixed K=1000, D = 1000 (so each version of the prototypes is 8MB big) and I =10. For different
%values of P, we run our clustering, replacing the processing part by waiting
%a fix period of time (15 seconds), so we could record communication time without being affected
%by straggler issues, as reported above. The following table provides wall time of the fake clustering
%(for 10 iterations), and the amount of time spent in communication (Wall Time - 10*15 seconds).
%
%\begin{center}\scriptsize
%\begin{tabular}{c|c|c|c|c|c|c|c|c|c|c|c|c|c|c}
%P &5&10&20&30&40&50&60&70&80&90&100&110&120&130\\
%\hline
%Wall Time & 287 & 300 & 335 & 359 & 392 & 421& 434 & 468 & 479 & 509 & 533 & 697 & 591 & 620\\
%\hline
%Communication & 137 & 150 & 185 & 209 & 242 & 271& 284 & 318 & 329 & 359 & 383 & 547 & 441 & 470\\
%\end{tabular}
%\end{center}
%
%One can see communication is not free, especially for small values of P. Yet, communication costs does not
%move much as the number of worker grows, since we design our communication algorithm to limit this. Most of
%communication costs are due to aggregated read bandwidth boundaries while all workers are trying to read one copy
%of the shared prototypes at the same time, the final reducer write bandwidth boundary (this worker has to push
%into the storage $\sqrt{P}$ copies of the shared prototypes in parallel), queues latencies, and some blob temporary unavailability
%(when we stress the storage, a given blob already pushed can take as much as 2 minutes 30 before being available to download).

%\section{Scale-up results}
%
%In a previous work to be published, we show that for given values of (N, K, D, I),
%the implementation proposed above had a computation cost of $\frac{3NDKF}{P}$ per
%iteration, and a communication cost of $2\sqrt{P}KDB$ per iteration. We therefore
%deduce that the optimal number of workers was $P^*=(3NF/B)^{2/3}$, and that 2 third
%of the wall time was spent in communication, and one third in computation, when using
%$P^*$ workers.\\
%
%Instead of trying to optimize P for a given value of (N, D, K) we set the
%number of workers to P, and try to provide the best possible speedup, namely P. In
%the following experiments, we set K=1000, D=1000, and set N to 50 000*P. Thus,
%each worker will be working on 50 000 points, which represent an important part
%of the RAM of the virtual machine (VM) (around 400 MB), helping to achieve best possible speed-ups by
%giving heavy computation loads on each worker. The algorithm is run for 10 iterations
%to get stable timing estimates (more iterations would be run on a real K-Means on such a heavy dataset).
%
%The following charts report scaleup and speedUp...
%
%Nevertheless, the obtained performances are very satisfactory for a reasonable
%number of mapping processing units (around 60 for this data size). While there
%is room for improving our implementation, the latency issues might prevent
%resorting on a tree like $O(log(P))$ reducer as available in MPI. Without
%native high performances API, communication aspects will probably remain a
%major concern in cloud implementations.

