forked from rucsgss/thesis
-
Notifications
You must be signed in to change notification settings - Fork 1
/
fsa-git.tex
executable file
·123 lines (105 loc) · 7.2 KB
/
fsa-git.tex
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
\section{Application Level Read-Aging: Git}\label{sec:fsa-git}
To measure aging in the ``real-world,'' we create a workload designed to
simulate a developer using git to work on a collaborative project.
Git is a distributed version control system that enables collaborating
developers to synchronize their source code changes. Git users \defn{pull}
changes from other developers, which then get merged with their own changes.
In a typical workload, a Git user may perform pulls multiple times per day over
several years in a long-running project. Git can synchronize all types of file
system changes, so performing a Git pull may result in the creation of new
source files, deletion of old files, file renames, and file modifications. Git
also maintains its own internal data structures, which it updates during pulls.
Thus, Git performs many operations which are similar to those shown in
\cref{sec:fsa-microbenchmarks} that cause file system aging.
We present a git benchmark that performs 10,000 pulls from the Linux git repository, starting
from the initial commit. After every 100 pulls, the benchmark performs a recursive grep
test and computes the file system's dynamic layout score.
This score is compared to the same contents copied to a freshly formatted partition.
\input{fsa-git-main-plots}
On a hard disk (\cref{fig:git-hdd}), there is a clear aging trend in all
file systems except \betrfs. By the end of the experiment, all the file
systems except \betrfs show performance drops under aging on the order of at
least 3x and as much as 15x relative to their unaged versions. All are at least
15x worse than \betrfs. The dynamic layout scores througout the benchmark are
shown in \cref{fig:git-layout}. In all of the experiments in this section,
\ftwofs ages considerably more than all other file systems, commensurate with
significantly lower layout scores than the other file systems---indicating less
effective locality in data placement. The overall correlation between grep
performance and dynamic layout score is moderate, at $-0.41$.
On an SSD (\cref{fig:git-ssd}), \btrfs and \xfs show
clear signs of aging, although they converge to a fully aged configuration
after only about 1,000 pulls. While the effect is not as drastic as on HDD, in
all the traditional file systems we see slowdowns of 2x-4x over \betrfs, which
does not slow down. In fact, aged \betrfs on the HDD outperforms all the other
aged file systems on an SSD, and is close even when they are unaged. Again,
this performance decline is strongly correlated ($-0.79$) with the dynamic
layout scores.
The aged and unaged performance of \ext and \zfs are comparable, and slower
than several other file systems. We believe this is because the average file
size decreases over the course of the test, and these file systems are not as
well-tuned for small files. To test this hypothesis, we constructed synthetic
workloads similar to the interfile fragmentation microbenchmark
(\cref{sec:fsa-microbenchmarks}), but varied the file size (in the
microbenchmark it was uniformly 4KB).\cref{fig:git:filesize} shows both
the measured, average file size of the git workload (one point is one pull),
and the microbenchmark. Overall, there is a clear relationship between the
average file size and grep cost.
\input{fsa-git-filesize-plots}
The zig-zag pattern in the graphs is created by an automatic garbage collection
process in Git. Once a certain number of ``loose objects'' are created (in git
terminology), many of them are collected and compressed into a ``pack.'' At the
file system level, this corresponds to merging numerous small files into a
single large file. According to the Git manual, this process is designed to
``reduce disk space and increase performance,'' so this is an example of an
application-level attempt to mitigate file system aging. If we turn off the git
garbage collection, as show in
\cref{fig:git:results:hdd:20gb:off,fig:git:results:ssd:20gb:off,fig:git:layout:20gb:off},
the effect of aging is even more pronounced, and the zig-zags essentially
disappear.
On both the HDD and SSD, the same patterns emerge as with garbage collection
on, but exacerbated: \ftwofs aging is by far the most extreme. \zfs ages
considerably on the HDD, but not on the SSD. \zfs on SSD and \ext perform
worse than the other file systems (except \ftwofs aged), but do not age
particularly. \xfs and \btrfs both aged significantly, around 2x each, and
\betrfs has strong, level performance in both states. This performance
correlates with dynamic layout score both on SSD ($-0.78$) and moderately so on
HDD ($-0.54$).
We note that this analysis, both of the microbenchmarks and of the git
workload, runs counter to the commonly held belief that locality is solely a
hard drive issue. While the random read performance of solid state drives does
somewhat mitigate the aging effects, aging clearly has a major performance
impact.
\paragraph{Git Workload with Warm Cache.} The tests we have presented so far
have all been performed with a cold cache, so that they more or less directly
test the performance of the file systems' on-disk layout under various aging
conditions. In practice, however, some data will be in cache, and so it is
natural to ask how much the layout choices that the file system makes will
affect the overall performance with a warm cache.
We evaluate the sensitivity of the git workloads to varying amounts of system
RAM. We use the same procedure as above, except that we do not flush any
caches or remount the hard drive between iterations. This test is performed on
a hard drive with git garbage collection off. The size of the data on disk is
initially about 280MiB and grows throughout the test to approximately 1GiB.
\input{fsa-git-wc-plots}
The results are summarized in \cref{fig:git:warmcache}. We present data for \ext
and \ftwofs; the results for \btrfs, \xfs and \zfs are similar. \betrfs is a
research prototype and unstable under memory pressure; although we plan to fix
these issues in the future, we omit this comparison.
In general, when the caches are warm and there is sufficient memory to keep all
the data in cache, then the read is very fast. However, as soon as there is no
longer sufficient memory, the performance of the aged file system with a warm
cache is generally worse than unaged with a cold cache. In general, unless all
data fits into DRAM, a good layout matters more than a having a warm cache.
\paragraph{Git Workload on BTRFS with Different Node Sizes}
We present the git test with a 4KiB node size, the default setting, as well as
8KiB, 16KiB, 32KiB, and 64KiB (the maximum).
\cref{fig:btrfsGrepTime:results:hdd:20gb:off} shows similar performance graphs
to \cref{fig:git-hdd}, one line for each node size. The 4KiB node size has
the worst read performance by the end of the test, and the performance
consistently improves as we increase the node size all the way to 64KiB.
\cref{fig:btrfsBlocksWritten:results:hdd:20gb:off} plots the number of 4KiB
blocks written to disk between each test (within the 100 pulls). As expected,
the 64KiB node size writes the maximum number of blocks and the 4KiB node
writes the least. We thus demonstrate---as predicted by our model---that aging
is reduced by a larger block size, but at the cost of write-amplification.
\input{fsa-git-node_size-plots}