|
From: | Joe Landman |
Subject: | Re: [Gluster-devel] Mirrored GlusterFS -- very poor read performance |
Date: | Sat, 04 Jan 2014 15:08:42 -0500 |
User-agent: | Mozilla/5.0 (X11; Linux x86_64; rv:24.0) Gecko/20100101 Thunderbird/24.2.0 |
On 01/03/2014 01:59 AM, Mikhail T. wrote:
[Please, CC replies to me directly as I am not subscribed to the list. Thank you.] Joe Landman wrote:As mentioned above, four test-files were used for the benchmark: 1. Small static file - 429 bytes 2. Larger static file - 93347 bytes 3. Small PHP file (a single php call in it – to phpinfo() function). Although the file is small, its output was over 64Kb. 4. Large PHP file (apc.php). Although the file is larger, its output was only about 12Kb. One of the questions I routinely ask our customers is what their definitions of "small" and "large" are, as their definitions might not match what I use for these terms. This is directly relevant in your case. What you call "larger" is considered small for GlusterFS. You want MB sized IOs to amortize the cost of the fuse system calls.But these files are representative of what our web-servers will be serving -- primarily. Though there may be an occasional video, mostly it is static HTML and "web-size" images, plus PHP-scripts... Are you saying, GlusterFS is not really for us? We expected to pay /some/ performance penalty for the features, but the actual numbers are causing a sticker-shock...
Possibly ... It has a range of use cases that make sense for users. But your files fall mostly into what is considered the "small" to "tiny" scale for the system. That means you are paying relatively huge overhead costs for relatively small data transport/storage.
(Writing to GlusterFS is even more horrid -- extracting thunderbird-24.0.tar.bz2, for example, on a GlusterFS share takes almost 30 minutes, instead of seconds on local FS, but we have very few writes, and so were willing to ignore that.)
Hmmm .... this sounds like something else might be wrong. I didn't comment on your use of LVM or ext4, but it is not unlikely that one of those layers are problematic.
Did you look at the CSW number? Is it also high?Around 8K per second, if I'm reading the output of vmstat correctly.
This is not what I expected ... you are not being context switched out of performance This is about 250 microseconds per context switch ... a moderate but not heavy load.
Something else in your stack is problematic, though your files are definitely on the small side. You have a number of layers, including the LVM. LVM usually does its IO in terms of 4MB physical extents (though I believe this size to be tunable). We might suggest looking at the LVM stats from the kernel.
-mi _______________________________________________ Gluster-devel mailing list address@hidden https://lists.nongnu.org/mailman/listinfo/gluster-devel
-- Joseph Landman, Ph.D Founder and CEO Scalable Informatics, Inc. email: address@hidden web : http://scalableinformatics.com twtr : @scalableinfo phone: +1 734 786 8423 x121 cell : +1 734 612 4615
[Prev in Thread] | Current Thread | [Next in Thread] |