I was wondering if there were any performance issues with having a data
directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
anyone done this before?

On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
> I was wondering if there were any performance issues with having a data
> directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
> anyone done this before?

My understanding is that NFS is pretty poor in performance in general,
so I would expect it to be particularly bad for a DB. You might run
some (non-DB) performance tests to get a feel for how bad it might me.
(Someone once told me that NFS topped out at around 12MB/s, but I don't
know if that's really true [they were trying to sell a competitive
networked filesystem]).

In any event, you're at least limited by ethernet speeds, if not more.

--
Steve Wampler -- swampler@noao.edu
The gods that smiled on your birth are now laughing out loud.

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

On Wed, Apr 26, 2006 at 07:35:42PM -0700, Steve Wampler wrote:
> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
> > I was wondering if there were any performance issues with having a data
> > directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
> > anyone done this before?
>
> My understanding is that NFS is pretty poor in performance in general,
> so I would expect it to be particularly bad for a DB. You might run
> some (non-DB) performance tests to get a feel for how bad it might me.
> (Someone once told me that NFS topped out at around 12MB/s, but I don't
> know if that's really true [they were trying to sell a competitive
> networked filesystem]).
>
> In any event, you're at least limited by ethernet speeds, if not more.

More importantly, the latency involved will kill commit performance. If
it doesn't then it's likely that fsync isn't being obeyed, which means 0
data integrity.
--
Jim C. Nasby, Sr. Engineering Consultant jnasby@pervasive.com
Pervasive Software http://pervasive.com work: 512-231-6117
vcard: http://jim.nasby.net/pervasive.vcf cell: 512-569-9461

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

We have gotten very good performance from netapp and postgres 7.4.11 .

I was able to push about 100MB/s over gigE, but that was limited by
our netapp.

DAS will generally always be faster, but if for example you have 2
disks vs. 100 NFS mounted ,NFS will be faster.

NFS is very reliable and I would stay away from iscsi.



Regards,
Dan Gorman

On Apr 26, 2006, at 7:35 PM, Steve Wampler wrote:

> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
>> I was wondering if there were any performance issues with having a
>> data
>> directory that was an nfs mounted drive? Say like a SAN or NAS
>> device? Has
>> anyone done this before?
>
> My understanding is that NFS is pretty poor in performance in general,
> so I would expect it to be particularly bad for a DB. You might run
> some (non-DB) performance tests to get a feel for how bad it might me.
> (Someone once told me that NFS topped out at around 12MB/s, but I
> don't
> know if that's really true [they were trying to sell a competitive
> networked filesystem]).
>
> In any event, you're at least limited by ethernet speeds, if not more.
>
> --
> Steve Wampler -- swampler@noao.edu
> The gods that smiled on your birth are now laughing out loud.
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 2: Don't 'kill -9' the postmaster



---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

I am looking for the best solution to have a large amount of disk storage
attached to my PostgreSQL 8.1 server. I was thinking of having a san or nas
attached device be mounted by the pg server over nfs, hence the question
about nfs performance. What other options/protocols are there to get high
performance and data integrity while having the benefit of not having the
physical storage attached to the db server?


On 4/27/06 12:55 AM, "Jim C. Nasby" <jnasby@pervasive.com> wrote:

> On Wed, Apr 26, 2006 at 07:35:42PM -0700, Steve Wampler wrote:
>> On Wed, Apr 26, 2006 at 10:06:58PM -0400, Ketema Harris wrote:
>>> I was wondering if there were any performance issues with having a data
>>> directory that was an nfs mounted drive? Say like a SAN or NAS device? Has
>>> anyone done this before?
>>
>> My understanding is that NFS is pretty poor in performance in general,
>> so I would expect it to be particularly bad for a DB. You might run
>> some (non-DB) performance tests to get a feel for how bad it might me.
>> (Someone once told me that NFS topped out at around 12MB/s, but I don't
>> know if that's really true [they were trying to sell a competitive
>> networked filesystem]).
>>
>> In any event, you're at least limited by ethernet speeds, if not more.
>
> More importantly, the latency involved will kill commit performance. If
> it doesn't then it's likely that fsync isn't being obeyed, which means 0
> data integrity.



---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

On Thu, Apr 27, 2006 at 08:38:55AM -0400, Ketema Harris wrote:
>I am looking for the best solution to have a large amount of disk storage
>attached to my PostgreSQL 8.1 server.

>What other options/protocols are there to get high performance and data
>integrity while having the benefit of not having the physical storage
>attached to the db server?

These are two distinct requirements. Are both really requirements or is
one "nice to have"? The "best" solution for "a large amount of disk
storage" isn't "not having the physical storage attached to the db
server". If you use non-local storage it will be slower and more
expensive, quite likely by a large margin. There may be other advantages
to doing so, but you haven't mentioned any of those as requirements.

Mike Stone

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

On Thu, Apr 27, 2006 at 08:57:51 -0400,
Ketema Harris <ketema@gmail.com> wrote:
> performance from the db. I also would hopefully then not have to do
> periodic backups from the db server to some other type of storage. Is this
> not a good idea? How bad of a performance hit are we talking about? Also,

You always need to do backups if you care about your data. What if someone
accidental deletes a lot of data? What if someone blows up your data
center (or there is a flood)?

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Yes, your right, I meant not have to do the backups from the db server
itself. I can do that within the storage device now, by allocating space
for it, and letting the device copy the data files on some periodic basis.


On 4/27/06 9:05 AM, "Bruno Wolff III" <bruno@wolff.to> wrote:

> On Thu, Apr 27, 2006 at 08:57:51 -0400,
> Ketema Harris <ketema@gmail.com> wrote:
>> performance from the db. I also would hopefully then not have to do
>> periodic backups from the db server to some other type of storage. Is this
>> not a good idea? How bad of a performance hit are we talking about? Also,
>
> You always need to do backups if you care about your data. What if someone
> accidental deletes a lot of data? What if someone blows up your data
> center (or there is a flood)?



---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

On Thu, Apr 27, 2006 at 08:57:51AM -0400, Ketema Harris wrote:
> OK. My thought process was that having non local storage as say a big raid
> 5 san ( I am talking 5 TB with expansion capability up to 10 ) would allow
> me to have redundancy, expandability, and hopefully still retain decent
> performance from the db. I also would hopefully then not have to do
> periodic backups from the db server to some other type of storage. Is this
> not a good idea? How bad of a performance hit are we talking about? Also,
> in regards to the commit data integrity, as far as the db is concerned once
> the data is sent to the san or nas isn't it "written"? The storage may have
> that write in cache, but from my reading and understanding of how these
> various storage devices work that is how they keep up performance. I would
> expect my bottleneck if any to be the actual Ethernet transfer to the
> storage, and I am going to try and compensate for that with a full gigabit
> backbone.

Well, if you have to have both the best performance and remote attach
storage, I think you'll find that a fibre-channel SAN is still the king
of the hill. 4Gb FC switches are common now, though finding a 4Gb
HBA for your computer might be a trick. 2Gb HBAs are everywhere in
FC land. That's a premium price solution, however, and I don't know
anything about how well PG would perform with a FC SAN. We use our
SAN for bulk science data and leave the PGDB on a separate machine
with local disk.

--
Steve Wampler -- swampler@noao.edu
The gods that smiled on your birth are now laughing out loud.

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On Thu, Apr 27, 2006 at 08:57:51AM -0400, Ketema Harris wrote:
>OK. My thought process was that having non local storage as say a big raid
>5 san ( I am talking 5 TB with expansion capability up to 10 )

That's two disk trays for a cheap slow array. (Versus a more expensive
solution with more spindles and better seek performance.)

>would allow
>me to have redundancy, expandability, and hopefully still retain decent
>performance from the db. I also would hopefully then not have to do
>periodic backups from the db server to some other type of storage.

No, backups are completely unrelated to your storage type; you need them
either way. On a SAN you can use a SAN backup solution to back multiple
systems up with a single backup unit without involving the host CPUs.
This is fairly useless if you aren't amortizing the cost over a large
environment.

>Is this not a good idea?

It really depends on what you're hoping to get. As described, it's not
clear. (I don't know what you mean by "redundancy, expandability" or
"decent performance".)

>How bad of a performance hit are we talking about?

Way too many factors for an easy answer. Consider the case of NAS vs
SCSI direct attach storage. You're probably in that case comparing a
single 125MB/s (peak) gigabit ethernet channel to (potentially several)
320MB/s (peak) SCSI channels. With a high-end NAS you might get 120MB/s
off that GBE. With a (more realistic) mid-range unit you're more likely
to get 40-60MB/s. Getting 200MB/s off the SCSI channel isn't a stretch,
and you can fairly easily stripe across multiple SCSI channels. (You can
also bond multiple GBEs, but then your cost & complexity start going way
up, and you're never going to scale as well.) If you have an environment
where you're doing a lot of sequential scans it isn't even a contest.
You can also substitute SATA for SCSI, etc.

For a FC SAN the peformance numbers are a lot better, but the costs &
complexity are a lot higher. An iSCSI SAN is somewhere in the middle.

>Also, in regards to the commit data integrity, as far as the db is
>concerned once the data is sent to the san or nas isn't it "written"?
>The storage may have that write in cache, but from my reading and
>understanding of how these various storage devices work that is how
>they keep up performance.

Depends on the configuration, but yes, most should be able to report
back a "write" once the data is in a non-volatile cache. You can do the
same with a direct-attached array and eliminate the latency inherent in
accessing the remote storage.

>I would expect my bottleneck if any to be the actual Ethernet transfer
>to the storage, and I am going to try and compensate for that with a
>full gigabit backbone.

see above.

The advantages of a NAS or SAN are in things you haven't really touched
on. Is the filesystem going to be accessed by several systems? Do you
need the ability to do snapshots? (You may be able to do this with
direct-attach also, but doing it on a remote storage device tends to be
simpler.) Do you want to share one big, expensive, reliable unit between
multiple systems? Will you be doing failover? (Note that failover
requires software to go with the hardware, and can be done in a
different way with local storage also.) In some environments the answers
to those questions are yes, and the price premium & performance
implications are well worth it. For a single DB server the answer is
almost certainly "no".

Mike Stone

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq