Unix Technical Forum

SAN vs Internal Disks

This is a discussion on SAN vs Internal Disks within the Pgsql Performance forums, part of the PostgreSQL category; --> On 9/6/07, Harsh Azad <harsh.azad@gmail.com> wrote: > Hi, > > How about the Dell Perc 5/i card, 512MB battery ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > Pgsql Performance

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #11 (permalink)  
Old 04-19-2008, 11:29 AM
Scott Marlowe
 
Posts: n/a
Default Re: SAN vs Internal Disks

On 9/6/07, Harsh Azad <harsh.azad@gmail.com> wrote:
> Hi,
>
> How about the Dell Perc 5/i card, 512MB battery backed cache or IBM
> ServeRAID-8k Adapter?


All Dell Percs have so far been based on either adaptec or LSI
controllers, and have ranged from really bad to fairly decent
performers. There were some recent posts on this list where someone
was benchmarking one, I believe. searching the list archives might
prove useful.

I am not at all familiar with IBM's ServeRAID controllers.

Do either of these come with or have the option for battery back
module for the cache?

> I hope I am sending relevant information here, I am not too well versed with
> RAID controllers.


Yep. Def look for a chance to evaluate whichever ones you're
considering. The Areca's are in the same price range as the IBM
controller you're considering, maybe a few hundred dollars more. See
if you can get one for review while looking at these other
controllers.

I'd recommend against Dell unless you're at a company that orders
computers by the hundred lot. My experience with Dell has been that
unless you are a big customer you're just another number (a small one
at that) on a spreadsheet.

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #12 (permalink)  
Old 04-19-2008, 11:30 AM
Joe Uhl
 
Posts: n/a
Default Re: SAN vs Internal Disks

Scott Marlowe wrote:
> On 9/6/07, Harsh Azad <harsh.azad@gmail.com> wrote:
>
>> Hi,
>>
>> How about the Dell Perc 5/i card, 512MB battery backed cache or IBM
>> ServeRAID-8k Adapter?
>>

>
> All Dell Percs have so far been based on either adaptec or LSI
> controllers, and have ranged from really bad to fairly decent
> performers. There were some recent posts on this list where someone
> was benchmarking one, I believe. searching the list archives might
> prove useful.
>
> I am not at all familiar with IBM's ServeRAID controllers.
>
> Do either of these come with or have the option for battery back
> module for the cache?
>
>
>> I hope I am sending relevant information here, I am not too well versed with
>> RAID controllers.
>>

>
> Yep. Def look for a chance to evaluate whichever ones you're
> considering. The Areca's are in the same price range as the IBM
> controller you're considering, maybe a few hundred dollars more. See
> if you can get one for review while looking at these other
> controllers.
>
> I'd recommend against Dell unless you're at a company that orders
> computers by the hundred lot. My experience with Dell has been that
> unless you are a big customer you're just another number (a small one
> at that) on a spreadsheet.
>

If you do go with Dell get connected with an account manager instead of
ordering online. You work with the same people every time you have an
order and in my experience they can noticeably beat the best prices I
can find. This is definitely the way to go if you don't want to get
lost in the volume. The group I have worked with for the past ~2 years
is very responsive, remembers me and my company across the 3 - 6 month
gaps between purchases, and the server/storage person in the group is
reasonably knowledgeable and helpful. This is for small lots of
machines, our first order was just 2 boxes and i've only placed 4 orders
total in the past 2 years.

Just my personal experience, i'd be happy to pass along the account
manager's information if anyone is interested.
> ---------------------------(end of broadcast)---------------------------
> TIP 1: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo@postgresql.org so that your
> message can get through to the mailing list cleanly
>

Joe Uhl
joeuhl@gmail.com

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #13 (permalink)  
Old 04-19-2008, 11:30 AM
Joel Fradkin
 
Posts: n/a
Default Re: SAN vs Internal Disks

I am not sure I agree with that evaluation.
I only have 2 dell database servers and they have been 100% reliable.
Maybe he is referring to support which does tend be up to who you get.
When I asked about performance on my new server they were very helpful but I
did have a bad time on my NAS device (but had the really cheap support plan
on it). They did help me get it fixed but I had to RMA all the drives on the
NAS as they were all bad and it was no fun installing the os as it had no
floppy. I got the better support for both the data base servers which are
using jbod from dell for the disk array. The quad proc opteron with duel
cores and 16gig of memory has been extremely fast (like 70%) over my older 4
proc 32 bit single core machine with 8 gig. But both are running postgres
and perform needed functionality. I would like to have redundant backups of
these as they are mission critical, but all in good time.

I'd recommend against Dell unless you're at a company that orders
computers by the hundred lot. My experience with Dell has been that
unless you are a big customer you're just another number (a small one
at that) on a spreadsheet.

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly


---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #14 (permalink)  
Old 04-19-2008, 11:30 AM
Arjen van der Meijden
 
Posts: n/a
Default Re: SAN vs Internal Disks

On 6-9-2007 20:42 Scott Marlowe wrote:
> On 9/6/07, Harsh Azad <harsh.azad@gmail.com> wrote:
>> Hi,
>>
>> How about the Dell Perc 5/i card, 512MB battery backed cache or IBM
>> ServeRAID-8k Adapter?

>
> All Dell Percs have so far been based on either adaptec or LSI
> controllers, and have ranged from really bad to fairly decent
> performers. There were some recent posts on this list where someone
> was benchmarking one, I believe. searching the list archives might
> prove useful.


The Dell PERC5-cards are based on LSI-chips and perform quite well.
Afaik Dell hasn't used adaptecs for a while now, but even recent
(non-cheap ) adaptecs aren't that bad afaik.

The disadvantage of using Areca or 3Ware is obviously the lack of
support in A-brand servers and the lack of support for SAS-disks. Only
recently Areca has stepped in the SAS-market, but I have no idea how
easily those controllers are integrated in standard servers (they tend
to be quite large, which will not fit in 2U and maybe not even in 3U or
4U-servers).

Arjen

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #15 (permalink)  
Old 04-19-2008, 11:30 AM
Arjen van der Meijden
 
Posts: n/a
Default Re: SAN vs Internal Disks

On 6-9-2007 20:29 Mark Lewis wrote:
> Maybe I'm jaded by past experiences, but the only real use case I can
> see to justify a SAN for a database would be something like Oracle RAC,
> but I'm not aware of any PG equivalent to that.


PG Cluster II seems to be able to do that, but I don't know whether
that's production quality already...

Arjen

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #16 (permalink)  
Old 04-19-2008, 11:30 AM
Scott Marlowe
 
Posts: n/a
Default Re: SAN vs Internal Disks

On 9/6/07, Joel Fradkin <jfradkin@wazagua.com> wrote:
> I am not sure I agree with that evaluation.
> I only have 2 dell database servers and they have been 100% reliable.
> Maybe he is referring to support which does tend be up to who you get.
> When I asked about performance on my new server they were very helpful but I
> did have a bad time on my NAS device (but had the really cheap support plan
> on it). They did help me get it fixed but I had to RMA all the drives on the
> NAS as they were all bad and it was no fun installing the os as it had no
> floppy. I got the better support for both the data base servers which are
> using jbod from dell for the disk array. The quad proc opteron with duel
> cores and 16gig of memory has been extremely fast (like 70%) over my older 4
> proc 32 bit single core machine with 8 gig. But both are running postgres
> and perform needed functionality. I would like to have redundant backups of
> these as they are mission critical, but all in good time.


Dell's ok if by support you mean replacing simple broken parts, etc...

I'm talking about issues like the one we had with our 26xx servers
which, thankfully, Dell hasn't made in a while. We had the adaptec
controllers that locked up once every 1 to 3 months for no reason, and
with 4 servers this meant a lockup of one every 1 to 2 weeks. Not
acceptable in a production environment. It took almost 2 years to get
Dell to agree to ship us the LSI based RAID controllers for those 4
machines. Our account rep told us not to worry about returning the
RAID controllers as they were so old as to be obsolete. One of the
four replacement controllers they sent us was bad, so they
cross-shipped another one. Sometimes you get a broken part, it
happens.

One month after we got all our RAID controllers replaced, we started
getting nasty calls from their parts people wanting those parts back,
saying they were gonna charge us for them, etc... Two thirds of the
parts were still in the server because we hadn't gotten a sheet
identifying them (the RAID key and battery) and had left in not
worrying about it because we'd been told they were needed.

These are production servers, we can't just shut them off for fun to
pull a part we were told we didn't need to return.

On top of that, the firmware updates come as an .exe that has to be
put on a windows floppy. We don't have a single windows machine with
a floppy at the company I work for (lots of laptops with windows). We
had to dig out a floppy drive and a windows CD to create a bootable
floppy to install a firmware update for a linux server.

So, my main complaint is about their customer service. The machines,
when they work, are pretty good. Normally the hardware gets along
with RH OSes. But when things go wrong, Dell will not admit to a
design problem that's staring them in the face, and they have wasted
literally hundreds of man hours for us with their stonewalling on this
subject. So, I see no reason to buy more hardware from them when
there are dealers big and small who have so far treated us much
better.

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #17 (permalink)  
Old 04-19-2008, 11:30 AM
Greg Smith
 
Posts: n/a
Default Re: SAN vs Internal Disks

On Thu, 6 Sep 2007, Harsh Azad wrote:

> Firstly, I could only find postgres 8.1.x RPM for CentOS 5, could not find
> any RPM for 8.2.4. Is there any 8.2.4 RPM for CentOS 5?


You've already been pointed in the right direction. Devrim, the person who
handles this packaging, does a great job of building all the RPMs. But I
have a small philisophical difference with the suggested instructions for
actually installing them though. I just finished an alternate
installation guide for RHEL/CentOS 5 that's now posted at
http://www.westnet.com/~gsmith/conte...esql/pgrpm.htm you may want
to take a look at.

> Secondly, would investing into Redhat enterprise edition give any
> performance advantage? I know all the SAN boxes are only certified on
> RHEL and not CentOS. Or since CentOS is similar to RHEL it would be
> fine?


Wouldn't expect a performance advantage. The situation you have to ask
consider is this: your SAN starts having funky problems, and your
database is down because of it. You call the vendor. They find out
you're running CentOS instead of RHEL and say that's the cause of your
problem (even though it probably isn't). How much will such a passing the
buck problem cost your company? If it's a significant number, you'd be
foolish to run CentOS instead of the real RHEL. Some SAN vendors can be
very, very picky about what they will support, and for most business
environments the RHEL subscription isn't so expensive that it's worth
wandering into an area where your support situation is fuzzy just to save
that money.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #18 (permalink)  
Old 04-19-2008, 11:30 AM
Florian Weimer
 
Posts: n/a
Default Re: SAN vs Internal Disks

* Arjen van der Meijden:

> The disadvantage of using Areca or 3Ware is obviously the lack of
> support in A-brand servers and the lack of support for SAS-disks. Only
> recently Areca has stepped in the SAS-market, but I have no idea how
> easily those controllers are integrated in standard servers (they tend
> to be quite large, which will not fit in 2U and maybe not even in 3U
> or 4U-servers).


Recent 3ware controllers are a bit on the hot side, too. We had to
switch from two 12 port controllers to a single 24 port controller
because of that (combined with an unlucky board layout: the two 8x
PCIe connectors are next to each other).

Unfortunately, read performance maxes out at about 8 disks in a
RAID-10 configuration. Software RAID-0 across hardware RAID-1 is
significantly faster (factor of 3 to 5 in low-level benchmarks).
However, it seems that something in this stack does not enforce write
barriers properly, so I don't think we will use this in production.

RAID-6 doesn't perform well, either (especially for several processes
reading different files sequentially).

We'll probably split the 24 disks into a couple of RAID-10s, and
distribute tables and indexes manually among the file systems. This
is a bit disappointing, especially because the system is able to read
at 800+ MB/s, as shown by the software-RAID-on-hardware-RAID
configuration.

I haven't seen 24-disk benchmarks with Areca controllers. A
comparison might be interesting.

--
Florian Weimer <fweimer@bfk.de>
BFK edv-consulting GmbH http://www.bfk.de/
Kriegsstraße 100 tel: +49-721-96201-1
D-76133 Karlsruhe fax: +49-721-96201-99

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #19 (permalink)  
Old 04-19-2008, 11:30 AM
Michael Stone
 
Posts: n/a
Default Re: SAN vs Internal Disks

On Fri, Sep 07, 2007 at 12:26:23AM -0400, Greg Smith wrote:
>consider is this: your SAN starts having funky problems, and your
>database is down because of it. You call the vendor. They find out
>you're running CentOS instead of RHEL and say that's the cause of your
>problem (even though it probably isn't). How much will such a passing the
>buck problem cost your company? If it's a significant number, you'd be
>foolish to run CentOS instead of the real RHEL. Some SAN vendors can be
>very, very picky about what they will support, and for most business
>environments the RHEL subscription isn't so expensive that it's worth
>wandering into an area where your support situation is fuzzy just to save
>that money.


Correct. Far more sensible to skip the expensive SAN solution, not worry
about having to play games, and save *even more* money.

SANs have their place, but postgres storage generally isn't it; you'll
get more bang/buck with DAS and very likely better absolute performance
as well. SANs make sense if you're doing a shared filesystem (don't
even think about doing this with postgres), or if you're consolidating
backups & DR (which doesn't work especially well with databases).

Mike Stone

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #20 (permalink)  
Old 04-19-2008, 11:30 AM
Tobias Brox
 
Posts: n/a
Default Re: SAN vs Internal Disks

We're also considering to install postgres on SAN - that is, my boss is
convinced this is the right way to go.

Advantages:

1. Higher I/O (at least the salesman claims so)
2. Easier to upgrade the disk capacity
3. Easy to set up "warm standby" functionality. (Then again, if the
postgres server fails miserably, it's likely to be due to a disk
crash).

Also, my boss states that "all big enterprises uses SAN nowadays".

Disadvantages:

1. Risky? One gets the impression that there are frequent problems
with data integrity when reading some of the posts in this thread.

2. Expensive

3. "Single point of failure" ... but that you have either it's a SAN or
a local disk, one will anyway need good backup systems (and eventually
"warm standby"-servers running from physically separated disks).

4. More complex setup?

5. If there are several hosts with write permission towards the same
disk, I can imagine the risks being higher for data integrity
breakages. Particularly, I can imagine that if two postgres instances
is started up towards the same disk (due to some sysadmin mistake), it
could be disasterous.


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 05:50 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.2.0
www.UnixAdminTalk.com