> If I'm reading the original post correctly, the biggest issue is
> likely to be that the 14 disks on each 2Gbit fibre channel will be
> throttled to 200Mb/s by the channel , when in fact you could expect
> (in RAID 10
> arrangement) to get about 7 * 70 Mb/s = 490 Mb/s.

> The two controllers and two FC switches/hubs are intended for
redundancy, rather than performance, so there's only one 2Gbit channel.
I > don't know if its possible to use both in parallel to get better
performance.

> I believe it's possible to join two or more FC ports on the switch
together, but as there's only port going to the controller internally
this presumably wouldn't help.

> There are two SCSI U320 buses, with seven bays on each. I don't know
what the overhead of SCSI is, but you're obviously not going to get >
490MB/s for each set of seven even if the FC could do it.


Darn. I was really looking forward to ~500Mb/s


> Of course your database may not spend all day doing sequential scans
one at a time over 14 disks, so it doesn't necessarily matter...


That's probably true, but *knowing* that the max seq scan speed is that
high gives you some confidence (true or fake) that the hardware will be
sufficient the next 2 years or so. So, if dual 2GBit FC:s still don't
deliver more than 200Mb/s, what does?

-Mikael

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Mikael Carneholm wrote:

>
>> There are two SCSI U320 buses, with seven bays on each. I don't know
> what the overhead of SCSI is, but you're obviously not going to get >
> 490MB/s for each set of seven even if the FC could do it.
>

You should be able to get close to 300Mb/s on each SCSI bus - provided
the PCI bus on the motherboard is 64-bit and runs at 133Mhz or better
(64-bit and 66Mhz give you a 524Mb/s limit).

>
>> Of course your database may not spend all day doing sequential scans
> one at a time over 14 disks, so it doesn't necessarily matter...
>

Yeah, it depends on the intended workload, but at some point most
databases end up IO bound... so you really want to ensure the IO system
is as capable as possible IMHO.

>
> That's probably true, but *knowing* that the max seq scan speed is that
> high gives you some confidence (true or fake) that the hardware will be
> sufficient the next 2 years or so. So, if dual 2GBit FC:s still don't
> deliver more than 200Mb/s, what does?
>

Most modern PCI-X or PCIe RAID cards will do better than 200Mb/s (e.g.
3Ware 9550SX will do ~800Mb/s).

By way of comparison my old PIII with a Promise TX4000 plus 4 IDE drives
will do 215Mb/s...so being throttled to 200Mb/s on modern hardware seems
unwise to me.

Cheers

Mark

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

On Tue, 25 Apr 2006, Mark Kirkwood wrote:

> Mikael Carneholm wrote:
>
> >
> >> There are two SCSI U320 buses, with seven bays on each. I don't know
> > what the overhead of SCSI is, but you're obviously not going to get >
> > 490MB/s for each set of seven even if the FC could do it.
> >
>
> You should be able to get close to 300Mb/s on each SCSI bus - provided
> the PCI bus on the motherboard is 64-bit and runs at 133Mhz or better
> (64-bit and 66Mhz give you a 524Mb/s limit).

I've no idea if the MSA1500's controllers use PCI internally. Obviously
this argument applies to the PCI bus you plug your FC adapters in to,
though.

AIUI it's difficult to get PCI to actually give you it's theoretical
maximum bandwidth. Those speeds are still a lot more than 200MB/s, though.

> >> Of course your database may not spend all day doing sequential scans
> > one at a time over 14 disks, so it doesn't necessarily matter...
> >
>
> Yeah, it depends on the intended workload, but at some point most
> databases end up IO bound... so you really want to ensure the IO system
> is as capable as possible IMHO.

IO bound doesn't imply IO bandwidth bound. 14 disks doing a 1ms seek
followed by an 8k read over and over again is a bit over 100MB/s. Adding
in write activity would make a difference, too, since it'd have to go to
at least two disks. There are presumably hot spares, too.

I still wouldn't really want to be limited to 200MB/s if I expected to use
a full set of 14 disks for active database data where utmost performance
really matters and where there may be some sequential scans going on,
though.

> > That's probably true, but *knowing* that the max seq scan speed is that
> > high gives you some confidence (true or fake) that the hardware will be
> > sufficient the next 2 years or so. So, if dual 2GBit FC:s still don't
> > deliver more than 200Mb/s, what does?
> >
>
> Most modern PCI-X or PCIe RAID cards will do better than 200Mb/s (e.g.
> 3Ware 9550SX will do ~800Mb/s).
>
> By way of comparison my old PIII with a Promise TX4000 plus 4 IDE drives
> will do 215Mb/s...so being throttled to 200Mb/s on modern hardware seems
> unwise to me.

Though, of course, these won't do many of the things you can do with a SAN
- like connect several computers, or split a single array in to two pieces
and have two computers access them as if they were separate drives, or
remotely shut down one database machine and then start up another using
the same disks and data. The number of IO operations per second they can
do is likely to be important, too...possibly more important.

There's 4GB FC, and so presumably 4GB SANs, but that's still not vast
bandwidth. Using multiple FC ports is the other obvious way to do it with
a SAN. I haven't looked, but I suspect you'll need quite a budget to get
that...

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Alex Hayward wrote:

>
> IO bound doesn't imply IO bandwidth bound. 14 disks doing a 1ms seek
> followed by an 8k read over and over again is a bit over 100MB/s. Adding
> in write activity would make a difference, too, since it'd have to go to
> at least two disks. There are presumably hot spares, too.
>

Very true - if your workload is primarily random, ~100Mb/s may be enough
bandwidth.

> I still wouldn't really want to be limited to 200MB/s if I expected to use
> a full set of 14 disks for active database data where utmost performance
> really matters and where there may be some sequential scans going on,
> though.
>

Yeah - thats the rub, Data mining, bulk loads, batch updates, backups
(restores....) often use significant bandwidth.

> Though, of course, these won't do many of the things you can do with a SAN
> - like connect several computers, or split a single array in to two pieces
> and have two computers access them as if they were separate drives, or
> remotely shut down one database machine and then start up another using
> the same disks and data. The number of IO operations per second they can
> do is likely to be important, too...possibly more important.
>

SAN flexibility is nice (when it works as advertised), the cost and
performance however, are the main detractors. On that note I don't
recall IO/s being anything special on most SAN gear I've seen (this
could have changed for later products I guess).

> There's 4GB FC, and so presumably 4GB SANs, but that's still not vast
> bandwidth. Using multiple FC ports is the other obvious way to do it with
> a SAN. I haven't looked, but I suspect you'll need quite a budget to get
> that...
>

Yes - the last place I worked were looking at doing this ('multiple
attachment' was the buzz word I think) - I recall it needed special
(read extra expensive) switches and particular cards...

Cheers

Mark


---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings