I have a couple of users that want to move their applications to our
new pSeries 575 cluster and I need to figure out which storage system
to use. We currently have an NFS server and a GPFS filesystem.

1) MySQL database
2) Flat ascii files of tcpdump results

Sorry I cannot give more information at this time, because I don't have
it. But due to internal budget cycles I need to make a guess on what
is best (more GFPS, more NFS, and/or MySQL server with local disk).

My real question is, it is most likely that both of these applications
are metadata intensive. If so, should I avoid using GPFS for them?

Can anyone make an educated guess on the appropriateness of GFPS?

Thanks,
Craig

Why do you want to avoid using GPFS?

ctierney42 wrote:
> I have a couple of users that want to move their applications to our
> new pSeries 575 cluster and I need to figure out which storage system
> to use. We currently have an NFS server and a GPFS filesystem.
>
> 1) MySQL database
> 2) Flat ascii files of tcpdump results
>
> Sorry I cannot give more information at this time, because I don't have
> it. But due to internal budget cycles I need to make a guess on what
> is best (more GFPS, more NFS, and/or MySQL server with local disk).
>
> My real question is, it is most likely that both of these applications
> are metadata intensive. If so, should I avoid using GPFS for them?
>
> Can anyone make an educated guess on the appropriateness of GFPS?
>
> Thanks,
> Craig

Blade wrote:
> Why do you want to avoid using GPFS?
>

I am happy to be corrected on this, but GPFS (formally multi-media
filesystem)
is designed to support large-block transfers. It isn't designed for
high-speed
metadata performance (databases). I have access to the system now, but
they way funding works, they are going to have to decide on hardware to
purchase before I can do some real testing.

I will run some tests this week to help me decide, but I don't have
time to
install mysql and run tests.

Thanks,
Craig



> ctierney42 wrote:
> > I have a couple of users that want to move their applications to our
> > new pSeries 575 cluster and I need to figure out which storage system
> > to use. We currently have an NFS server and a GPFS filesystem.
> >
> > 1) MySQL database
> > 2) Flat ascii files of tcpdump results
> >
> > Sorry I cannot give more information at this time, because I don't have
> > it. But due to internal budget cycles I need to make a guess on what
> > is best (more GFPS, more NFS, and/or MySQL server with local disk).
> >
> > My real question is, it is most likely that both of these applications
> > are metadata intensive. If so, should I avoid using GPFS for them?
> >
> > Can anyone make an educated guess on the appropriateness of GFPS?
> >
> > Thanks,
> > Craig

ctierney42 wrote:
> Blade wrote:
> > Why do you want to avoid using GPFS?
> >
>
> I am happy to be corrected on this, but GPFS (formally multi-media
> filesystem)
> is designed to support large-block transfers. It isn't designed for
> high-speed
> metadata performance (databases). I have access to the system now, but
> they way funding works, they are going to have to decide on hardware to
> purchase before I can do some real testing.
>
> I will run some tests this week to help me decide, but I don't have
> time to
> install mysql and run tests.
>
> Thanks,
> Craig
>
>
>
> > ctierney42 wrote:
> > > I have a couple of users that want to move their applications to our
> > > new pSeries 575 cluster and I need to figure out which storage system
> > > to use. We currently have an NFS server and a GPFS filesystem.
> > >
> > > 1) MySQL database
> > > 2) Flat ascii files of tcpdump results
> > >
> > > Sorry I cannot give more information at this time, because I don't have
> > > it. But due to internal budget cycles I need to make a guess on what
> > > is best (more GFPS, more NFS, and/or MySQL server with local disk).
> > >
> > > My real question is, it is most likely that both of these applications
> > > are metadata intensive. If so, should I avoid using GPFS for them?
> > >
> > > Can anyone make an educated guess on the appropriateness of GFPS?
> > >
> > > Thanks,
> > > Craig

Without having done any testing, and without any knowledge of your
applications, I would **guess** that gpfs would perform better than nfs
in most cases since the disk is locally attached - depending on the
network and method of attachment.

Have no experience of running MySQL on GPFS...however, running GPFS is
one of the options to run Oracle RAC recommended by both IBM and
Oracle. Also, GPFS is designed for high performance (according to IBM's
doc). Not done any tunning yet...sorry for not having more to share.


ctierney42 wrote:
> Blade wrote:
> > Why do you want to avoid using GPFS?
> >
>
> I am happy to be corrected on this, but GPFS (formally multi-media
> filesystem)
> is designed to support large-block transfers. It isn't designed for
> high-speed
> metadata performance (databases). I have access to the system now, but
> they way funding works, they are going to have to decide on hardware to
> purchase before I can do some real testing.
>
> I will run some tests this week to help me decide, but I don't have
> time to
> install mysql and run tests.
>
> Thanks,
> Craig
>
>
>
> > ctierney42 wrote:
> > > I have a couple of users that want to move their applications to our
> > > new pSeries 575 cluster and I need to figure out which storage system
> > > to use. We currently have an NFS server and a GPFS filesystem.
> > >
> > > 1) MySQL database
> > > 2) Flat ascii files of tcpdump results
> > >
> > > Sorry I cannot give more information at this time, because I don't have
> > > it. But due to internal budget cycles I need to make a guess on what
> > > is best (more GFPS, more NFS, and/or MySQL server with local disk).
> > >
> > > My real question is, it is most likely that both of these applications
> > > are metadata intensive. If so, should I avoid using GPFS for them?
> > >
> > > Can anyone make an educated guess on the appropriateness of GFPS?
> > >
> > > Thanks,
> > > Craig

aix@mail.com wrote:
> ctierney42 wrote:
> > Blade wrote:
> > > Why do you want to avoid using GPFS?
> > >
> >
> > I am happy to be corrected on this, but GPFS (formally multi-media
> > filesystem)
> > is designed to support large-block transfers. It isn't designed for
> > high-speed
> > metadata performance (databases). I have access to the system now, but
> > they way funding works, they are going to have to decide on hardware to
> > purchase before I can do some real testing.
> >
> > I will run some tests this week to help me decide, but I don't have
> > time to
> > install mysql and run tests.
> >
> > Thanks,
> > Craig
> >
> >
> >
> > > ctierney42 wrote:
> > > > I have a couple of users that want to move their applications to our
> > > > new pSeries 575 cluster and I need to figure out which storage system
> > > > to use. We currently have an NFS server and a GPFS filesystem.
> > > >
> > > > 1) MySQL database
> > > > 2) Flat ascii files of tcpdump results
> > > >
> > > > Sorry I cannot give more information at this time, because I don't have
> > > > it. But due to internal budget cycles I need to make a guess on what
> > > > is best (more GFPS, more NFS, and/or MySQL server with local disk).
> > > >
> > > > My real question is, it is most likely that both of these applications
> > > > are metadata intensive. If so, should I avoid using GPFS for them?
> > > >
> > > > Can anyone make an educated guess on the appropriateness of GFPS?
> > > >
> > > > Thanks,
> > > > Craig
>
> Without having done any testing, and without any knowledge of your
> applications, I would **guess** that gpfs would perform better than nfs
> in most cases since the disk is locally attached - depending on the
> network and method of attachment.

You are probably correct here. NFS won't be a marked improvement if
at all. I have some time today and I downloaded MySQL and will run
their benchmark
over GPFS and local /tmp (15k SCSI) and compare the difference.

I was talking to someone who knows more than I about databases, and
there could be an other option. Build a database server with
15K disks and but the database on the local filesystem. Access the
database over the network. The latency of the disk (a few ms) is much
worse than the network (gigE 30 micorsec one way), and may also
be an option.

- Craig