Unix Technical Forum

shared_buffers > 284263 on OS X

This is a discussion on shared_buffers > 284263 on OS X within the Pgsql Performance forums, part of the PostgreSQL category; --> On 27-Nov-06, at 4:04 AM, Tom Lane wrote: > Brendan Duddridge <brendan@clickspace.com> writes: >> I think the main issue ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > Pgsql Performance

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #11 (permalink)  
Old 04-19-2008, 09:47 AM
Brian Wipf
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X

On 27-Nov-06, at 4:04 AM, Tom Lane wrote:
> Brendan Duddridge <brendan@clickspace.com> writes:
>> I think the main issue is that we can't seem to get PostgreSQL
>> compiled for 64 bit on OS X on an Xserve G5. Has anyone done that?

>
> There is no obvious reason why it would not work, and if anyone has
> tried and failed, they've not bothered to provide details on any PG
> list
> I read ...


I'll post details of the problems I've had compiling for 64-bit on OS
X Tiger to the pgsql-ports when I get a chance.

Brian Wipf


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #12 (permalink)  
Old 04-19-2008, 09:47 AM
Brendan Duddridge
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X

I think the main issue is that we can't seem to get PostgreSQL
compiled for 64 bit on OS X on an Xserve G5. Has anyone done that?

We have 8 GB of RAM on that server, but we can't seem to utilize it
all. At least not for the shared_buffers setting.

Thanks,

__________________________________________________ __________________
Brendan Duddridge | CTO | 403-277-5591 x24 | brendan@clickspace.com

ClickSpace Interactive Inc.
Suite L100, 239 - 10th Ave. SE
Calgary, AB T2G 0V9

http://www.clickspace.com

On Nov 26, 2006, at 4:25 PM, Jim C. Nasby wrote:

> On Sat, Nov 18, 2006 at 08:13:26PM -0700, Brian Wipf wrote:
>> It certainly is unfortunate if Guido's right and this is an upper
>> limit for OS X. The performance benefit of having high shared_buffers
>> on our mostly read database is remarkable.

>
> Got any data about that you can share? People have been wondering
> about
> cases where drastically increasing shared_buffers makes a difference.
> --
> Jim Nasby jim@nasby.net
> EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 6: explain analyze is your friend
>




---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #13 (permalink)  
Old 04-19-2008, 09:47 AM
Guido Neitzer
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X

Am 27.11.2006 um 04:20 schrieb Brendan Duddridge:

> I think the main issue is that we can't seem to get PostgreSQL
> compiled for 64 bit on OS X on an Xserve G5. Has anyone done that?
>
> We have 8 GB of RAM on that server, but we can't seem to utilize it
> all. At least not for the shared_buffers setting.


One VERY ugly idea is: if you have your stuff in more than one db,
let two PostgreSQL installations run on the same machine and put some
databases on one and others on the second installation (on different
ports and different data directories of course) and give either one
the 2GB shared mem you like. So you can use the 50% of the available
RAM.

I don't know whether Mac OS X itself is able to handle a larger
amount of shared memory but I believe it can.

But nevertheless this is only a very ugly workaround on a problem
that shouldn't exist. The correct way would be to get a 64 Bit binary
of PostgreSQL - which I wasn't able to create.

But, be aware of another thing here: As far as I have read about 64
Bit applications on G5, these apps are definitely slower than their
32 bit counterparts (I'm currently on the train so I can't be more
precise here without Google ...). Was it something with not enough
registers in the CPU? Something like that ... So it might be, that
the 64 bit version is able to use more shared memory but is slower
than the 32 bit version and you come out with the same performance.
Nobody knows ...

cug

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #14 (permalink)  
Old 04-19-2008, 09:47 AM
Guido Neitzer
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X

Am 27.11.2006 um 00:25 schrieb Jim C. Nasby:

> Got any data about that you can share? People have been wondering
> about
> cases where drastically increasing shared_buffers makes a difference.


I have tried to compile PostgreSQL as a 64Bit application on my G5
but wasn't successful. But I must admit, that I'm not a C programmer
at all. I know enough to work with source packages and configure /
make but not enough to work with the errors I got from the compile.
And as I'm extremely busy right now, I can't follow the trail and
learn more about it.

Perhaps someone with more knowledge can take a look at it.

cug

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #15 (permalink)  
Old 04-19-2008, 09:47 AM
Brian Wipf
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X

On 26-Nov-06, at 11:25 PM, Jim C. Nasby wrote:
> On Sat, Nov 18, 2006 at 08:13:26PM -0700, Brian Wipf wrote:
>> It certainly is unfortunate if Guido's right and this is an upper
>> limit for OS X. The performance benefit of having high shared_buffers
>> on our mostly read database is remarkable.

>
> Got any data about that you can share? People have been wondering
> about
> cases where drastically increasing shared_buffers makes a difference.


Unfortunately, there are more differences than just the
shared_buffers setting in production right now; it's a completely
different set up, so the numbers I have to compare against aren't
particularly useful.

When I get the chance, I will try to post data that shows the benefit
of having a higher value of shared_buffers for our usage pattern
(with all other settings being constant -- well, except maybe
effective_cache_size). Basically, in our current configuration, we
can cache all of the data we care about 99% of the time in about 3GB
of shared_buffers. Having shared_buffers set to 512MB as it was
originally, we were needlessly going to disk all of the time.

Brian Wipf


---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #16 (permalink)  
Old 04-19-2008, 09:47 AM
Guido Neitzer
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X

Am 27.11.2006 um 08:04 schrieb Guido Neitzer:

> But, be aware of another thing here: As far as I have read about 64
> Bit applications on G5, these apps are definitely slower than their
> 32 bit counterparts (I'm currently on the train so I can't be more
> precise here without Google ...). Was it something with not enough
> registers in the CPU? Something like that ... So it might be, that
> the 64 bit version is able to use more shared memory but is slower
> than the 32 bit version and you come out with the same performance.
> Nobody knows ...


Some information about that:

<http://www.geekpatrol.ca/2006/09/32-bit-vs-64-bit-performance/>

So, the impact doesn't seem to high. So it seems to depend on the
usage pattern whether the 32 bit with less RAM and slightly higher
performance might be faster than 64 bit with more shared memory and
slightly lower performance.

cug


---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #17 (permalink)  
Old 04-19-2008, 09:47 AM
Jim C. Nasby
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X

On Mon, Nov 27, 2006 at 07:23:47AM +0000, Brian Wipf wrote:
> On 26-Nov-06, at 11:25 PM, Jim C. Nasby wrote:
> >On Sat, Nov 18, 2006 at 08:13:26PM -0700, Brian Wipf wrote:
> >>It certainly is unfortunate if Guido's right and this is an upper
> >>limit for OS X. The performance benefit of having high shared_buffers
> >>on our mostly read database is remarkable.

> >
> >Got any data about that you can share? People have been wondering
> >about
> >cases where drastically increasing shared_buffers makes a difference.

>
> Unfortunately, there are more differences than just the
> shared_buffers setting in production right now; it's a completely
> different set up, so the numbers I have to compare against aren't
> particularly useful.
>
> When I get the chance, I will try to post data that shows the benefit
> of having a higher value of shared_buffers for our usage pattern
> (with all other settings being constant -- well, except maybe
> effective_cache_size). Basically, in our current configuration, we
> can cache all of the data we care about 99% of the time in about 3GB
> of shared_buffers. Having shared_buffers set to 512MB as it was
> originally, we were needlessly going to disk all of the time.


Disk or to the kernel cache?
--
Jim Nasby jim@nasby.net
EnterpriseDB http://enterprisedb.com 512.569.9461 (cell)

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #18 (permalink)  
Old 04-19-2008, 09:47 AM
AgentM
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X


On Nov 27, 2006, at 2:23 , Brian Wipf wrote:

> On 26-Nov-06, at 11:25 PM, Jim C. Nasby wrote:
>> On Sat, Nov 18, 2006 at 08:13:26PM -0700, Brian Wipf wrote:
>>> It certainly is unfortunate if Guido's right and this is an upper
>>> limit for OS X. The performance benefit of having high
>>> shared_buffers
>>> on our mostly read database is remarkable.

>>
>> Got any data about that you can share? People have been wondering
>> about
>> cases where drastically increasing shared_buffers makes a difference.

>
> Unfortunately, there are more differences than just the
> shared_buffers setting in production right now; it's a completely
> different set up, so the numbers I have to compare against aren't
> particularly useful.
>
> When I get the chance, I will try to post data that shows the
> benefit of having a higher value of shared_buffers for our usage
> pattern (with all other settings being constant -- well, except
> maybe effective_cache_size). Basically, in our current
> configuration, we can cache all of the data we care about 99% of
> the time in about 3GB of shared_buffers. Having shared_buffers set
> to 512MB as it was originally, we were needlessly going to disk all
> of the time.


There is a known unfortunate limitation on Darwin for SysV shared
memory which, incidentally, does not afflict POSIX or mmap'd shared
memory.

http://archives.postgresql.org/pgsql...2/msg00176.php

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #19 (permalink)  
Old 04-19-2008, 09:47 AM
Guido Neitzer
 
Posts: n/a
Default Re: shared_buffers > 284263 on OS X

Am 27.11.2006 um 17:05 schrieb AgentM:

> There is a known unfortunate limitation on Darwin for SysV shared
> memory which, incidentally, does not afflict POSIX or mmap'd shared
> memory.


Hmmm. The article from Chris you have linked does not mention the
size of the mem segment you can allocate. Nevertheless - if you
compile a 32 Bit binary, there is the limitation Brian mentioned.

You can easily simulate this with a small C program that allocates
memory - if you compile it as 64 Bit binary - not problem, if you
compile as 32 Bit - crash.

cug

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 04:03 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.2.0
www.UnixAdminTalk.com