Unix Technical Forum

Re: More shared_buffers instead of effective_cache_size?

This is a discussion on Re: More shared_buffers instead of effective_cache_size? within the Pgsql Performance forums, part of the PostgreSQL category; --> On Thu, Sep 4, 2008 at 1:24 PM, Ulrich <ulrich.mierendorff@gmx.net> wrote: > Hi, > I have a virtual server ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > Pgsql Performance

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 09-05-2008, 12:05 PM
Scott Marlowe
 
Posts: n/a
Default Re: More shared_buffers instead of effective_cache_size?

On Thu, Sep 4, 2008 at 1:24 PM, Ulrich <ulrich.mierendorff@gmx.net> wrote:
> Hi,
> I have a virtual server with 256 MB of RAM. I am using it as a webserver,
> mailserver and for postgres. So there is something like 150MB left for
> postgres.
>
> Here are my configs (I haven't benchmarked...)
> max_connections = 12 (I think, I will not have more parallel connections,
> because I only have 10 PHP worker threads)
> shared_buffers = 24MB
> work_mem = 1MB
> maintenance_work_mem = 16MB
>
> (effective_cache_size = 80MB)
>
> Normally, the file-cache is part of the free ram. But on my virtual server,
> it looks like if there is one big file cache for the whole hardware node and
> I do not have my own reserved cached, so it is not easy to find a good value
> for effective_cache_size.
>
> I've also benchmarked the file-cache using dd (100MB file)
>
> 1. Read from HDD:
> 104857600 bytes (105 MB) copied, 8.38522 seconds, 12.5 MB/s
> 2. Read from Cache:
> 104857600 bytes (105 MB) copied, 3.48694 seconds, 30.1 MB/s
>
> That is really really slow (10 times slower than on my other machine).
>
> What would you do now? Increasing shared_buffers to 100MB and setting
> effective_cache_size to 0MB? Or increasing effective_cache_size, too?


Stop using a virtual server? I wouldn't set shared_buffers that high
just because things like vacuum and sorts need memory too.

>
> Thanks for help.
>
> Regards,
> -Ulrich
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>


--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 09-05-2008, 12:05 PM
Ulrich
 
Posts: n/a
Default Re: More shared_buffers instead of effective_cache_size?

Scott Marlowe wrote:
> Stop using a virtual server?

That is not possible...
> I wouldn't set shared_buffers that high
> just because things like vacuum and sorts need memory too

Okay, I understand that vacuum uses memory, but I thought sorts are done
in work_mem? I am only sorting the result of one query which will never
return more than 500 rows.

-Ulrich

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 09-05-2008, 12:05 PM
Scott Marlowe
 
Posts: n/a
Default Re: More shared_buffers instead of effective_cache_size?

On Thu, Sep 4, 2008 at 1:39 PM, Ulrich <ulrich.mierendorff@gmx.net> wrote:
> Scott Marlowe wrote:
>>
>> Stop using a virtual server?

>
> That is not possible...


Sorry shoulda had a smiley face at the end of that. <-- there

>> I wouldn't set shared_buffers that high
>> just because things like vacuum and sorts need memory too

>
> Okay, I understand that vacuum uses memory, but I thought sorts are done in
> work_mem? I am only sorting the result of one query which will never return
> more than 500 rows.


You can probably play with larger shared memory, but I'm betting that
the fact that you're running under a VM is gonna weigh eveything down
a great deal, to the point that you're tuning is going to have minimal
effect.

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 09-05-2008, 12:05 PM
Ulrich
 
Posts: n/a
Default Re: More shared_buffers instead of effective_cache_size?

Scott Marlowe wrote:
> On Thu, Sep 4, 2008 at 1:39 PM, Ulrich <ulrich.mierendorff@gmx.net> wrote:
>
>>> I wouldn't set shared_buffers that high
>>> just because things like vacuum and sorts need memory too
>>>

>> Okay, I understand that vacuum uses memory, but I thought sorts are done in
>> work_mem? I am only sorting the result of one query which will never return
>> more than 500 rows.
>>

>
> You can probably play with larger shared memory, but I'm betting that
> the fact that you're running under a VM is gonna weigh eveything down
> a great deal, to the point that you're tuning is going to have minimal
> effect.
>

Hmm... Why do you think so? Is there a reason for it or do other people
have problems with virtual servers and databases?
I have reserved cpu power and reserved ram (okay, not much, but it is
reserved ;-) ), the only thing I dont have is reserved file-cache.

-Ulrich

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 09-05-2008, 12:05 PM
Scott Marlowe
 
Posts: n/a
Default Re: More shared_buffers instead of effective_cache_size?

On Thu, Sep 4, 2008 at 2:01 PM, Ulrich <ulrich.mierendorff@gmx.net> wrote:
> Scott Marlowe wrote:
>>
>> On Thu, Sep 4, 2008 at 1:39 PM, Ulrich <ulrich.mierendorff@gmx.net> wrote:
>>
>>>>
>>>> I wouldn't set shared_buffers that high
>>>> just because things like vacuum and sorts need memory too
>>>>
>>>
>>> Okay, I understand that vacuum uses memory, but I thought sorts are done
>>> in
>>> work_mem? I am only sorting the result of one query which will never
>>> return
>>> more than 500 rows.
>>>

>>
>> You can probably play with larger shared memory, but I'm betting that
>> the fact that you're running under a VM is gonna weigh eveything down
>> a great deal, to the point that you're tuning is going to have minimal
>> effect.
>>

>
> Hmm... Why do you think so? Is there a reason for it or do other people have
> problems with virtual servers and databases?
> I have reserved cpu power and reserved ram (okay, not much, but it is
> reserved ;-) ), the only thing I dont have is reserved file-cache.


Well, Databases tend to be IO bound, and VMs don't tend to focus on IO
performance as much as CPU/Memory performance. Also, things like
shared memory likely don't get as much attention in a VM either. Just
guessing, I haven't tested a lot of VMs.

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump

Similar Threads for: Re: More shared_buffers instead of effective_cache_size?

Thread Thread Starter Forum Replies Last Post
max_connections / shared_buffers / effective_cache_size questions Puddle Pgsql Performance 3 04-18-2008 12:57 PM
effective_cache_size vs units Magnus Hagander pgsql Hackers 52 04-12-2008 07:19 AM
Re: effective_cache_size vs units Peter Eisentraut pgsql Hackers 1 04-12-2008 07:02 AM
effective_cache_size is a real? Peter Eisentraut pgsql Hackers 6 04-12-2008 04:42 AM
effective_cache_size v824 dx k9 pgsql Admins 0 04-10-2008 09:17 AM


All times are GMT. The time now is 07:11 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.2.0
www.UnixAdminTalk.com