On Dec 26, 2007 10:52 PM, Guillaume Smet <guillaume.smet@gmail.com> wrote:
> Let's go with 8.2.5 on the same server (-s 100 / 16 clients / 50k
> transactions per client / only read using -S option):
> 64MB: 33814 tps
> 512MB: 35833 tps
> 1024MB: 36986 tps
> It's more consistent with what I expected.

I had the same numbers with 8.3b4.x86_64 RPMs compiled by Devrim than
with the ones I compiled myself. While discussing with Devrim, I
checked the .spec with a little more attention and... I noticed that
beta RPMs are by default compiled with --enable-debug and
--enable-cassert which doesn't help them to fly fast...
I did all my previous benchmarks with binaries compiled directly from
CVS so I didn't notice it before and this new server was far faster
than the box I tested 8.3devel before so I wasn't surprised by the
other results..

So, the conclusion is: if you really want to test/benchmark 8.3beta4
using the RPM packages, you'd better compile your own set of RPMs
using --define "beta 0".

Really sorry for the noise but anyway quite happy to have discovered
the pgbench-tools of Greg.

I hope it will be useful to other people. I'll post new results
yesterday with a clean beta4 install.

--
Guillaume

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

On Wed, 26 Dec 2007, Guillaume Smet wrote:

> beta RPMs are by default compiled with --enable-debug and
> --enable-cassert which doesn't help them to fly fast...

Got that right. Last time I was going crazy after running pgbench with
those options and not having realized what I changed, I was getting a 50%
slowdown on results that way compared to without the debugging stuff.
Didn't realize it scaled with shared_buffers though.

> Really sorry for the noise

Nothing to be sorry for, I know I wasn't aware the beta RPMs were compiled
that way. Probably need to put a disclaimer about that fact *somewhere*.
It's unfortunate for you, but I know I'm glad you run into it rather than
someone who wouldn't have followed through to figure out the cause.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Hi,

On Wed, 2007-12-26 at 18:35 -0500, Greg Smith wrote:
> Probably need to put a disclaimer about that fact *somewhere*.

We mention about that in README.rpm-dist file, but I think we should
mention about that at a more visible place.

Regards,
--
Devrim GÃœNDÃœZ , RHCE
PostgreSQL Replication, Consulting, Custom Development, 24x7 support
Managed Services, Shared and Dedicated Hosting
Co-Authors: plPHP, ODBCng - http://www.commandprompt.com/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)

iD8DBQBHcuaDpme12CBXnxERAk5vAJ9AF+ok9dBGpkgeYWAJR3 bI6/yvnACbBzdo
0vdreNqHjUEeuS+mIn6d2xM=
=BOsJ
-----END PGP SIGNATURE-----

Greg Smith <gsmith@gregsmith.com> writes:
> On Wed, 26 Dec 2007, Guillaume Smet wrote:
>> beta RPMs are by default compiled with --enable-debug and
>> --enable-cassert which doesn't help them to fly fast...

> Got that right. Last time I was going crazy after running pgbench with
> those options and not having realized what I changed, I was getting a 50%
> slowdown on results that way compared to without the debugging stuff.
> Didn't realize it scaled with shared_buffers though.

See AtEOXact_Buffers(). There are probably any number of other
interesting scaling behaviors --- in my tests, AllocSetCheck() is
normally a major cycle-eater if --enable-cassert is set, and that costs
time proportional to the number of memory chunks allocated by the query.

Currently the docs say that --enable-cassert

Enables <firstterm>assertion</> checks in the server, which test for
many <quote>cannot happen</> conditions. This is invaluable for
code development purposes, but the tests slow things down a little.

Maybe we ought to put that more strongly --- s/a little/significantly/,
perhaps?

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

On Dec 27, 2007 7:10 AM, Tom Lane <tgl@sss.pgh.pa.us> wrote:
> Enables <firstterm>assertion</> checks in the server, which test for
> many <quote>cannot happen</> conditions. This is invaluable for
> code development purposes, but the tests slow things down a little.
>
> Maybe we ought to put that more strongly --- s/a little/significantly/,
> perhaps?

+1. It seems closer to the reality.

--
Guillaume

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

On Thu, Dec 27, 2007 at 01:10:29AM -0500, Tom Lane wrote:
> Greg Smith <gsmith@gregsmith.com> writes:
> > On Wed, 26 Dec 2007, Guillaume Smet wrote:
> >> beta RPMs are by default compiled with --enable-debug and
> >> --enable-cassert which doesn't help them to fly fast...
>
> > Got that right. Last time I was going crazy after running pgbench with
> > those options and not having realized what I changed, I was getting a 50%
> > slowdown on results that way compared to without the debugging stuff.
> > Didn't realize it scaled with shared_buffers though.
>
> See AtEOXact_Buffers(). There are probably any number of other
> interesting scaling behaviors --- in my tests, AllocSetCheck() is
> normally a major cycle-eater if --enable-cassert is set, and that costs
> time proportional to the number of memory chunks allocated by the query.
>
> Currently the docs say that --enable-cassert
>
> Enables <firstterm>assertion</> checks in the server, which test for
> many <quote>cannot happen</> conditions. This is invaluable for
> code development purposes, but the tests slow things down a little.
>
> Maybe we ought to put that more strongly --- s/a little/significantly/,
> perhaps?

Sounds like a good idea. We got bit by the same thing when doing some
benchmarks on the MSVC port (and with we I mean Dave did the work, and several
people couldn't understand why the numbers sucked)

//Magnus

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Tom Lane escribió:

> Currently the docs say that --enable-cassert
>
> Enables <firstterm>assertion</> checks in the server, which test for
> many <quote>cannot happen</> conditions. This is invaluable for
> code development purposes, but the tests slow things down a little.
>
> Maybe we ought to put that more strongly --- s/a little/significantly/,
> perhaps?

I don't think it will make any difference, because people don't read
configure documentation. They read configure --help.

--
Alvaro Herrera http://www.CommandPrompt.com/
PostgreSQL Replication, Consulting, Custom Development, 24x7 support

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

"Alvaro Herrera" <alvherre@commandprompt.com> writes:

> Tom Lane escribió:
>
>> Currently the docs say that --enable-cassert
>>
>> Enables <firstterm>assertion</> checks in the server, which test for
>> many <quote>cannot happen</> conditions. This is invaluable for
>> code development purposes, but the tests slow things down a little.
>>
>> Maybe we ought to put that more strongly --- s/a little/significantly/,
>> perhaps?
>
> I don't think it will make any difference, because people don't read
> configure documentation. They read configure --help.

Fwiw I think you're all getting a bit caught up in this one context. While the
slowdown is significant when you take out the stopwatch, under normal
interactive use you're not going to notice your queries being especially slow.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Ask me about EnterpriseDB's On-Demand Production Tuning

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

On Thu, 27 Dec 2007, Gregory Stark wrote:

> Fwiw I think you're all getting a bit caught up in this one context.

I lost a day once over this problem. Guillaume lost at least that much.
Sounds like Magnus and Dave got a good sized dose as well. Seems like
something worth warning people about to me. The worst time people can run
into a performance regression is when they're running a popular
benchmarking tool. I didn't think this was a big problem because I
thought it was limited to developers who shot their own foot, but if there
are packagers turning this on to improve beta feedback it deserves some
wider mention.

As for the suggestion that people don't read the documentation, take a
look at the above list of developers and tell me whether that group is
aware of what's in the docs or not. I had never seen anyone bring this up
before I ran into it, and I dumped a strong warning into
http://developer.postgresql.org/inde...#Initial_setup
so at least it was written down somewhere.

--
* Greg Smith gsmith@gregsmith.com http://www.gregsmith.com Baltimore, MD

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Greg Smith <gsmith@gregsmith.com> writes:
> ... I didn't think this was a big problem because I
> thought it was limited to developers who shot their own foot, but if there
> are packagers turning this on to improve beta feedback it deserves some
> wider mention.

Yeah, binary packages that are built with --enable-cassert perhaps need
to be labeled as "not intended for benchmarking" or some such.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate