This is a discussion on Re: pg primary key bug? within the pgsql Sql forums, part of the PostgreSQL category; --> There are not enough spaces available in the column to allow for the number of DOC_IDs. There are three ...
| |||||||
| Register | FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| ||||
| There are not enough spaces available in the column to allow for the number of DOC_IDs. There are three spaces, allowing for at most four occurrences of DOC_ID, but there are eleven. --Rick Michael Glaesemann <grzm@myrealbox.com To: pginfo <pginfo@t1.unisoftbg.com> > cc: pgsql-sql@postgresql.org Sent by: Subject: Re: [SQL] pg primary key bug? pgsql-sql-owner@pos tgresql.org 02/03/2005 09:14 AM On Feb 3, 2005, at 21:53, pginfo wrote: > I am using pg 7.4.5 on RedHat AS 3.0. > sklad21=# \d a_constants_str > Table "public.a_constants_str" > Column | Type | Modifiers > ------------+-----------------------+----------- > constname | character varying(30) | not null > fid | integer | not null > constvalue | character varying(30) | > Indexes: > "a_constants_str_pkey" primary key, btree (constname, fid) > > sklad21=# select * from a_constants_str ; > constname | fid | constvalue -----------+-----+------------- > AACCGRID | 0 | SOF_3 > AKLTYPID | 0 | SOF_3 > ADARID | 0 | SOF_2 > AOBLASTID | 0 | SOF_6 > AUSERID | 0 | SOF_17 > ANMGRID | 0 | SOF_21 > LOCAID | 0 | SOF_41 > DOCID | 0 | SOF_1585254 > DOCPLAID | 0 | SOF_1052900 > AKLIID | 0 | SOF_18740 > DOCRID | 0 | SOF_2268142 > DOCPOGPLA | 0 | SOF_324586 > DOCID | 0 | SOF_1585254 > DOCID | 0 | SOF_1585254 > DOCID | 0 | SOF_1585254 > AKLGRID | 0 | SOF_45 > DOCID | 0 | SOF_1585254 > DOCID | 0 | SOF_1585254 > ASETUPID | 0 | SOF_4605 > DOCID | 0 | SOF_1585254 > DOCID | 0 | SOF_1585254 > TDOCID | 0 | SOF_337 > TDOCRID | 0 | SOF_19450 > DOCID | 0 | SOF_1585254 > DOCID | 0 | SOF_1585254 > AGRADID | 0 | SOF_256 > DOCID | 0 | SOF_1585254 > ASLUID | 0 | SOF_46 > NASTRF | 0 | SOF_88 > ANOMID | 0 | SOF_1200 > (30 rows) > > > Pls., see the records with 'DOCID' and note we have primary key > defined. It's unclear from just this data, but there's a chance that there are varying numbers of spaces after 'DOCID', which would appear as separate values for the index, even though they aren't readily apparent. Could you show us the results of the following query? select constname, length(constname) as constname_length from a_constants_str; Here's another one which would show if PostgreSQL is treating them equally: select constname, count(constname) from a_constants_str; The results of these queries might shed some light on the issue. Hope this helps. Michael Glaesemann grzm myrealbox com ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) ---------------------------(end of broadcast)--------------------------- TIP 2: you can get off all lists at once with the unregister command (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) |
| |||
| Hi all, Sorry for my post, but I think it is pg primary key bug. It is secont time in that we found the bug (see the replays for this message). We have many server with pg and use it over jdbc + jboss. I am not able to stop this server for long time, but I have dumped the problem table. It is very important to know if it is bug, because we have many server running pg + our ERP and continuing to install new. 01=# select * from a_constants_str order by constname; constname | fid | constvalue -----------+-----+------------ AACCGRID | 0 | SOF_3 ADARID | 0 | SOF_2 AGRADID | 0 | SOF_165 AKLGRID | 0 | SOF_8 AKLIID | 0 | SOF_3513 AKLTYPID | 0 | SOF_3 ANMGRID | 0 | SOF_10 ANOMID | 0 | SOF_747 AOBLASTID | 0 | SOF_3 ASETUPID | 0 | SOF_399 ASLUID | 0 | SOF_17 AUSERID | 0 | SOF_3 DOCID | 0 | SOF_25658 DOCPLAID | 0 | SOF_19738 DOCPLAID | 0 | SOF_19738 DOCPOGPLA | 0 | SOF_24281 DOCRID | 0 | SOF_184547 LOCAID | 0 | SOF_13 NASTRF | 0 | SOF_1 TDOCID | 0 | SOF_47 TDOCRID | 0 | SOF_2439 (21 rows) 01=# select * from a_constants_str where constname='DOCPLAID' ; constname | fid | constvalue -----------+-----+------------ DOCPLAID | 0 | SOF_19738 DOCPLAID | 0 | SOF_19738 (2 rows) 01=# \d a_constants_str; Table "public.a_constants_str" Column | Type | Modifiers ------------+-----------------------+----------- constname | character varying(30) | not null fid | integer | not null constvalue | character varying(30) | Indexes: "a_constants_str_pkey" primary key, btree (constname, fid) regards, ivan Richard_D_Levine@raytheon.com wrote: >There are not enough spaces available in the column to allow for the number >of DOC_IDs. There are three spaces, allowing for at most four occurrences >of DOC_ID, but there are eleven. --Rick > > > > Michael Glaesemann > <grzm@myrealbox.com To: pginfo <pginfo@t1.unisoftbg.com> > > cc: pgsql-sql@postgresql.org > Sent by: Subject: Re: [SQL] pg primary key bug? > pgsql-sql-owner@pos > tgresql.org > > > 02/03/2005 09:14 AM > > > > > > > >On Feb 3, 2005, at 21:53, pginfo wrote: > > > >>I am using pg 7.4.5 on RedHat AS 3.0. >> >> > > > >>sklad21=# \d a_constants_str >> Table "public.a_constants_str" >> Column | Type | Modifiers >>------------+-----------------------+----------- >>constname | character varying(30) | not null >>fid | integer | not null >>constvalue | character varying(30) | >>Indexes: >> "a_constants_str_pkey" primary key, btree (constname, fid) >> >>sklad21=# select * from a_constants_str ; >>constname | fid | constvalue -----------+-----+------------- >>AACCGRID | 0 | SOF_3 >>AKLTYPID | 0 | SOF_3 >>ADARID | 0 | SOF_2 >>AOBLASTID | 0 | SOF_6 >>AUSERID | 0 | SOF_17 >>ANMGRID | 0 | SOF_21 >>LOCAID | 0 | SOF_41 >>DOCID | 0 | SOF_1585254 >>DOCPLAID | 0 | SOF_1052900 >>AKLIID | 0 | SOF_18740 >>DOCRID | 0 | SOF_2268142 >>DOCPOGPLA | 0 | SOF_324586 >>DOCID | 0 | SOF_1585254 >>DOCID | 0 | SOF_1585254 >>DOCID | 0 | SOF_1585254 >>AKLGRID | 0 | SOF_45 >>DOCID | 0 | SOF_1585254 >>DOCID | 0 | SOF_1585254 >>ASETUPID | 0 | SOF_4605 >>DOCID | 0 | SOF_1585254 >>DOCID | 0 | SOF_1585254 >>TDOCID | 0 | SOF_337 >>TDOCRID | 0 | SOF_19450 >>DOCID | 0 | SOF_1585254 >>DOCID | 0 | SOF_1585254 >>AGRADID | 0 | SOF_256 >>DOCID | 0 | SOF_1585254 >>ASLUID | 0 | SOF_46 >>NASTRF | 0 | SOF_88 >>ANOMID | 0 | SOF_1200 >>(30 rows) >> >> >>Pls., see the records with 'DOCID' and note we have primary key >>defined. >> >> > > >It's unclear from just this data, but there's a chance that there are >varying numbers of spaces after 'DOCID', which would appear as separate >values for the index, even though they aren't readily apparent. Could >you show us the results of the following query? > >select constname, length(constname) as constname_length >from a_constants_str; > >Here's another one which would show if PostgreSQL is treating them >equally: > >select constname, count(constname) >from a_constants_str; > >The results of these queries might shed some light on the issue. > >Hope this helps. > >Michael Glaesemann >grzm myrealbox com > > >---------------------------(end of broadcast)--------------------------- >TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > > > > >---------------------------(end of broadcast)--------------------------- >TIP 2: you can get off all lists at once with the unregister command > (send "unregister YourEmailAddressHere" to majordomo@postgresql.org) > > > > |
| |||
| On Thu, Feb 17, 2005 at 04:12:38PM +0100, pginfo wrote: > 01=# select * from a_constants_str where constname='DOCPLAID' ; > constname | fid | constvalue > -----------+-----+------------ > DOCPLAID | 0 | SOF_19738 > DOCPLAID | 0 | SOF_19738 > (2 rows) Do you have any inherited tables? What's the result of the following query? SELECT tableoid::regclass, * FROM a_constants_str WHERE constname = 'DOCPLAID'; Inherited tables are documented to have deficiencies regarding constraints. Observe: CREATE TABLE parent ( constname varchar(30) NOT NULL, fid integer NOT NULL, constvalue varchar(30), PRIMARY KEY (constname, fid) ); CREATE TABLE child () INHERITS (parent); INSERT INTO parent VALUES ('DOCPLAID', 0, 'SOF_19738'); INSERT INTO parent VALUES ('DOCPLAID', 0, 'SOF_19738'); ERROR: duplicate key violates unique constraint "parent_pkey" INSERT INTO child VALUES ('DOCPLAID', 0, 'SOF_19738'); SELECT tableoid::regclass, * FROM parent; tableoid | constname | fid | constvalue ----------+-----------+-----+------------ parent | DOCPLAID | 0 | SOF_19738 child | DOCPLAID | 0 | SOF_19738 (2 rows) -- Michael Fuhr http://www.fuhr.org/~mfuhr/ ---------------------------(end of broadcast)--------------------------- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match |
| |||
| Hi, No, I do not inherited tables. The result of this query is: 01=# SELECT tableoid::regclass, * 01-# FROM a_constants_str 01-# WHERE constname = 'DOCPLAID'; tableoid | constname | fid | constvalue -----------------+-----------+-----+------------ a_constants_str | DOCPLAID | 0 | SOF_19738 a_constants_str | DOCPLAID | 0 | SOF_19738 (2 rows) regards, ivan. Michael Fuhr wrote: >On Thu, Feb 17, 2005 at 04:12:38PM +0100, pginfo wrote: > > > >>01=# select * from a_constants_str where constname='DOCPLAID' ; >>constname | fid | constvalue >>-----------+-----+------------ >>DOCPLAID | 0 | SOF_19738 >>DOCPLAID | 0 | SOF_19738 >>(2 rows) >> >> > >Do you have any inherited tables? What's the result of the following >query? > >SELECT tableoid::regclass, * >FROM a_constants_str >WHERE constname = 'DOCPLAID'; > >Inherited tables are documented to have deficiencies regarding >constraints. Observe: > >CREATE TABLE parent ( > constname varchar(30) NOT NULL, > fid integer NOT NULL, > constvalue varchar(30), > PRIMARY KEY (constname, fid) >); > >CREATE TABLE child () INHERITS (parent); > >INSERT INTO parent VALUES ('DOCPLAID', 0, 'SOF_19738'); > >INSERT INTO parent VALUES ('DOCPLAID', 0, 'SOF_19738'); >ERROR: duplicate key violates unique constraint "parent_pkey" > >INSERT INTO child VALUES ('DOCPLAID', 0, 'SOF_19738'); > >SELECT tableoid::regclass, * FROM parent; > tableoid | constname | fid | constvalue >----------+-----------+-----+------------ > parent | DOCPLAID | 0 | SOF_19738 > child | DOCPLAID | 0 | SOF_19738 >(2 rows) > > > |
| |||
| pginfo <pginfo@t1.unisoftbg.com> writes: > 01=# select * from a_constants_str where constname='DOCPLAID' ; > constname | fid | constvalue > -----------+-----+------------ > DOCPLAID | 0 | SOF_19738 > DOCPLAID | 0 | SOF_19738 > (2 rows) Could we see the system columns on these rows? select oid,xmin,cmin,xmax,cmax,ctid,* from a_constants_str where ... regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 6: Have you searched our list archives? http://archives.postgresql.org |
| |||
| Hi, Tom Lane wrote: >pginfo <pginfo@t1.unisoftbg.com> writes: > > >>01=# select * from a_constants_str where constname='DOCPLAID' ; >> constname | fid | constvalue >>-----------+-----+------------ >> DOCPLAID | 0 | SOF_19738 >> DOCPLAID | 0 | SOF_19738 >>(2 rows) >> >> > >Could we see the system columns on these rows? > > select oid,xmin,cmin,xmax,cmax,ctid,* from a_constants_str where ... > > regards, tom lane > > > > yes, 01=# select oid,xmin,cmin,xmax,cmax,ctid,* from a_constants_str; oid | xmin | cmin | xmax | cmax | ctid | constname | fid | constvalue --------+---------+---------+---------+---------+---------+-----------+-----+------------ 17910 | 22331 | 2 | 2 | 26679 | (0,7) | ANMGRID | 0 | SOF_10 17908 | 985 | 6 | 6 | 0 | (0,18) | AKLTYPID | 0 | SOF_3 17907 | 985 | 30 | 30 | 0 | (0,21) | ADARID | 0 | SOF_2 17921 | 985 | 34 | 34 | 0 | (0,22) | AOBLASTID | 0 | SOF_3 17911 | 4640 | 6 | 6 | 26679 | (0,24) | AACCGRID | 0 | SOF_3 17920 | 1220598 | 2 | 2 | 1475630 | (0,47) | ASLUID | 0 | SOF_17 17917 | 643083 | 2 | 2 | 1475630 | (0,49) | LOCAID | 0 | SOF_13 17918 | 762851 | 3 | 3 | 1475630 | (0,50) | AUSERID | 0 | SOF_3 17923 | 35539 | 165 | 165 | 1475630 | (0,51) | ASETUPID | 0 | SOF_399 283686 | 514327 | 3 | 3 | 1475630 | (0,52) | NASTRF | 0 | SOF_1 17909 | 2156667 | 2 | 2 | 2193198 | (0,54) | AKLGRID | 0 | SOF_8 17922 | 2103298 | 2 | 2 | 2193198 | (0,55) | AGRADID | 0 | SOF_165 17913 | 2092705 | 2 | 2 | 2193198 | (0,56) | ANOMID | 0 | SOF_747 63247 | 2226373 | 2 | 2 | 2233003 | (0,126) | TDOCID | 0 | SOF_47 17914 | 2232893 | 2235861 | 2235861 | 3 | (1,125) | DOCID | 0 | SOF_25658 17915 | 2232893 | 2235861 | 2235861 | 19 | (3,38) | DOCRID | 0 | SOF_184547 17916 | 2232893 | 2235861 | 2235861 | 42 | (4,71) | DOCPLAID | 0 | SOF_19738 17919 | 2232893 | 231 | 231 | 2233003 | (5,104) | DOCPOGPLA | 0 | SOF_24281 17912 | 2231139 | 2 | 2 | 0 | (6,1) | AKLIID | 0 | SOF_3513 17916 | 2232893 | 2235861 | 2235861 | 41 | (7,62) | DOCPLAID | 0 | SOF_19738 63249 | 2226373 | 103 | 103 | 0 | (16,31) | TDOCRID | 0 | SOF_2439 (21 rows) and 01=# select oid,xmin,cmin,xmax,cmax,ctid,* from a_constants_str where constname='DOCPLAID'; oid | xmin | cmin | xmax | cmax | ctid | constname | fid | constvalue -------+---------+---------+---------+------+--------+-----------+-----+------------ 17916 | 2232893 | 2235861 | 2235861 | 42 | (4,71) | DOCPLAID | 0 | SOF_19738 17916 | 2232893 | 2235861 | 2235861 | 41 | (7,62) | DOCPLAID | 0 | SOF_19738 (2 rows) regards, ivan. |
| |||
| pginfo <pginfo@t1.unisoftbg.com> writes: > Tom Lane wrote: >> Could we see the system columns on these rows? > 01=# select oid,xmin,cmin,xmax,cmax,ctid,* from a_constants_str where > constname='DOCPLAID'; > oid | xmin | cmin | xmax | cmax | ctid | constname | fid | > constvalue > -------+---------+---------+---------+------+--------+-----------+-----+------------ > 17916 | 2232893 | 2235861 | 2235861 | 42 | (4,71) | DOCPLAID | 0 | > SOF_19738 > 17916 | 2232893 | 2235861 | 2235861 | 41 | (7,62) | DOCPLAID | 0 | > SOF_19738 > (2 rows) Given the identical OID and xmin values, it seems certain that these are the "same" row, ie there was only one insertion event. My bet is that the one at (7,62) is the original, and that the one at (4,71) is a copy that was made by VACUUM FULL trying to move the row to compact the table. So the question is how did both copies get to be marked simultaneously valid? That should be impossible, unless a disk write got dropped. Have you had any system crashes during VACUUM FULL operations recently? regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 9: the planner will ignore your desire to choose an index scan if your joining column's datatypes do not match |
| |||
| Hi, Tom Lane wrote: >pginfo <pginfo@t1.unisoftbg.com> writes: > > >>Tom Lane wrote: >> >> >>>Could we see the system columns on these rows? >>> >>> > > > >>01=# select oid,xmin,cmin,xmax,cmax,ctid,* from a_constants_str where >>constname='DOCPLAID'; >> oid | xmin | cmin | xmax | cmax | ctid | constname | fid | >>constvalue >>-------+---------+---------+---------+------+--------+-----------+-----+------------ >> 17916 | 2232893 | 2235861 | 2235861 | 42 | (4,71) | DOCPLAID | 0 | >>SOF_19738 >> 17916 | 2232893 | 2235861 | 2235861 | 41 | (7,62) | DOCPLAID | 0 | >>SOF_19738 >>(2 rows) >> >> > >Given the identical OID and xmin values, it seems certain that these are >the "same" row, ie there was only one insertion event. My bet is that >the one at (7,62) is the original, and that the one at (4,71) is a copy >that was made by VACUUM FULL trying to move the row to compact the >table. So the question is how did both copies get to be marked >simultaneously valid? That should be impossible, unless a disk write >got dropped. Have you had any system crashes during VACUUM FULL >operations recently? > > regards, tom lane > > > > I do not know exact, but it is possible. This is customer server and I do not have any info about server/os crash, but I am 100% sure. I will ask as soon as possible and replay. Also I will check the logs. We start pg witl pg_ctl ...-D... -l /logfile and if we can found the needet info in pg log file I can check it ( I do not nkow what to search in this file). We are running vacuum once per 24 h. The OS is RedHat AS 3. PostgreSQL is 7.4.1 (but we have the same problem on 7.4.5). Will upgrade to 8.0 solve this type of problems ? regards, ivan. |
| |||
| pginfo <pginfo@t1.unisoftbg.com> writes: > Will upgrade to 8.0 solve this type of problems ? The problem is probably not Postgres' fault at all. I'm wondering about disks with write cacheing enabled. And you didn't turn off fsync, I trust? regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 8: explain analyze is your friend |
| ||||
| Hi, Tom Lane wrote: >pginfo <pginfo@t1.unisoftbg.com> writes: > > >>Will upgrade to 8.0 solve this type of problems ? >> >> > >The problem is probably not Postgres' fault at all. I'm wondering about >disks with write cacheing enabled. And you didn't turn off fsync, >I trust? > > About fsync (part from postgresql.conf) : #--------------------------------------------------------------------------- # WRITE AHEAD LOG #--------------------------------------------------------------------------- # - Settings - #fsync = true # turns forced synchronization on or off #wal_sync_method = fsync # the default varies across platforms: # fsync, fdatasync, open_sync, or open_datasync #wal_buffers = 8 # min 4, 8KB each # - Checkpoints - #checkpoint_segments = 3 # in logfile segments, min 1, 16MB each #checkpoint_timeout = 300 # range 30-3600, in seconds #checkpoint_warning = 30 # 0 is off, in seconds Also part from pg logfile: LOG: statistics collector process (PID 2716) exited with exit code 1 LOG: shutting down LOG: database system is shut down LOG: could not create IPv6 socket: Address family not supported by protocol LOG: database system was shut down at 2005-02-11 19:58:26 EET LOG: checkpoint record is at 2/BAC39188 LOG: redo record is at 2/BAC39188; undo record is at 0/0; shutdown TRUE LOG: next transaction ID: 2221145; next OID: 826607 LOG: database system is ready LOG: recycled transaction log file "00000002000000BA" LOG: recycled transaction log file "00000002000000BB" LOG: recycled transaction log file "00000002000000BC" LOG: recycled transaction log file "00000002000000BD" LOG: recycled transaction log file "00000002000000BE" WARNING: index "a_constants_str_pkey" contains 1449 row versions, but table contains 1422 row versions HINT: Rebuild the index with REINDEX. ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" ERROR: duplicate key violates unique constraint "a_constants_str_pkey" LOG: received smart shutdown request FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command FATAL: terminating connection due to administrator command LOG: statistics collector process (PID 2713) exited with exit code 1 LOG: shutting down LOG: database system is shut down LOG: could not create IPv6 socket: Address family not supported by protocol LOG: database system was shut down at 2005-02-16 08:32:21 EET LOG: checkpoint record is at 2/BFAE09EC LOG: redo record is at 2/BFAE09EC; undo record is at 0/0; shutdown TRUE Note we was informed about the problem on 2005-02-16 and rebooted the box. As I see in log file the last restart was on 2005-02-11 and after it all worked well. The affected table is critical and the most used table in this system and if the insert stop to work stop also the system. I will notice also that in the first case when we found the same problem an this system worked ~80 users. Is it possible pg_dump or reindex table or vacuum analyze to make this problem? We are using it very regular. Will we need to stop using vacuum full at all? regards, ivan. > regards, tom lane > >---------------------------(end of broadcast)--------------------------- >TIP 8: explain analyze is your friend > > > > |