Unix Technical Forum

Massive delete of rows, how to proceed?

This is a discussion on Massive delete of rows, how to proceed? within the Pgsql Performance forums, part of the PostgreSQL category; --> Hi all, I have a table with statistics with more than 15 million rows. I'd like to delete the ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > Pgsql Performance

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-19-2008, 09:46 AM
Arnau
 
Posts: n/a
Default Massive delete of rows, how to proceed?

Hi all,

I have a table with statistics with more than 15 million rows. I'd
like to delete the oldest statistics and this can be about 7 million
rows. Which method would you recommend me to do this? I'd be also
interested in calculate some kind of statistics about these deleted
rows, like how many rows have been deleted for date. I was thinking in
creating a function, any recommendations?

Thank you very much
--
Arnau

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-19-2008, 09:46 AM
Peter Childs
 
Posts: n/a
Default Massive delete of rows, how to proceed?

On 24/11/06, Arnau <arnaulist@andromeiberica.com> wrote:
> Hi all,
>
> I have a table with statistics with more than 15 million rows. I'd
> like to delete the oldest statistics and this can be about 7 million
> rows. Which method would you recommend me to do this? I'd be also
> interested in calculate some kind of statistics about these deleted
> rows, like how many rows have been deleted for date. I was thinking in
> creating a function, any recommendations?



Copy and drop old table. If you delete you will have a massive problem
with a bloated table and vacuum will not help unless you expect the
table to grow to this size regulally otherwise vacuum full will take
ages.

Peter.

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-19-2008, 09:47 AM
Merlin Moncure
 
Posts: n/a
Default Re: Massive delete of rows, how to proceed?

On 11/25/06, Arnau <arnaulist@andromeiberica.com> wrote:
> Hi all,
>
> I have a table with statistics with more than 15 million rows. I'd
> like to delete the oldest statistics and this can be about 7 million
> rows. Which method would you recommend me to do this? I'd be also
> interested in calculate some kind of statistics about these deleted
> rows, like how many rows have been deleted for date. I was thinking in
> creating a function, any recommendations?


a function, like an sql statement, operates in a single transaction
and you are locking quite a few records in this operation. merlin's
3rd rule: long running transactions are (usually) evil.

my gut says moving the keeper records to a swap table, dropping the
main table, and swapping the tables back might be better. However,
this kind of stuff can cause problems with logged in sessions because
of plan issues, beware.

do not write a function to delete records row by row unless you have
exhausted all other courses of action.

merlin

---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 04:27 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.2.0
www.UnixAdminTalk.com