Unix Technical Forum

select .. where ..in TOO LONG

This is a discussion on select .. where ..in TOO LONG within the MySQL forums, part of the Database Server Software category; --> I need to perform a select based on a list of id's , it's fine when the number of ...


Go Back   Unix Technical Forum > Database Server Software > MySQL

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 02-28-2008, 09:48 AM
Josselin
 
Posts: n/a
Default select .. where ..in TOO LONG

I need to perform a select based on a list of id's , it's fine when the
number of id's is not too big, but when It can be very large..

what options could I have ?

create a temporary table and match with this table ... each time I need
to perform the select... ?
no better option in term of performances ?

thanks for yoru lights

joss

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 02-28-2008, 09:48 AM
Jared
 
Posts: n/a
Default Re: select .. where ..in TOO LONG

Strange problem to have. I would nuke a view everytime since i think you
would get better string length in a view. And then select the view.
Assumming nonquery can take larger sql statement then datareader.




"Josselin" <josselin@wanadoo.fr> wrote in message
news:4587b04c$0$27412$ba4acef3@news.orange.fr...
>I need to perform a select based on a list of id's , it's fine when the
>number of id's is not too big, but when It can be very large..
>
> what options could I have ?
>
> create a temporary table and match with this table ... each time I need to
> perform the select... ?
> no better option in term of performances ?
>
> thanks for yoru lights
>
> joss
>



Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 02-28-2008, 09:48 AM
Michael Austin
 
Posts: n/a
Default Re: select .. where ..in TOO LONG

Josselin wrote:
> I need to perform a select based on a list of id's , it's fine when the
> number of id's is not too big, but when It can be very large..
>
> what options could I have ?
>
> create a temporary table and match with this table ... each time I need
> to perform the select... ?
> no better option in term of performances ?
>
> thanks for yoru lights
>
> joss
>


The method used depends on how you choose which ID's to look for. If the data is
derived from the database and not user input, then you can use a derived table -
basically a view on-the-fly:

select field1,field2 from tablea where id in (select id from table2 where
somevalue=somecriteria);

or

select temptab.field1, temptab.field2 from (select id, field1,field2 where
somevalue=someothervalue) temptab ;

or use a join

select a.field1, b.field2 from table1 a join table2 b on a.id=b.id
where b.status = 1;

You have a database engine and these are just starting points...

--
Michael Austin.
Database Consultant
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 02-28-2008, 09:48 AM
Josselin
 
Posts: n/a
Default Re: select .. where ..in TOO LONG

On 2006-12-20 04:28:37 +0100, Michael Austin <maustin@firstdbasource.com> said:

> Josselin wrote:
>> I need to perform a select based on a list of id's , it's fine when the
>> number of id's is not too big, but when It can be very large..
>>
>> what options could I have ?
>>
>> create a temporary table and match with this table ... each time I need
>> to perform the select... ?
>> no better option in term of performances ?
>>
>> thanks for yoru lights
>>
>> joss
>>

>
> The method used depends on how you choose which ID's to look for. If
> the data is derived from the database and not user input, then you can
> use a derived table - basically a view on-the-fly:
>
> select field1,field2 from tablea where id in (select id from table2
> where somevalue=somecriteria);
>
> or
>
> select temptab.field1, temptab.field2 from (select id, field1,field2
> where somevalue=someothervalue) temptab ;
>
> or use a join
>
> select a.field1, b.field2 from table1 a join table2 b on a.id=b.id
> where b.status = 1;
>
> You have a database engine and these are just starting points...


thanks ..
the data is.. well ... not actually derived from a DB it's coming from
a serialized Ruby Array of arrays , stored into a text column (km25)
in a table 'cities' (I am using Rails..) , the original array is like
[ [ 23455, 2.45025], [ 45896, 4.56876], ..... ]
km25 means 'all cities around 25 km) , each array store the id of the
city (int) and the km- distance (float).

Getting a current_city from the table in memory, brings this data ,
getting immediatly all the id's/distance of all cities around 25km
makes a BIG difference in performance vs calculating it from the
LAT-LONG data
then I need to extract 'proposals' for the current city (one-to-many)
AND from the cities in this list... that's why I am using the SELECT ..
WHERE cities_id IN (the array of id's)....

drawback : I had to run a batch app to perform the calculous for all
the cities in the table (40'000 ) and it tooks many hours... but these
data will never change... no new city.. and the distances are
immutable (at least until the global earth warming... )))

It's running fine now .. ! I got it

joss

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 04:13 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
Search Engine Optimization by vBSEO 3.2.0
www.UnixAdminTalk.com