Hi,

I am storing a log of HTTP requests in a database table (including IP
address):

http_log: id(PK), path, time, ip

I have another table that contains CIDR ranges and names for them:

network_names: id(PK), cidr, name

Some example data for both tables:

network_names:
1, 192.168.0.0/24, 'Engineering'
2, 192.168.1.0/24, 'Media'
3, 192.168.2.0/24, 'Engineering'
4, 192.168.3.0/24, 'Accounting'
5, 192.168.4.0/24, 'Engineering'
6, 10.0.0.0/8, 'Engineering'

http_log:
1, '/index.html', 110000001, 192.168.0.47/32
2, '/index.html', 110000023, 200.1.2.3/32
3, '/index.html', 110000059, 1.2.3.4/32
4, '/index.html', 110000232, 192.168.2.1/32
5, '/index.html', 113919102, 192.168.1.39/32
6, '/index.html', 129101293, 10.2.2.4/32
7, '/index.html', 132828282, 192.168.4.2/32

Now, in trying to produce a report on this data, I've come up against an
interesting (to me at least!) problem..

I basically want the same output as in http_log, but substituting the IP
with the network name where available, i.e:

1, '/index.html', 110000001, Engineering
2, '/index.html', 110000023, 200.1.2.3/32
3, '/index.html', 110000059, 1.2.3.4/32
4, '/index.html', 110000232, Engineering
5, '/index.html', 113919102, Media
6, '/index.html', 129101293, Engineering
7, '/index.html', 132828282, Engineering

I'm wondering what the best way of doing this is (considering that http_log
could have >100000 rows) Is it possible to somehow JOIN using the <<= and
>>= network operators? Or would I have to iterate the network_names table
manually with LOOP (or something) on every row of the http_log?

If anyone can share some advice, that would be great!

Thanks,
JST

On Oct 25, 2007, at 1:22 PM, Jamie Tufnell wrote:

> Hi,
>
> I am storing a log of HTTP requests in a database table (including
> IP address):
>
> http_log: id(PK), path, time, ip
>
> I have another table that contains CIDR ranges and names for them:
>
> network_names: id(PK), cidr, name
>
> Some example data for both tables:
>
> network_names:
> 1, 192.168.0.0/24, 'Engineering'
> 2, 192.168.1.0/24, 'Media'
> 3, 192.168.2.0/24, 'Engineering'
> 4, 192.168.3.0/24, 'Accounting'
> 5, 192.168.4.0/24, 'Engineering'
> 6, 10.0.0.0/8, 'Engineering'
>
> http_log:
> 1, '/index.html', 110000001, 192.168.0.47/32
> 2, '/index.html', 110000023, 200.1.2.3/32
> 3, '/index.html', 110000059, 1.2.3.4/32
> 4, '/index.html', 110000232, 192.168.2.1/32
> 5, '/index.html', 113919102, 192.168.1.39/32
> 6, '/index.html', 129101293, 10.2.2.4/32
> 7, '/index.html', 132828282, 192.168.4.2/32
>
> Now, in trying to produce a report on this data, I've come up
> against an interesting (to me at least!) problem..
>
> I basically want the same output as in http_log, but substituting
> the IP with the network name where available, i.e:
>
> 1, '/index.html', 110000001, Engineering
> 2, '/index.html', 110000023, 200.1.2.3/32
> 3, '/index.html', 110000059, 1.2.3.4/32
> 4, '/index.html', 110000232, Engineering
> 5, '/index.html', 113919102, Media
> 6, '/index.html', 129101293, Engineering
> 7, '/index.html', 132828282, Engineering
>
> I'm wondering what the best way of doing this is (considering that
> http_log could have >100000 rows) Is it possible to somehow JOIN
> using the <<= and >>= network operators? Or would I have to
> iterate the network_names table manually with LOOP (or something)
> on every row of the http_log?
>
> If anyone can share some advice, that would be great!

Check out: http://www.postgresql.org/docs/8.2/i...ive/functions-
net.html

Erik Jones

Software Developer | Emma®
erik@myemma.com
800.595.4401 or 615.292.5888
615.292.0777 (fax)

Emma helps organizations everywhere communicate & market in style.
Visit us online at http://www.myemma.com



---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

"Jamie Tufnell" <diesql@googlemail.com> writes:
> Is it possible to somehow JOIN using the <<= and
> = network operators?

Sure, why not?

You probably won't get a plan that's smarter than a nestloop, but I
imagine you'll have few enough rows in network_names that it won't
matter.

One potential problem is if there could be overlapping entries in
network_names --- the join would give you multiple rows for any
log entry that matches more than one network_names entry. You
could fix that using a subselect with LIMIT 1, as long as you
weren't too picky about which name got reported.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

In article <b0a4f3350710251122y10648d4id322f12a81001c07@mail. gmail.com>,
"Jamie Tufnell" <diesql@googlemail.com> writes:

> Hi,
> I am storing a log of HTTP requests in a database table (including IP address):

> http_log: id(PK), path, time, ip

> I have another table that contains CIDR ranges and names for them:

> network_names: id(PK), cidr, name

> Some example data for both tables:

> network_names:
> 1, 192.168.0.0/24, 'Engineering'
> 2, 192.168.1.0/24, 'Media'
> 3, 192.168.2.0/24, 'Engineering'
> 4, 192.168.3.0/24, 'Accounting'
> 5, 192.168.4.0/24, 'Engineering'
> 6, 10.0.0.0/8, 'Engineering'

> http_log:
> 1, '/index.html', 110000001, 192.168.0.47/32
> 2, '/index.html', 110000023, 200.1.2.3/32
> 3, '/index.html', 110000059, 1.2.3.4/32
> 4, '/index.html', 110000232, 192.168.2.1/32
> 5, '/index.html', 113919102, 192.168.1.39/32
> 6, '/index.html', 129101293, 10.2.2.4/32
> 7, '/index.html', 132828282, 192.168.4.2/32

> Now, in trying to produce a report on this data, I've come up against an
> interesting (to me at least!) problem..

> I basically want the same output as in http_log, but substituting the IP with
> the network name where available, i.e:

> 1, '/index.html', 110000001, Engineering
> 2, '/index.html', 110000023, 200.1.2.3/32
> 3, '/index.html', 110000059, 1.2.3.4/32
> 4, '/index.html', 110000232, Engineering
> 5, '/index.html', 113919102, Media
> 6, '/index.html', 129101293, Engineering
> 7, '/index.html', 132828282, Engineering

> I'm wondering what the best way of doing this is (considering that http_log
> could have >100000 rows) Is it possible to somehow JOIN using the <<= and >>=
> network operators?

There are PostgreSQL builtin functions for that, but I think they are
unable to use indexes. I use http://pgfoundry.org/projects/ip4r/
and I think it's the best thing since the invention of sliced bread ;-)


---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

On 10/26/07, Harald Fuchs <hf0217x@protecting.net> wrote:
> There are PostgreSQL builtin functions for that, but I think they are
> unable to use indexes. I use http://pgfoundry.org/projects/ip4r/
> and I think it's the best thing since the invention of sliced bread ;-)

Yes:

ip4r('ip') <<= ip4r(cidr('cidr'))

--
Jonah H. Harris, Sr. Software Architect | phone: 732.331.1324
EnterpriseDB Corporation | fax: 732.331.1301
499 Thornall Street, 2nd Floor | jonah.harris@enterprisedb.com
Edison, NJ 08837 | http://www.enterprisedb.com/

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq