Ip type and support for a port?


(Chris Neal) #1

Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That
doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good feature
to add support for to this type?

Thanks!
Chris

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #2

Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is used
to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That
doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good
feature to add support for to this type?

Thanks!
Chris

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Chris Neal) #3

Hi Jorg,

Here's an example:

An ip field named as 'host' in mapping.
"host": {
"type" :"ip"
},

Then I add below to ES
POST /test_index/test
{
"host":"127.0.0.1:80 http://127.0.0.1/"
}

ES returns this error
{
"error": "MapperParsingException[failed to parse [host]]; nested:
ElasticsearchIllegalArgumentException[failed to parse ip [127.0.0.1:80]];
nested: NumberFormatException[For input string: "1:80"]; ",
"status": 400
}

If I only post IP without port like this
POST /test_index/test
{
"host":"127.0.0.1"
}
ES return success.

Does that help explain the issue?
Thanks!
Chris

On Wed, Jul 30, 2014 at 4:35 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is
used to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That
doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good
feature to add support for to this type?

Thanks!
Chris

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #4

Yes. :80 should not be part of the IP. It's a port.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 31 juillet 2014 à 16:56:29, Chris Neal (chris.neal@derbysoft.net) a écrit:

Hi Jorg,

Here's an example:

An ip field named as 'host' in mapping.
"host": {
"type" :"ip"
},

Then I add below to ES
POST /test_index/test
{
"host":"127.0.0.1:80"
}

ES returns this error
{
"error": "MapperParsingException[failed to parse [host]]; nested: ElasticsearchIllegalArgumentException[failed to parse ip [127.0.0.1:80]]; nested: NumberFormatException[For input string: "1:80"]; ",
"status": 400
}

If I only post IP without port like this
POST /test_index/test
{
"host":"127.0.0.1"
}
ES return success.

Does that help explain the issue?
Thanks!
Chris

On Wed, Jul 30, 2014 at 4:35 PM, joergprante@gmail.com joergprante@gmail.com wrote:
Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is used to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal chris.neal@derbysoft.net wrote:
Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good feature to add support for to this type?

Thanks!
Chris

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.53da6df3.75c6c33a.f0d0%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.


(Chris Neal) #5

Right, but I was more wondering about the use case of supporting a port in
some fashion, in some type?
Or is a string type the best type for holding an ip:port combination?

Thanks for the help :slight_smile:

On Thu, Jul 31, 2014 at 11:25 AM, David Pilato david@pilato.fr wrote:

Yes. :80 should not be part of the IP. It's a port.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr

Le 31 juillet 2014 à 16:56:29, Chris Neal (chris.neal@derbysoft.net) a
écrit:

Hi Jorg,

Here's an example:

An ip field named as 'host' in mapping.
"host": {
"type" :"ip"
},

Then I add below to ES
POST /test_index/test
{
"host":"127.0.0.1:80 http://127.0.0.1/"
}

ES returns this error
{
"error": "MapperParsingException[failed to parse [host]]; nested:
ElasticsearchIllegalArgumentException[failed to parse ip [127.0.0.1:80]];
nested: NumberFormatException[For input string: "1:80"]; ",
"status": 400
}

If I only post IP without port like this
POST /test_index/test
{
"host":"127.0.0.1"
}
ES return success.

Does that help explain the issue?
Thanks!
Chris

On Wed, Jul 30, 2014 at 4:35 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is
used to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That
doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good
feature to add support for to this type?

Thanks!
Chris

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.53da6df3.75c6c33a.f0d0%40MacBook-Air-de-David.local
https://groups.google.com/d/msgid/elasticsearch/etPan.53da6df3.75c6c33a.f0d0%40MacBook-Air-de-David.local?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpiAi7F3f6YE_QP5VjtPDzx-bxtmW1PbpwYWtU2KN02U%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #6

Maybe it would help to implement a new ES data type for URI/URL?

Usage example:

POST /test/test
{
"url" : "http://127.0.0.1:80"
}

Jörg

On Thu, Jul 31, 2014 at 4:56 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hi Jorg,

Here's an example:

An ip field named as 'host' in mapping.
"host": {
"type" :"ip"
},

Then I add below to ES
POST /test_index/test
{
"host":"127.0.0.1:80 http://127.0.0.1/"
}

ES returns this error
{
"error": "MapperParsingException[failed to parse [host]]; nested:
ElasticsearchIllegalArgumentException[failed to parse ip [127.0.0.1:80]];
nested: NumberFormatException[For input string: "1:80"]; ",
"status": 400
}

If I only post IP without port like this
POST /test_index/test
{
"host":"127.0.0.1"
}
ES return success.

Does that help explain the issue?
Thanks!
Chris

On Wed, Jul 30, 2014 at 4:35 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is
used to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That
doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good
feature to add support for to this type?

Thanks!
Chris

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHkTDvuqYVFBy-L-WvgWA9NjeiHBAuj%2BmmiLW%3DJkn1jgQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #7

I think you should handle that at index time and separate IP from port and index both in separate fields.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 31 juillet 2014 à 18:38:09, Chris Neal (chris.neal@derbysoft.net) a écrit:

Right, but I was more wondering about the use case of supporting a port in some fashion, in some type?
Or is a string type the best type for holding an ip:port combination?

Thanks for the help :slight_smile:

On Thu, Jul 31, 2014 at 11:25 AM, David Pilato david@pilato.fr wrote:
Yes. :80 should not be part of the IP. It's a port.

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 31 juillet 2014 à 16:56:29, Chris Neal (chris.neal@derbysoft.net) a écrit:

Hi Jorg,

Here's an example:

An ip field named as 'host' in mapping.
"host": {
"type" :"ip"
},

Then I add below to ES
POST /test_index/test
{
"host":"127.0.0.1:80"
}

ES returns this error
{
"error": "MapperParsingException[failed to parse [host]]; nested: ElasticsearchIllegalArgumentException[failed to parse ip [127.0.0.1:80]]; nested: NumberFormatException[For input string: "1:80"]; ",
"status": 400
}

If I only post IP without port like this
POST /test_index/test
{
"host":"127.0.0.1"
}
ES return success.

Does that help explain the issue?
Thanks!
Chris

On Wed, Jul 30, 2014 at 4:35 PM, joergprante@gmail.com joergprante@gmail.com wrote:
Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is used to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal chris.neal@derbysoft.net wrote:
Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good feature to add support for to this type?

Thanks!
Chris

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.53da6df3.75c6c33a.f0d0%40MacBook-Air-de-David.local.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpiAi7F3f6YE_QP5VjtPDzx-bxtmW1PbpwYWtU2KN02U%2Bg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.53da7154.520eedd1.f0d0%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.


(David Pilato) #8

I don't see the value we can add here.
For example, does a Range query on URL make sense?

If it's only for breaking the string into two subfields at index time, it can be done by a plugin like the mapper attachment. Although, I'd prefer using logstash for example to do that before sending the index request to ES.

Just my 0.05 cents here :slight_smile:

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet | @elasticsearchfr

Le 31 juillet 2014 à 18:39:33, joergprante@gmail.com (joergprante@gmail.com) a écrit:

Maybe it would help to implement a new ES data type for URI/URL?

Usage example:

POST /test/test
{
"url" : "http://127.0.0.1:80"
}

Jörg

On Thu, Jul 31, 2014 at 4:56 PM, Chris Neal chris.neal@derbysoft.net wrote:
Hi Jorg,

Here's an example:

An ip field named as 'host' in mapping.
"host": {
"type" :"ip"
},

Then I add below to ES
POST /test_index/test
{
"host":"127.0.0.1:80"
}

ES returns this error
{
"error": "MapperParsingException[failed to parse [host]]; nested: ElasticsearchIllegalArgumentException[failed to parse ip [127.0.0.1:80]]; nested: NumberFormatException[For input string: "1:80"]; ",
"status": 400
}

If I only post IP without port like this
POST /test_index/test
{
"host":"127.0.0.1"
}
ES return success.

Does that help explain the issue?
Thanks!
Chris

On Wed, Jul 30, 2014 at 4:35 PM, joergprante@gmail.com joergprante@gmail.com wrote:
Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is used to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal chris.neal@derbysoft.net wrote:
Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good feature to add support for to this type?

Thanks!
Chris

You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHkTDvuqYVFBy-L-WvgWA9NjeiHBAuj%2BmmiLW%3DJkn1jgQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/etPan.53da728f.23f9c13c.f0d0%40MacBook-Air-de-David.local.
For more options, visit https://groups.google.com/d/optout.


(Jörg Prante) #9

Of course, range queries make much sense. For example, over schemes, IP
ranges /subnets, or subdomains, or host names.

URL/URIs must also be canonicalized to make them comparable which is a
non-trivial issue, not resolvable by regex/pattern parsing.

URL/URIs data types are a building block for web crawlers.

I do not see Logstash being a webcrawler such as Nutch, Heritrix, or
Crawler4j

My 2¢s.

Jörg

On Thu, Jul 31, 2014 at 6:45 PM, David Pilato david@pilato.fr wrote:

I don't see the value we can add here.
For example, does a Range query on URL make sense?

If it's only for breaking the string into two subfields at index time, it
can be done by a plugin like the mapper attachment. Although, I'd prefer
using logstash for example to do that before sending the index request to
ES.

Just my 0.05 cents here :slight_smile:

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr

Le 31 juillet 2014 à 18:39:33, joergprante@gmail.com (
joergprante@gmail.com) a écrit:

Maybe it would help to implement a new ES data type for URI/URL?

Usage example:

POST /test/test
{
"url" : "http://127.0.0.1:80"
}

Jörg

On Thu, Jul 31, 2014 at 4:56 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hi Jorg,

Here's an example:

An ip field named as 'host' in mapping.
"host": {
"type" :"ip"
},

Then I add below to ES
POST /test_index/test
{
"host":"127.0.0.1:80 http://127.0.0.1/"
}

ES returns this error
{
"error": "MapperParsingException[failed to parse [host]]; nested:
ElasticsearchIllegalArgumentException[failed to parse ip [127.0.0.1:80]];
nested: NumberFormatException[For input string: "1:80"]; ",
"status": 400
}

If I only post IP without port like this
POST /test_index/test
{
"host":"127.0.0.1"
}
ES return success.

Does that help explain the issue?
Thanks!
Chris

On Wed, Jul 30, 2014 at 4:35 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is
used to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That
doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good
feature to add support for to this type?

Thanks!
Chris

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com?utm_medium=email&utm_source=footer.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHkTDvuqYVFBy-L-WvgWA9NjeiHBAuj%2BmmiLW%3DJkn1jgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHkTDvuqYVFBy-L-WvgWA9NjeiHBAuj%2BmmiLW%3DJkn1jgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.53da728f.23f9c13c.f0d0%40MacBook-Air-de-David.local
https://groups.google.com/d/msgid/elasticsearch/etPan.53da728f.23f9c13c.f0d0%40MacBook-Air-de-David.local?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHPE6gbsdJL7yY6OFQw8CZNjgtMPf0w8%3DB9i4RKD-Cn8w%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(Chris Neal) #10

Thanks guys for the discussion. I'm glad we're talking it through!

For an immediate solution it is easy to parse off the port and and store it
separately. Longer term I do see value in a url/uri type as Jorg mentions.

Thanks again. Much appreciated!
Chris

On Thu, Jul 31, 2014 at 12:35 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Of course, range queries make much sense. For example, over schemes, IP
ranges /subnets, or subdomains, or host names.

URL/URIs must also be canonicalized to make them comparable which is a
non-trivial issue, not resolvable by regex/pattern parsing.

URL/URIs data types are a building block for web crawlers.

I do not see Logstash being a webcrawler such as Nutch, Heritrix, or
Crawler4j

My 2¢s.

Jörg

On Thu, Jul 31, 2014 at 6:45 PM, David Pilato david@pilato.fr wrote:

I don't see the value we can add here.
For example, does a Range query on URL make sense?

If it's only for breaking the string into two subfields at index time, it
can be done by a plugin like the mapper attachment. Although, I'd prefer
using logstash for example to do that before sending the index request to
ES.

Just my 0.05 cents here :slight_smile:

--
David Pilato | Technical Advocate | Elasticsearch.com
@dadoonet https://twitter.com/dadoonet | @elasticsearchfr
https://twitter.com/elasticsearchfr

Le 31 juillet 2014 à 18:39:33, joergprante@gmail.com (
joergprante@gmail.com) a écrit:

Maybe it would help to implement a new ES data type for URI/URL?

Usage example:

POST /test/test
{
"url" : "http://127.0.0.1:80"
}

Jörg

On Thu, Jul 31, 2014 at 4:56 PM, Chris Neal chris.neal@derbysoft.net
wrote:

Hi Jorg,

Here's an example:

An ip field named as 'host' in mapping.
"host": {
"type" :"ip"
},

Then I add below to ES
POST /test_index/test
{
"host":"127.0.0.1:80 http://127.0.0.1/"
}

ES returns this error
{
"error": "MapperParsingException[failed to parse [host]]; nested:
ElasticsearchIllegalArgumentException[failed to parse ip [127.0.0.1:80]];
nested: NumberFormatException[For input string: "1:80"]; ",
"status": 400
}

If I only post IP without port like this
POST /test_index/test
{
"host":"127.0.0.1"
}
ES return success.

Does that help explain the issue?
Thanks!
Chris

On Wed, Jul 30, 2014 at 4:35 PM, joergprante@gmail.com <
joergprante@gmail.com> wrote:

Can you give an example what you mean by IP ports?

Transport protocols like TCP has ports, but IP (Internet addresses) is
used to address hosts on a network.

Jörg

On Wed, Jul 30, 2014 at 11:02 PM, Chris Neal <chris.neal@derbysoft.net

wrote:

Hi all,

I'm trying to use the ip type in ES, but my IPs also have ports. That
doesn't seem to be supported, which was a bit of a surprise!

Does anyone know of a way to do this? Or does it sound like a good
feature to add support for to this type?

Thanks!
Chris

You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpgEWdxZcaFe1wmSWtjRfXqxvUC0vbq8bfE0mC7DusS_%2Bw%40mail.gmail.com?utm_medium=email&utm_source=footer
.
For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoFKah6pkCWgKZYASM%3DfxyBq8waPQjyCBHj2aZaAeyiRwQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google
Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send
an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAND3DpjhbQunFsT8gi0bspkNDmj9u5kOyhohoAuEX20P0rcBkQ%40mail.gmail.com?utm_medium=email&utm_source=footer.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHkTDvuqYVFBy-L-WvgWA9NjeiHBAuj%2BmmiLW%3DJkn1jgQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHkTDvuqYVFBy-L-WvgWA9NjeiHBAuj%2BmmiLW%3DJkn1jgQ%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/etPan.53da728f.23f9c13c.f0d0%40MacBook-Air-de-David.local
https://groups.google.com/d/msgid/elasticsearch/etPan.53da728f.23f9c13c.f0d0%40MacBook-Air-de-David.local?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHPE6gbsdJL7yY6OFQw8CZNjgtMPf0w8%3DB9i4RKD-Cn8w%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CAKdsXoHPE6gbsdJL7yY6OFQw8CZNjgtMPf0w8%3DB9i4RKD-Cn8w%40mail.gmail.com?utm_medium=email&utm_source=footer
.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAND3DpheG2foOQJ2p89C_De%3DpWPYcOFMQvMdDEGJpCqTpZE45g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


(system) #11