ES attempting to parse dates automatically (possibly Hadoop related)

Brian_Stempin · March 19, 2014, 9:14pm

Hi List,
I have an ES cluster that takes in some data from our logs. We use Hadoop
to parse the individual log entries into JSON strings, which does a bulk
insert using ES's output format. For whatever reason, ES attempts to parse
base64 strings as a dates and fails. Here's a line from one of my Hadoop
logs:

java.lang.IllegalStateException: Found unrecoverable error [Bad Request(400) - MapperParsingException[failed to parse [csUriParams.d]]; nested: MapperParsingException[failed to parse date field [REDACTED BASE64 STRING], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "Y2lkPURFJml0ZW1zPWE2NTJjLXgxZTFj..."]; ]; Bailing out..

at org.elasticsearch.hadoop.rest.RestClient.retryFailedEntries(RestClient.java:145)

at org.elasticsearch.hadoop.rest.RestClient.bulk(RestClient.java:120)

at org.elasticsearch.hadoop.rest.RestRepository.sendBatch(RestRepository.java:147)

<SNIP>

csUriParams.d does not appear in my mapping, so I never explicitly asked
for it to be treated as a date.

Any idea why ES is trying to treat it as a date?

Thanks,
Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

costin · March 19, 2014, 9:35pm

Hi,

How do you pass the json to es-hadoop? Do you have an example? By the way, you can enable TRACE logging on
org.elasticsearch.hadoop and see everything that es-hadoop does, including the data that goes over the wire.
My guess is that the conversion of logs to JSON creates some extra artifacts which are later on interpreted as Writable
object (instead of raw JSON) by ES Hadoop.
Make sure you tell es-hadoop that its source it's json (through es.input.json set to true).
The logs will likely confirm (or not) the above

Cheers,

On 3/19/14 11:14 PM, Brian Stempin wrote:

Hi List,
I have an ES cluster that takes in some data from our logs. We use Hadoop to parse the individual log entries into JSON
strings, which does a bulk insert using ES's output format. For whatever reason, ES attempts to parse base64 strings as
a dates and fails. Here's a line from one of my Hadoop logs:
java.lang.IllegalStateException: Found unrecoverable error [Bad Request(400) - MapperParsingException[failed to parse [csUriParams.d]]; nested: MapperParsingException[failed to parse date field [REDACTED BASE64 STRING], tried both date format [dateOptionalTime], and timestamp number with locale []]; nested: IllegalArgumentException[Invalid format: "Y2lkPURFJml0ZW1zPWE2NTJjLXgxZTFj..."]; ]; Bailing out..

	at org.elasticsearch.hadoop.rest.RestClient.retryFailedEntries(RestClient.java:145)

	at org.elasticsearch.hadoop.rest.RestClient.bulk(RestClient.java:120)

	at org.elasticsearch.hadoop.rest.RestRepository.sendBatch(RestRepository.java:147)

	<SNIP>
csUriParams.d does not appear in my mapping, so I never explicitly asked for it to be treated as a date.

Any idea why ES is trying to treat it as a date?

Thanks,
Brian

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com
https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.
--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/532A0D9C.7010401%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Brian_Stempin · March 20, 2014, 2:48pm

Hi,
All I'm doing is building a map and passing that to Gson for serialization.
A snippet from my map method:

logEntryMap.put("cs(User-Agent)", values[9]);
context.write(NullWritable.get(), new Text(gson.toJson(logEntryMap)));

values is a String array. Everything that goes into the map that gets
serialized is a string.

I do have es.input.json set to true. This failure doesn't occur until

100,000,000 records are in the index, so its happening late in the load
process. The part that I find strange is that the field in question isn't
in my mapping, and I've not touched the default mapping. I'm not sure why
it would try to parse it as anything other than a string.

I'll turn on TRACE logging and see what happens.

Brian

On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau costin.leau@gmail.com wrote:

Hi,

How do you pass the json to es-hadoop? Do you have an example? By the way,
you can enable TRACE logging on org.elasticsearch.hadoop and see everything
that es-hadoop does, including the data that goes over the wire.
My guess is that the conversion of logs to JSON creates some extra
artifacts which are later on interpreted as Writable object (instead of raw
JSON) by ES Hadoop.
Make sure you tell es-hadoop that its source it's json (through
es.input.json set to true).
The logs will likely confirm (or not) the above

Cheers,

On 3/19/14 11:14 PM, Brian Stempin wrote:
Hi List,
I have an ES cluster that takes in some data from our logs. We use
Hadoop to parse the individual log entries into JSON
strings, which does a bulk insert using ES's output format. For whatever
reason, ES attempts to parse base64 strings as
a dates and fails. Here's a line from one of my Hadoop logs:
java.lang.IllegalStateException: Found unrecoverable error [Bad
Request(400) - MapperParsingException[failed to parse [csUriParams.d]];
nested: MapperParsingException[failed to parse date field [REDACTED BASE64
STRING], tried both date format [dateOptionalTime], and timestamp number
with locale ]; nested: IllegalArgumentException[Invalid format: "
Y2lkPURFJml0ZW1zPWE2NTJjLXgxZTFj..."]; ]; Bailing out..
    at org.elasticsearch.hadoop.rest.RestClient.retryFailedEntries(
RestClient.java:145)
    at org.elasticsearch.hadoop.rest.RestClient.bulk(RestClient.
java:120)
    at org.elasticsearch.hadoop.rest.RestRepository.sendBatch(
RestRepository.java:147)
    <SNIP>
csUriParams.d does not appear in my mapping, so I never explicitly asked
for it to be treated as a date.

Any idea why ES is trying to treat it as a date?

Thanks,
Brian

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.

To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.
--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/_iE0t92CUzA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/532A0D9C.7010401%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

costin · March 20, 2014, 3:18pm

My guess is that GSON adds the said field in its result. The base64 suggests that there's some binary data in the mix.

By the way, can you show up more of your code - any reason why you create the JSON yourself rather than just pass
logEntryMap to Es-Hadoop?
It can create the json for you - which is what I recommend; unless you have the JSON in HDFS, it's best to rely on
es-hadoop to do it instead of an external tool.

Cheers,

On 3/20/14 4:48 PM, Brian Stempin wrote:

Hi,
All I'm doing is building a map and passing that to Gson for serialization. A snippet from my map method:

logEntryMap.put("cs(User-Agent)", values[9]);
context.write(NullWritable.get(), new Text(gson.toJson(logEntryMap)));

values is a String array. Everything that goes into the map that gets serialized is a string.

I do have es.input.json set to true. This failure doesn't occur until >100,000,000 records are in the index, so its
happening late in the load process. The part that I find strange is that the field in question isn't in my mapping, and
I've not touched the default mapping. I'm not sure why it would try to parse it as anything other than a string.

I'll turn on TRACE logging and see what happens.

Brian

On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com> wrote:

Hi,

How do you pass the json to es-hadoop? Do you have an example? By the way, you can enable TRACE logging on
org.elasticsearch.hadoop and see everything that es-hadoop does, including the data that goes over the wire.
My guess is that the conversion of logs to JSON creates some extra artifacts which are later on interpreted as
Writable object (instead of raw JSON) by ES Hadoop.
Make sure you tell es-hadoop that its source it's json (through es.input.json set to true).
The logs will likely confirm (or not) the above :)

Cheers,


On 3/19/14 11:14 PM, Brian Stempin wrote:

    Hi List,
    I have an ES cluster that takes in some data from our logs.  We use Hadoop to parse the individual log entries
    into JSON
    strings, which does a bulk insert using ES's output format.  For whatever reason, ES attempts to parse base64
    strings as
    a dates and fails.  Here's a line from one of my Hadoop logs:

         java.lang.__IllegalStateException: Found unrecoverable error [Bad Request(400) -
    MapperParsingException[failed to parse [csUriParams.d]]; nested: MapperParsingException[failed to parse date
    field [REDACTED BASE64 STRING], tried both date format [dateOptionalTime], and timestamp number with locale []];
    nested: IllegalArgumentException[__Invalid format: "__Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT__Fj..."]; ]; Bailing out..

             at org.elasticsearch.hadoop.rest.__RestClient.retryFailedEntries(__RestClient.java:145)

             at org.elasticsearch.hadoop.rest.__RestClient.bulk(RestClient.__java:120)

             at org.elasticsearch.hadoop.rest.__RestRepository.sendBatch(__RestRepository.java:147)

             <SNIP>


    csUriParams.d does not appear in my mapping, so I never explicitly asked for it to be treated as a date.

    Any idea why ES is trying to treat it as a date?

    Thanks,
    Brian

    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.

    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
Costin


--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532A0D9C.7010401%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/532B06B1.9010206%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Brian_Stempin · March 20, 2014, 4:10pm

I have unit tests for this MR job, and they show that the JSON output is a
string as I'd expect, so Gson is most likely not the cause.

I'm hesitant to show more code (owned by the work-place), but I can
describe it a little bit further:

The mapper gets a W3C log entry
The log entry is broken into its components and put into document X
The request URL is then taken and broken down into its query
parameters and the key-value pairs are put into document Y
Some elements are then explicitly filtered from X and Y
Those two documents are placed inside of document Z, which is
ultimately what is serialized and sent to ES

We do have a base64 encoded param that we expect and handle appropriately.
In this case, someone most likely sent it as the wrong param name, hence
why its making its way into document Y without further processing. Since
its being sent as a name that's not listed in the mapping, I expect it to
just be treated as a string.

The only reason that I chose to go the Gson route vs building MapWritables
is that building MapWritables is terribly verbose. Also, it comes with the
overhead of having to wrap each String with a Text type, which just seems
silly. Using the built-in JSON serializer is just not convenient in this
case.

Brian

On Thu, Mar 20, 2014 at 11:18 AM, Costin Leau costin.leau@gmail.com wrote:

My guess is that GSON adds the said field in its result. The base64
suggests that there's some binary data in the mix.

By the way, can you show up more of your code - any reason why you create
the JSON yourself rather than just pass logEntryMap to Es-Hadoop?
It can create the json for you - which is what I recommend; unless you
have the JSON in HDFS, it's best to rely on es-hadoop to do it instead of
an external tool.

Cheers,

On 3/20/14 4:48 PM, Brian Stempin wrote:
Hi,
All I'm doing is building a map and passing that to Gson for
serialization. A snippet from my map method:

logEntryMap.put("cs(User-Agent)", values[9]);
context.write(NullWritable.get(), new Text(gson.toJson(logEntryMap)));

values is a String array. Everything that goes into the map that gets
serialized is a string.

I do have es.input.json set to true. This failure doesn't occur until

100,000,000 records are in the index, so its
happening late in the load process. The part that I find strange is that
the field in question isn't in my mapping, and
I've not touched the default mapping. I'm not sure why it would try to
parse it as anything other than a string.

I'll turn on TRACE logging and see what happens.

Brian

On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <costin.leau@gmail.com<mailto:
costin.leau@gmail.com>> wrote:
Hi,

How do you pass the json to es-hadoop? Do you have an example? By the
way, you can enable TRACE logging on
org.elasticsearch.hadoop and see everything that es-hadoop does,
including the data that goes over the wire.
My guess is that the conversion of logs to JSON creates some extra
artifacts which are later on interpreted as
Writable object (instead of raw JSON) by ES Hadoop.
Make sure you tell es-hadoop that its source it's json (through
es.input.json set to true).
The logs will likely confirm (or not) the above
Cheers,


On 3/19/14 11:14 PM, Brian Stempin wrote:

    Hi List,
    I have an ES cluster that takes in some data from our logs.  We
use Hadoop to parse the individual log entries
into JSON
strings, which does a bulk insert using ES's output format. For
whatever reason, ES attempts to parse base64
strings as
a dates and fails. Here's a line from one of my Hadoop logs:
         java.lang.__IllegalStateException: Found unrecoverable
error [Bad Request(400) -
    MapperParsingException[failed to parse [csUriParams.d]]; nested:
MapperParsingException[failed to parse date
field [REDACTED BASE64 STRING], tried both date format
[dateOptionalTime], and timestamp number with locale ];
nested: IllegalArgumentException[Invalid format: "
Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT__Fj..."]; ]; Bailing out..
             at org.elasticsearch.hadoop.rest.__RestClient.
retryFailedEntries(__RestClient.java:145)
             at org.elasticsearch.hadoop.rest.
__RestClient.bulk(RestClient.__java:120)
             at org.elasticsearch.hadoop.rest.
__RestRepository.sendBatch(__RestRepository.java:147)
             <SNIP>


    csUriParams.d does not appear in my mapping, so I never
explicitly asked for it to be treated as a date.
    Any idea why ES is trying to treat it as a date?

    Thanks,
    Brian

    --
    You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-_
_cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
email&utm_source=footer>>.
    For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.
--
Costin


--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__
topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@__googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532A0D9C._
_7010401%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532A0D9C.
7010401%40gmail.com>.
For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_
68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_
68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm_
medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.
--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/_iE0t92CUzA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/532B06B1.9010206%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

costin · March 20, 2014, 4:26pm

Then what you could do is to minimize the bulk size to say 100 documents, turn on logging and run your data through.
This way you can catch the 'special' document in the act.

As for expectations - Elasticsearch tries to guess the field type by looking at its value - it seems the base64 entry
looks like a date, hence the error. You can avoid this by defining the field (either directly or through a template) in
your mapping so it always gets mapped to a string.
As a rule of thumb, whenever you want full control over the index, mapping is the way to do it.

On 3/20/14 6:10 PM, Brian Stempin wrote:

I have unit tests for this MR job, and they show that the JSON output is a string as I'd expect, so Gson is most likely
not the cause.

I'm hesitant to show more code (owned by the work-place), but I can describe it a little bit further:

The mapper gets a W3C log entry
The log entry is broken into its components and put into document X
The request URL is then taken and broken down into its query parameters and the key-value pairs are put into document Y
Some elements are then explicitly filtered from X and Y
Those two documents are placed inside of document Z, which is ultimately what is serialized and sent to ES

We do have a base64 encoded param that we expect and handle appropriately. In this case, someone most likely sent it as
the wrong param name, hence why its making its way into document Y without further processing. Since its being sent as
a name that's not listed in the mapping, I expect it to just be treated as a string.

The only reason that I chose to go the Gson route vs building MapWritables is that building MapWritables is terribly
verbose. Also, it comes with the overhead of having to wrap each String with a Text type, which just seems silly.
Using the built-in JSON serializer is just not convenient in this case.

Brian

On Thu, Mar 20, 2014 at 11:18 AM, Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com> wrote:

My guess is that GSON adds the said field in its result. The base64 suggests that there's some binary data in the mix.

By the way, can you show up more of your code - any reason why you create the JSON yourself rather than just pass
logEntryMap to Es-Hadoop?
It can create the json for you - which is what I recommend; unless you have the JSON in HDFS, it's best to rely on
es-hadoop to do it instead of an external tool.

Cheers,


On 3/20/14 4:48 PM, Brian Stempin wrote:

    Hi,
    All I'm doing is building a map and passing that to Gson for serialization.  A snippet from my map method:

    logEntryMap.put("cs(User-__Agent)", values[9]);
    context.write(NullWritable.__get(), new Text(gson.toJson(logEntryMap))__);

    values[] is a String array.  Everything that goes into the map that gets serialized is a string.

    I do have es.input.json set to true.  This failure doesn't occur until >100,000,000 records are in the index, so its
    happening late in the load process.  The part that I find strange is that the field in question isn't in my
    mapping, and
    I've not touched the default mapping.  I'm not sure why it would try to parse it as anything other than a string.

    I'll turn on TRACE logging and see what happens.

    Brian


    On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__> wrote:

         Hi,

         How do you pass the json to es-hadoop? Do you have an example? By the way, you can enable TRACE logging on
         org.elasticsearch.hadoop and see everything that es-hadoop does, including the data that goes over the wire.
         My guess is that the conversion of logs to JSON creates some extra artifacts which are later on interpreted as
         Writable object (instead of raw JSON) by ES Hadoop.
         Make sure you tell es-hadoop that its source it's json (through es.input.json set to true).
         The logs will likely confirm (or not) the above :)

         Cheers,


         On 3/19/14 11:14 PM, Brian Stempin wrote:

             Hi List,
             I have an ES cluster that takes in some data from our logs.  We use Hadoop to parse the individual log
    entries
             into JSON
             strings, which does a bulk insert using ES's output format.  For whatever reason, ES attempts to parse
    base64
             strings as
             a dates and fails.  Here's a line from one of my Hadoop logs:

                  java.lang.____IllegalStateException: Found unrecoverable error [Bad Request(400) -

             MapperParsingException[failed to parse [csUriParams.d]]; nested: MapperParsingException[failed to parse
    date
             field [REDACTED BASE64 STRING], tried both date format [dateOptionalTime], and timestamp number with
    locale []];
             nested: IllegalArgumentException[____Invalid format: "____Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT____Fj..."]; ];
    Bailing out..

                      at org.elasticsearch.hadoop.rest.____RestClient.__retryFailedEntries(____RestClient.java:145)

                      at org.elasticsearch.hadoop.rest.____RestClient.bulk(RestClient.____java:120)

                      at org.elasticsearch.hadoop.rest.____RestRepository.sendBatch(____RestRepository.java:147)


                      <SNIP>


             csUriParams.d does not appear in my mapping, so I never explicitly asked for it to be treated as a date.

             Any idea why ES is trying to treat it as a date?

             Thanks,
             Brian

             --
             You received this message because you are subscribed to the Google Groups "elasticsearch" group.
             To unsubscribe from this group and stop receiving emails from it, send an email to
             elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch+____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B__unsubscribe@googlegroups.com> <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>>.


             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com?utm_medium=email&utm_source=footer>>>.

             For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


         --
         Costin


         --
         You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
         To unsubscribe from this topic, visit
    https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
         <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>.
         To unsubscribe from this group and all its topics, send an email to
    elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
         <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>.

         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532A0D9C.7010401%40gmail.com>>.

         For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.



    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm_medium=email&utm_source=footer>__>.

    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B06B1.9010206%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/532B16AF.7030701%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Brian_Stempin · March 20, 2014, 4:32pm

That's the problem -- it's a web log that contains a URL that could have
literally anything in it. Anyone could put a base64 value as a random
query parameter. I could have the M/R job ignore all fields that I don't
explicitly expect, but that's not very flexible and prevents me from
spotting possible abuse or user-error. Is there any way for me to disable
ES's type-guessing or to provide a default guess? I'd rather have ES
default to a string than to fail a M/R job because its type-guess was wrong.

Brian

On Thu, Mar 20, 2014 at 12:26 PM, Costin Leau costin.leau@gmail.com wrote:

Then what you could do is to minimize the bulk size to say 100 documents,
turn on logging and run your data through.
This way you can catch the 'special' document in the act.

As for expectations - Elasticsearch tries to guess the field type by
looking at its value - it seems the base64 entry looks like a date, hence
the error. You can avoid this by defining the field (either directly or
through a template) in your mapping so it always gets mapped to a string.
As a rule of thumb, whenever you want full control over the index, mapping
is the way to do it.

On 3/20/14 6:10 PM, Brian Stempin wrote:
I have unit tests for this MR job, and they show that the JSON output is
a string as I'd expect, so Gson is most likely
not the cause.

I'm hesitant to show more code (owned by the work-place), but I can
describe it a little bit further:

The mapper gets a W3C log entry

The log entry is broken into its components and put into document X

The request URL is then taken and broken down into its query
parameters and the key-value pairs are put into document Y

Some elements are then explicitly filtered from X and Y

Those two documents are placed inside of document Z, which is
ultimately what is serialized and sent to ES

We do have a base64 encoded param that we expect and handle
appropriately. In this case, someone most likely sent it as
the wrong param name, hence why its making its way into document Y
without further processing. Since its being sent as
a name that's not listed in the mapping, I expect it to just be treated
as a string.

The only reason that I chose to go the Gson route vs building
MapWritables is that building MapWritables is terribly
verbose. Also, it comes with the overhead of having to wrap each String
with a Text type, which just seems silly.
Using the built-in JSON serializer is just not convenient in this case.

Brian

On Thu, Mar 20, 2014 at 11:18 AM, Costin Leau <costin.leau@gmail.com<mailto:
costin.leau@gmail.com>> wrote:
My guess is that GSON adds the said field in its result. The base64
suggests that there's some binary data in the mix.
By the way, can you show up more of your code - any reason why you
create the JSON yourself rather than just pass
logEntryMap to Es-Hadoop?
It can create the json for you - which is what I recommend; unless
you have the JSON in HDFS, it's best to rely on
es-hadoop to do it instead of an external tool.
Cheers,


On 3/20/14 4:48 PM, Brian Stempin wrote:

    Hi,
    All I'm doing is building a map and passing that to Gson for
serialization. A snippet from my map method:
    logEntryMap.put("cs(User-__Agent)", values[9]);
    context.write(NullWritable.__get(), new
Text(gson.toJson(logEntryMap))__);
    values[] is a String array.  Everything that goes into the map
that gets serialized is a string.
    I do have es.input.json set to true.  This failure doesn't occur
until >100,000,000 records are in the index, so its
happening late in the load process. The part that I find strange
is that the field in question isn't in my
mapping, and
I've not touched the default mapping. I'm not sure why it would
try to parse it as anything other than a string.
    I'll turn on TRACE logging and see what happens.

    Brian


    On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <
costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>__>
wrote:
         Hi,

         How do you pass the json to es-hadoop? Do you have an
example? By the way, you can enable TRACE logging on
org.elasticsearch.hadoop and see everything that es-hadoop
does, including the data that goes over the wire.
My guess is that the conversion of logs to JSON creates some
extra artifacts which are later on interpreted as
Writable object (instead of raw JSON) by ES Hadoop.
Make sure you tell es-hadoop that its source it's json
(through es.input.json set to true).
The logs will likely confirm (or not) the above
         Cheers,


         On 3/19/14 11:14 PM, Brian Stempin wrote:

             Hi List,
             I have an ES cluster that takes in some data from our
logs. We use Hadoop to parse the individual log
entries
into JSON
strings, which does a bulk insert using ES's output
format. For whatever reason, ES attempts to parse
base64
strings as
a dates and fails. Here's a line from one of my Hadoop
logs:
                  java.lang.____IllegalStateException: Found
unrecoverable error [Bad Request(400) -
             MapperParsingException[failed to parse [csUriParams.d]];
nested: MapperParsingException[failed to parse
date
field [REDACTED BASE64 STRING], tried both date format
[dateOptionalTime], and timestamp number with
locale ];
nested: IllegalArgumentException[____Invalid format:
"____Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT____Fj..."]; ];
Bailing out..
                      at org.elasticsearch.hadoop.rest.
____RestClient.__retryFailedEntries(____RestClient.java:145)
                      at org.elasticsearch.hadoop.rest.
____RestClient.bulk(RestClient.____java:120)
                      at org.elasticsearch.hadoop.rest.
____RestRepository.sendBatch(____RestRepository.java:147)
                      <SNIP>


             csUriParams.d does not appear in my mapping, so I never
explicitly asked for it to be treated as a date.
             Any idea why ES is trying to treat it as a date?

             Thanks,
             Brian

             --
             You received this message because you are subscribed to
the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>>
<mailto:elasticsearch+____unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>>.
             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer>
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
email&utm_source=footer>>>.
             For more options, visit https://groups.google.com/d/__
__optout https://groups.google.com/d/__optout
    <https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.
         --
         Costin


         --
         You received this message because you are subscribed to a
topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/____topic/elasticsearch/_____
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>
         <https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>>.
To unsubscribe from this group and all its topics, send an
email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>.
         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/
532A0D9C.____7010401%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.
__7010401%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.
__7010401%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532A0D9C.
7010401%40gmail.com>>.
         For more options, visit https://groups.google.com/d/__
__optout https://groups.google.com/d/__optout
    <https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.
    --
    You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@_googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_
68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com>
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?
utm___medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_
68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm
medium=email&utm_source=footer>>.
    For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.
--
Costin

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__
topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@__googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532B06B1._
_9010206%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B06B1.
9010206%40gmail.com>.
For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%
40mail.gmail.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.
--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/_iE0t92CUzA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/532B16AF.7030701%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

costin · March 20, 2014, 6:12pm

Sure - take a look at dynamic_templates - you define one under your index/type and specify the match for your field.
You can either define the mapping for the fields that you want and leave the so-called catch-all (*) directive last or,
if you have type of naming convention, use that:

On 3/20/14 6:32 PM, Brian Stempin wrote:

That's the problem -- it's a web log that contains a URL that could have literally anything in it. Anyone could put a
base64 value as a random query parameter. I could have the M/R job ignore all fields that I don't explicitly expect,
but that's not very flexible and prevents me from spotting possible abuse or user-error. Is there any way for me to
disable ES's type-guessing or to provide a default guess? I'd rather have ES default to a string than to fail a M/R job
because its type-guess was wrong.

Brian

On Thu, Mar 20, 2014 at 12:26 PM, Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com> wrote:

Then what you could do is to minimize the bulk size to say 100 documents, turn on logging and run your data through.
This way you can catch the 'special' document in the act.

As for expectations - Elasticsearch tries to guess the field type by looking at its value - it seems the base64
entry looks like a date, hence the error. You can avoid this by defining the field (either directly or through a
template) in your mapping so it always gets mapped to a string.
As a rule of thumb, whenever you want full control over the index, mapping is the way to do it.



On 3/20/14 6:10 PM, Brian Stempin wrote:

    I have unit tests for this MR job, and they show that the JSON output is a string as I'd expect, so Gson is most
    likely
    not the cause.

    I'm hesitant to show more code (owned by the work-place), but I can describe it a little bit further:

       * The mapper gets a W3C log entry
       * The log entry is broken into its components and put into document X
       * The request URL is then taken and broken down into its query parameters and the key-value pairs are put
    into document Y
       * Some elements are then explicitly filtered from X and Y
       * Those two documents are placed inside of document Z, which is ultimately what is serialized and sent to ES


    We do have a base64 encoded param that we expect and handle appropriately.  In this case, someone most likely
    sent it as
    the wrong param name, hence why its making its way into document Y without further processing.  Since its being
    sent as
    a name that's not listed in the mapping, I expect it to just be treated as a string.

    The only reason that I chose to go the Gson route vs building MapWritables is that building MapWritables is terribly
    verbose.  Also, it comes with the overhead of having to wrap each String with a Text type, which just seems silly.
       Using the built-in JSON serializer is just not convenient in this case.

    Brian


    On Thu, Mar 20, 2014 at 11:18 AM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__> wrote:

         My guess is that GSON adds the said field in its result. The base64 suggests that there's some binary data
    in the mix.

         By the way, can you show up more of your code - any reason why you create the JSON yourself rather than
    just pass
         logEntryMap to Es-Hadoop?
         It can create the json for you - which is what I recommend; unless you have the JSON in HDFS, it's best to
    rely on
         es-hadoop to do it instead of an external tool.

         Cheers,


         On 3/20/14 4:48 PM, Brian Stempin wrote:

             Hi,
             All I'm doing is building a map and passing that to Gson for serialization.  A snippet from my map method:

             logEntryMap.put("cs(User-____Agent)", values[9]);
             context.write(NullWritable.____get(), new Text(gson.toJson(logEntryMap))____);


             values[] is a String array.  Everything that goes into the map that gets serialized is a string.

             I do have es.input.json set to true.  This failure doesn't occur until >100,000,000 records are in the
    index, so its
             happening late in the load process.  The part that I find strange is that the field in question isn't in my
             mapping, and
             I've not touched the default mapping.  I'm not sure why it would try to parse it as anything other than
    a string.

             I'll turn on TRACE logging and see what happens.

             Brian


             On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>>__>__> wrote:

                  Hi,

                  How do you pass the json to es-hadoop? Do you have an example? By the way, you can enable TRACE
    logging on
                  org.elasticsearch.hadoop and see everything that es-hadoop does, including the data that goes over
    the wire.
                  My guess is that the conversion of logs to JSON creates some extra artifacts which are later on
    interpreted as
                  Writable object (instead of raw JSON) by ES Hadoop.
                  Make sure you tell es-hadoop that its source it's json (through es.input.json set to true).
                  The logs will likely confirm (or not) the above :)

                  Cheers,


                  On 3/19/14 11:14 PM, Brian Stempin wrote:

                      Hi List,
                      I have an ES cluster that takes in some data from our logs.  We use Hadoop to parse the
    individual log
             entries
                      into JSON
                      strings, which does a bulk insert using ES's output format.  For whatever reason, ES attempts
    to parse
             base64
                      strings as
                      a dates and fails.  Here's a line from one of my Hadoop logs:

                           java.lang.______IllegalStateException: Found unrecoverable error [Bad Request(400) -


                      MapperParsingException[failed to parse [csUriParams.d]]; nested: MapperParsingException[failed
    to parse
             date
                      field [REDACTED BASE64 STRING], tried both date format [dateOptionalTime], and timestamp
    number with
             locale []];
                      nested: IllegalArgumentException[______Invalid format:
    "______Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT______Fj..."]; ];
             Bailing out..

                               at
    org.elasticsearch.hadoop.rest.______RestClient.____retryFailedEntries(______RestClient.java:145)

                               at org.elasticsearch.hadoop.rest.______RestClient.bulk(__RestClient.____java:120)

                               at
    org.elasticsearch.hadoop.rest.______RestRepository.sendBatch(______RestRepository.java:147)



                               <SNIP>


                      csUriParams.d does not appear in my mapping, so I never explicitly asked for it to be treated
    as a date.

                      Any idea why ES is trying to treat it as a date?

                      Thanks,
                      Brian

                      --
                      You received this message because you are subscribed to the Google Groups "elasticsearch" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to
                      elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com>
    <http://googlegroups.com>
             <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>
                      <mailto:elasticsearch+______unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B____unsubscribe@googlegroups.com>
             <mailto:elasticsearch%2B____unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B__unsubscribe@googlegroups.com>>
    <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>>.



                      To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>


    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>>>


    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>>



    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com?utm_medium=email&utm_source=footer>>>>.

                      For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>

             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.


                  --
                  Costin


                  --
                  You received this message because you are subscribed to a topic in the Google Groups
    "elasticsearch" group.
                  To unsubscribe from this topic, visit
    https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>

                  <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>>.
                  To unsubscribe from this group and all its topics, send an email to
             elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com> <http://googlegroups.com>
                  <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>.


                  To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>>

                  <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>
             <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532A0D9C.7010401%40gmail.com>>>.

                  For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>

             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.



             --
             You received this message because you are subscribed to the Google Groups "elasticsearch" group.
             To unsubscribe from this group and stop receiving emails from it, send an email to
             elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch+____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B__unsubscribe@googlegroups.com> <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>>.
             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm_medium=email&utm_source=footer>__>__>.


             For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


         --
         Costin

         --
         You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
         To unsubscribe from this topic, visit
    https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
         <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>.
         To unsubscribe from this group and all its topics, send an email to
    elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
         <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>.
         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532B06B1.9010206%40gmail.com>>.


         For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532B16AF.__7030701%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B16AF.7030701%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/532B2F71.7060302%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Brian_Stempin · March 21, 2014, 6:37pm

Hi Costin,
Thanks for the response -- that sounds like what I need. I re-ran my job
with a mapping that included dynamic_templates, and I'm still having
issues. Here's my mapping...am I using dynamic_templates correctly?

Thanks for all of your help so far,
Brian

curl -XPUT 'http://localhost:9200/dau/log/_mapping?pretty' -d '
{
"log" : {
"properties" : {
"csUriParams" : {
"type" : "object",
"dynamic_templates" : [{
"defualt" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}],
"properties" : {
"aoi" : {"type" : "date", "ignore_malformed" : "true"},
"zoneid" : {"type" : "long", "ignore_malformed" : "true"},
"ftr" : {"type" : "date", "ignore_malformed" : "true"},
"pid" : {"type" : "string", "index" : "not_analyzed"},
"cid" : {"type" : "string", "index" : "not_analyzed"},
"ext" : {"type" : "string", "index" : "not_analyzed"},
"systemid" : {"type" : "string", "index" :
"not_analyzed"}
}
},
"rawLog" : {
"type" : "object",
"dynamic_templates" : [{
"defualt" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}],
"properties" : {
"datetime" : {"type" : "date", "format" :
"date_hour_minute_second", "ignore_malformed" : "true"},
"cs(User-Agent)" : {"type" : "string", "index" :
"not_analyzed"},
"cs-ip" : {"type" : "string", "index" : "not_analyzed"},
"cs-uri" : {"type" : "string"},
"os" : {"type" : "string", "index" : "not_analyzed"},
"browser" : {"type" : "string", "index" :
"not_analyzed"}
}
}
}
}
}
'

On Thu, Mar 20, 2014 at 2:12 PM, Costin Leau costin.leau@gmail.com wrote:

Sure - take a look at dynamic_templates - you define one under your
index/type and specify the match for your field.
You can either define the mapping for the fields that you want and leave
the so-called catch-all (*) directive last or, if you have type of naming
convention, use that:

Elasticsearch Platform — Find real-time answers at scale | Elastic
reference/current/mapping-root-object-type.html#_dynamic_templates

On 3/20/14 6:32 PM, Brian Stempin wrote:
That's the problem -- it's a web log that contains a URL that could have
literally anything in it. Anyone could put a
base64 value as a random query parameter. I could have the M/R job
ignore all fields that I don't explicitly expect,
but that's not very flexible and prevents me from spotting possible abuse
or user-error. Is there any way for me to
disable ES's type-guessing or to provide a default guess? I'd rather
have ES default to a string than to fail a M/R job
because its type-guess was wrong.

Brian

On Thu, Mar 20, 2014 at 12:26 PM, Costin Leau <costin.leau@gmail.com<mailto:
costin.leau@gmail.com>> wrote:
Then what you could do is to minimize the bulk size to say 100
documents, turn on logging and run your data through.
This way you can catch the 'special' document in the act.
As for expectations - Elasticsearch tries to guess the field type by
looking at its value - it seems the base64
entry looks like a date, hence the error. You can avoid this by
defining the field (either directly or through a
template) in your mapping so it always gets mapped to a string.
As a rule of thumb, whenever you want full control over the index,
mapping is the way to do it.
On 3/20/14 6:10 PM, Brian Stempin wrote:

    I have unit tests for this MR job, and they show that the JSON
output is a string as I'd expect, so Gson is most
likely
not the cause.
    I'm hesitant to show more code (owned by the work-place), but I
can describe it a little bit further:
       * The mapper gets a W3C log entry
       * The log entry is broken into its components and put into
document X
* The request URL is then taken and broken down into its query
parameters and the key-value pairs are put
into document Y
* Some elements are then explicitly filtered from X and Y
* Those two documents are placed inside of document Z, which
is ultimately what is serialized and sent to ES
    We do have a base64 encoded param that we expect and handle
appropriately. In this case, someone most likely
sent it as
the wrong param name, hence why its making its way into document
Y without further processing. Since its being
sent as
a name that's not listed in the mapping, I expect it to just be
treated as a string.
    The only reason that I chose to go the Gson route vs building
MapWritables is that building MapWritables is terribly
verbose. Also, it comes with the overhead of having to wrap each
String with a Text type, which just seems silly.
Using the built-in JSON serializer is just not convenient in
this case.
    Brian


    On Thu, Mar 20, 2014 at 11:18 AM, Costin Leau <
costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>__>
wrote:
         My guess is that GSON adds the said field in its result. The
base64 suggests that there's some binary data
in the mix.
         By the way, can you show up more of your code - any reason
why you create the JSON yourself rather than
just pass
logEntryMap to Es-Hadoop?
It can create the json for you - which is what I recommend;
unless you have the JSON in HDFS, it's best to
rely on
es-hadoop to do it instead of an external tool.
         Cheers,


         On 3/20/14 4:48 PM, Brian Stempin wrote:

             Hi,
             All I'm doing is building a map and passing that to Gson
for serialization. A snippet from my map method:
             logEntryMap.put("cs(User-____Agent)", values[9]);
             context.write(NullWritable.____get(), new
Text(gson.toJson(logEntryMap))____);
             values[] is a String array.  Everything that goes into
the map that gets serialized is a string.
             I do have es.input.json set to true.  This failure
doesn't occur until >100,000,000 records are in the
index, so its
happening late in the load process. The part that I
find strange is that the field in question isn't in my
mapping, and
I've not touched the default mapping. I'm not sure why
it would try to parse it as anything other than
a string.
             I'll turn on TRACE logging and see what happens.

             Brian


             On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <
costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>
<mailto:costin.leau@gmail.com <mailto:
costin.leau@gmail.com> <mailto:costin.leau@gmail.com
mailto:costin.leau@gmail.com>>> wrote:
                  Hi,

                  How do you pass the json to es-hadoop? Do you have
an example? By the way, you can enable TRACE
logging on
org.elasticsearch.hadoop and see everything that
es-hadoop does, including the data that goes over
the wire.
My guess is that the conversion of logs to JSON
creates some extra artifacts which are later on
interpreted as
Writable object (instead of raw JSON) by ES Hadoop.
Make sure you tell es-hadoop that its source it's
json (through es.input.json set to true).
The logs will likely confirm (or not) the above
                  Cheers,


                  On 3/19/14 11:14 PM, Brian Stempin wrote:

                      Hi List,
                      I have an ES cluster that takes in some data
from our logs. We use Hadoop to parse the
individual log
entries
into JSON
strings, which does a bulk insert using ES's
output format. For whatever reason, ES attempts
to parse
base64
strings as
a dates and fails. Here's a line from one of
my Hadoop logs:
                           java.lang.______IllegalStateException:
Found unrecoverable error [Bad Request(400) -
                      MapperParsingException[failed to parse
[csUriParams.d]]; nested: MapperParsingException[failed
to parse
date
field [REDACTED BASE64 STRING], tried both date
format [dateOptionalTime], and timestamp
number with
locale ];
nested: IllegalArgumentException[______Invalid
format:
"______Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT______Fj..."]; ];
Bailing out..
                               at
    org.elasticsearch.hadoop.rest.______RestClient.____
retryFailedEntries(______RestClient.java:145)
                               at org.elasticsearch.hadoop.rest.
______RestClient.bulk(__RestClient.____java:120)
                               at
    org.elasticsearch.hadoop.rest.______RestRepository.
sendBatch(______RestRepository.java:147)
                               <SNIP>


                      csUriParams.d does not appear in my mapping, so
I never explicitly asked for it to be treated
as a date.
                      Any idea why ES is trying to treat it as a date?

                      Thanks,
                      Brian

                      --
                      You received this message because you are
subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop
receiving emails from it, send an email to
elasticsearch+unsubscribe@_go
____oglegroups.com http://go__oglegroups.com
http://googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%252Bunsubscribe@googlegroups.com
mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>
<mailto:elasticsearch+
unsubscribe@googlegroups.com
mailto:elasticsearch%2B____unsubscribe@googlegroups.com
<mailto:elasticsearch%2B____unsubscribe@googlegroups.com
mailto:elasticsearch%252B__unsubscribe@googlegroups.com>
<mailto:elasticsearch%____2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%25__2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch%252Bunsubscribe@googlegroups.
_com
mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>>.
                      To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
<https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>>>
    <https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-cec3-4914-b8d6-99440dd5fb69%
40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
<https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer>>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer>
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
email&utm_source=footer>>>>.
                      For more options, visit
https://groups.google.com/d/______optout
https://groups.google.com/d/____optout <
https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>

             <https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>.
                  --
                  Costin


                  --
                  You received this message because you are
subscribed to a topic in the Google Groups
"elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/______topic/elasticsearch/______
_iE0t92CUzA/unsubscribe
<https://groups.google.com/d/____topic/elasticsearch/_____
iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/_
___topic/elasticsearch/_____iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>>
                  <https://groups.google.com/d/_
___topic/elasticsearch/__iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>>>.
To unsubscribe from this group and all its topics,
send an email to
elasticsearch+unsubscribe@go____oglegroups.com <
http://go__oglegroups.com> http://googlegroups.com
<mailto:elasticsearch%
2Bunsubscribe@googlegroups.com
mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%252Bunsubscribe@googlegroups.
_com
mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>.
                  To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__
532A0D9C.____7010401%40gmail.__com
<https://groups.google.com/d/____msgid/elasticsearch/
532A0D9C.____7010401%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/
532A0D9C.____7010401%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.
__7010401%40gmail.com>>
                  <https://groups.google.com/d/_
___msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.
__7010401%40gmail.com>
<https://groups.google.com/d/_
_msgid/elasticsearch/532A0D9C.__7010401%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532A0D9C.
7010401%40gmail.com>>>.
                  For more options, visit
https://groups.google.com/d/______optout
https://groups.google.com/d/____optout <
https://groups.google.com/d/____optout
https://groups.google.com/d/__optout>
             <https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>.
             --
             You received this message because you are subscribed to
the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>>
<mailto:elasticsearch+____unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
____pqGp0dSA%40mail.gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_
68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com>>
    <https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?
utm___medium=email&utm_source=footer>
    <https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?
utm___medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_
68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm_
medium=email&utm_source=footer>>>.
             For more options, visit https://groups.google.com/d/__
__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.
         --
         Costin

         --
         You received this message because you are subscribed to a
topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/____topic/elasticsearch/_____
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>>.
To unsubscribe from this group and all its topics, send an
email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
532B06B1.____9010206%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532B06B1.
__9010206%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.
__9010206%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B06B1.
9010206%40gmail.com>>.
         For more options, visit https://groups.google.com/d/__
__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.
    --
    You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%
40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com>
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%
40mail.gmail.com?utm_medium=email&utm_source=footer>>.
    For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.
--
Costin

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__
topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@__googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532B16AF._
_7030701%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B16AF.
7030701%40gmail.com>.
For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-
CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-
CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com?
utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.
--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/_iE0t92CUzA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/532B2F71.7060302%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANB1ciCgHpf6k67UJ0Bbq1sABPxYdRPczEUrO8P0NAW9RU9wgg%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

costin · March 21, 2014, 6:46pm

You have a type - should be "default" not "defualt"

On 3/21/14 8:37 PM, Brian Stempin wrote:

Hi Costin,
Thanks for the response -- that sounds like what I need. I re-ran my job with a mapping that included
dynamic_templates, and I'm still having issues. Here's my mapping...am I using dynamic_templates correctly?

Thanks for all of your help so far,
Brian

curl -XPUT 'http://localhost:9200/dau/log/_mapping?pretty' -d '
{
"log" : {
"properties" : {
"csUriParams" : {
"type" : "object",
"dynamic_templates" : [{
"defualt" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}],
"properties" : {
"aoi" : {"type" : "date", "ignore_malformed" : "true"},
"zoneid" : {"type" : "long", "ignore_malformed" : "true"},
"ftr" : {"type" : "date", "ignore_malformed" : "true"},
"pid" : {"type" : "string", "index" : "not_analyzed"},
"cid" : {"type" : "string", "index" : "not_analyzed"},
"ext" : {"type" : "string", "index" : "not_analyzed"},
"systemid" : {"type" : "string", "index" : "not_analyzed"}
}
},
"rawLog" : {
"type" : "object",
"dynamic_templates" : [{
"defualt" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}],
"properties" : {
"datetime" : {"type" : "date", "format" : "date_hour_minute_second", "ignore_malformed" : "true"},
"cs(User-Agent)" : {"type" : "string", "index" : "not_analyzed"},
"cs-ip" : {"type" : "string", "index" : "not_analyzed"},
"cs-uri" : {"type" : "string"},
"os" : {"type" : "string", "index" : "not_analyzed"},
"browser" : {"type" : "string", "index" : "not_analyzed"}
}
}
}
}
}
'

On Thu, Mar 20, 2014 at 2:12 PM, Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com> wrote:

Sure - take a look at dynamic_templates - you define one under your index/type and specify the match for your field.
You can either define the mapping for the fields that you want and leave the so-called catch-all (*) directive last
or, if you have type of naming convention, use that:

http://www.elasticsearch.org/__guide/en/elasticsearch/__reference/current/mapping-__root-object-type.html#___dynamic_templates
<http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-root-object-type.html#_dynamic_templates>



On 3/20/14 6:32 PM, Brian Stempin wrote:

    That's the problem -- it's a web log that contains a URL that could have literally anything in it.  Anyone could
    put a
    base64 value as a random query parameter.  I could have the M/R job ignore all fields that I don't explicitly
    expect,
    but that's not very flexible and prevents me from spotting possible abuse or user-error.  Is there any way for me to
    disable ES's type-guessing or to provide a default guess?  I'd rather have ES default to a string than to fail a
    M/R job
    because its type-guess was wrong.

    Brian


    On Thu, Mar 20, 2014 at 12:26 PM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__> wrote:

         Then what you could do is to minimize the bulk size to say 100 documents, turn on logging and run your data
    through.
         This way you can catch the 'special' document in the act.

         As for expectations - Elasticsearch tries to guess the field type by looking at its value - it seems the base64
         entry looks like a date, hence the error. You can avoid this by defining the field (either directly or
    through a
         template) in your mapping so it always gets mapped to a string.
         As a rule of thumb, whenever you want full control over the index, mapping is the way to do it.



         On 3/20/14 6:10 PM, Brian Stempin wrote:

             I have unit tests for this MR job, and they show that the JSON output is a string as I'd expect, so
    Gson is most
             likely
             not the cause.

             I'm hesitant to show more code (owned by the work-place), but I can describe it a little bit further:

                * The mapper gets a W3C log entry
                * The log entry is broken into its components and put into document X
                * The request URL is then taken and broken down into its query parameters and the key-value pairs
    are put
             into document Y
                * Some elements are then explicitly filtered from X and Y
                * Those two documents are placed inside of document Z, which is ultimately what is serialized and
    sent to ES


             We do have a base64 encoded param that we expect and handle appropriately.  In this case, someone most
    likely
             sent it as
             the wrong param name, hence why its making its way into document Y without further processing.  Since
    its being
             sent as
             a name that's not listed in the mapping, I expect it to just be treated as a string.

             The only reason that I chose to go the Gson route vs building MapWritables is that building
    MapWritables is terribly
             verbose.  Also, it comes with the overhead of having to wrap each String with a Text type, which just
    seems silly.
                Using the built-in JSON serializer is just not convenient in this case.

             Brian


             On Thu, Mar 20, 2014 at 11:18 AM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>>__>__> wrote:

                  My guess is that GSON adds the said field in its result. The base64 suggests that there's some
    binary data
             in the mix.

                  By the way, can you show up more of your code - any reason why you create the JSON yourself rather
    than
             just pass
                  logEntryMap to Es-Hadoop?
                  It can create the json for you - which is what I recommend; unless you have the JSON in HDFS, it's
    best to
             rely on
                  es-hadoop to do it instead of an external tool.

                  Cheers,


                  On 3/20/14 4:48 PM, Brian Stempin wrote:

                      Hi,
                      All I'm doing is building a map and passing that to Gson for serialization.  A snippet from my
    map method:

                      logEntryMap.put("cs(User-______Agent)", values[9]);
                      context.write(NullWritable.______get(), new Text(gson.toJson(logEntryMap))______);



                      values[] is a String array.  Everything that goes into the map that gets serialized is a string.

                      I do have es.input.json set to true.  This failure doesn't occur until >100,000,000 records
    are in the
             index, so its
                      happening late in the load process.  The part that I find strange is that the field in
    question isn't in my
                      mapping, and
                      I've not touched the default mapping.  I'm not sure why it would try to parse it as anything
    other than
             a string.

                      I'll turn on TRACE logging and see what happens.

                      Brian


                      On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <costin.leau@gmail.com
    <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>>__>
                      <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>> <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__>__>__> wrote:

                           Hi,

                           How do you pass the json to es-hadoop? Do you have an example? By the way, you can enable
    TRACE
             logging on
                           org.elasticsearch.hadoop and see everything that es-hadoop does, including the data that
    goes over
             the wire.
                           My guess is that the conversion of logs to JSON creates some extra artifacts which are
    later on
             interpreted as
                           Writable object (instead of raw JSON) by ES Hadoop.
                           Make sure you tell es-hadoop that its source it's json (through es.input.json set to true).
                           The logs will likely confirm (or not) the above :)

                           Cheers,


                           On 3/19/14 11:14 PM, Brian Stempin wrote:

                               Hi List,
                               I have an ES cluster that takes in some data from our logs.  We use Hadoop to parse the
             individual log
                      entries
                               into JSON
                               strings, which does a bulk insert using ES's output format.  For whatever reason, ES
    attempts
             to parse
                      base64
                               strings as
                               a dates and fails.  Here's a line from one of my Hadoop logs:

                                    java.lang.________IllegalStateException: Found unrecoverable error [Bad
    Request(400) -



                               MapperParsingException[failed to parse [csUriParams.d]]; nested:
    MapperParsingException[failed
             to parse
                      date
                               field [REDACTED BASE64 STRING], tried both date format [dateOptionalTime], and timestamp
             number with
                      locale []];
                               nested: IllegalArgumentException[________Invalid format:
             "________Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT________Fj..."]; ];
                      Bailing out..

                                        at
             org.elasticsearch.hadoop.rest.________RestClient.______retryFailedEntries(________RestClient.java:145)

                                        at
    org.elasticsearch.hadoop.rest.________RestClient.bulk(____RestClient.____java:120)

                                        at
             org.elasticsearch.hadoop.rest.________RestRepository.__sendBatch(________RestRepository.java:147)




                                        <SNIP>


                               csUriParams.d does not appear in my mapping, so I never explicitly asked for it to be
    treated
             as a date.

                               Any idea why ES is trying to treat it as a date?

                               Thanks,
                               Brian

                               --
                               You received this message because you are subscribed to the Google Groups
    "elasticsearch" group.
                               To unsubscribe from this group and stop receiving emails from it, send an email to
                               elasticsearch+unsubscribe@__go______oglegroups.com <http://go____oglegroups.com>
    <http://go__oglegroups.com>
             <http://googlegroups.com>
                      <mailto:elasticsearch%______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch%____252Bunsubscribe@googlegroups.
    <mailto:elasticsearch%25__252Bunsubscribe@googlegroups.>____com
             <mailto:elasticsearch%__25252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com>>>__>
                               <mailto:elasticsearch+________unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B______unsubscribe@googlegroups.com>
             <mailto:elasticsearch%2B______unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B____unsubscribe@googlegroups.com>>
                      <mailto:elasticsearch%2B______unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B____unsubscribe@googlegroups.com>
             <mailto:elasticsearch%252B____unsubscribe@googlegroups.com
    <mailto:elasticsearch%25252B__unsubscribe@googlegroups.com>>>
             <mailto:elasticsearch%______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com>__>
                      <mailto:elasticsearch%____252Bunsubscribe@googlegroups.
    <mailto:elasticsearch%25__252Bunsubscribe@googlegroups.>____com
             <mailto:elasticsearch%__25252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com>>>__>>.




                               To view this discussion on the web visit
    https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com>

    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>>



    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>>



    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>


    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>>>>



    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com?utm_______medium=__email&utm_source=______footer
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer>

    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>>



    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>>>




    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>>


    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com?utm_medium=email&utm_source=footer>>>>>.

                               For more options, visit https://groups.google.com/d/________optout
    <https://groups.google.com/d/______optout>
             <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>>
    <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>

             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>>

                      <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>>.


                           --
                           Costin


                           --
                           You received this message because you are subscribed to a topic in the Google Groups
             "elasticsearch" group.
                           To unsubscribe from this topic, visit
    https://groups.google.com/d/________topic/elasticsearch/_________iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>>

                      <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>>

                           <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>
                      <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>>>.
                           To unsubscribe from this group and all its topics, send an email to
                      elasticsearch+unsubscribe@__go______oglegroups.com <http://go____oglegroups.com>
    <http://go__oglegroups.com> <http://googlegroups.com>
                           <mailto:elasticsearch%______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com>__>
                      <mailto:elasticsearch%____252Bunsubscribe@googlegroups.
    <mailto:elasticsearch%25__252Bunsubscribe@googlegroups.>____com
             <mailto:elasticsearch%__25252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com>>>__>.



                           To view this discussion on the web visit
    https://groups.google.com/d/________msgid/elasticsearch/____532A0D9C.____7010401%40gmail.____com
    <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com>
             <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>>

                      <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>>>


    <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>>
                      <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>
             <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532A0D9C.7010401%40gmail.com>>>>.

                           For more options, visit https://groups.google.com/d/________optout
    <https://groups.google.com/d/______optout>
             <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>>
    <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>>

                      <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>>.



                      --
                      You received this message because you are subscribed to the Google Groups "elasticsearch" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to
                      elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com>
    <http://googlegroups.com>
             <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>
                      <mailto:elasticsearch+______unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B____unsubscribe@googlegroups.com>
             <mailto:elasticsearch%2B____unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B__unsubscribe@googlegroups.com>>
    <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>>.
                      To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com>>


    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com>>>


    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com?____utm___medium=email&utm___source=__footer
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer>>


    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm_medium=email&utm_source=footer>__>__>__>.



                      For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.


                  --
                  Costin

                  --
                  You received this message because you are subscribed to a topic in the Google Groups
    "elasticsearch" group.
                  To unsubscribe from this topic, visit
    https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>
                  <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>>.
                  To unsubscribe from this group and all its topics, send an email to
             elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com> <http://googlegroups.com>
                  <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>.
                  To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__532B06B1.____9010206%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com>>

                  <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com>
             <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532B06B1.9010206%40gmail.com>>>.


                  For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.


             --
             You received this message because you are subscribed to the Google Groups "elasticsearch" group.
             To unsubscribe from this group and stop receiving emails from it, send an email to
             elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch+____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B__unsubscribe@googlegroups.com> <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>>.
             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%____40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%____40mail.gmail.com?utm_medium=____email&utm_source=footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.

             For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


         --
         Costin

         --
         You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
         To unsubscribe from this topic, visit
    https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
         <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>.
         To unsubscribe from this group and all its topics, send an email to
    elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
         <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>.
         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/532B16AF.____7030701%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B16AF.__7030701%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/532B16AF.__7030701%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532B16AF.7030701%40gmail.com>>.


         For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCw1u-__CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb__xujNMiT4A%40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCw1u-__CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb__xujNMiT4A%40mail.gmail.com?__utm_medium=email&utm_source=__footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532B2F71.__7060302%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B2F71.7060302%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANB1ciCgHpf6k67UJ0Bbq1sABPxYdRPczEUrO8P0NAW9RU9wgg%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANB1ciCgHpf6k67UJ0Bbq1sABPxYdRPczEUrO8P0NAW9RU9wgg%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/532C890E.3060404%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Brian_Stempin · March 24, 2014, 3:17pm

Hi Costin,
I fixed my typo (I swear that I know how to spell ) and paired it
with "date_detection" : false. I haven't tried them independently, but the
pair seems to have resolved my issue.

Thanks for your help,
Brian

On Fri, Mar 21, 2014 at 2:46 PM, Costin Leau costin.leau@gmail.com wrote:

You have a type - should be "default" not "defualt"

On 3/21/14 8:37 PM, Brian Stempin wrote:
Hi Costin,
Thanks for the response -- that sounds like what I need. I re-ran my job
with a mapping that included
dynamic_templates, and I'm still having issues. Here's my mapping...am I
using dynamic_templates correctly?

Thanks for all of your help so far,
Brian

curl -XPUT 'http://localhost:9200/dau/log/_mapping?pretty' -d '
{
"log" : {
"properties" : {
"csUriParams" : {
"type" : "object",
"dynamic_templates" : [{
"defualt" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}],
"properties" : {
"aoi" : {"type" : "date", "ignore_malformed" : "true"},
"zoneid" : {"type" : "long", "ignore_malformed" : "true"},
"ftr" : {"type" : "date", "ignore_malformed" : "true"},
"pid" : {"type" : "string", "index" : "not_analyzed"},
"cid" : {"type" : "string", "index" : "not_analyzed"},
"ext" : {"type" : "string", "index" : "not_analyzed"},
"systemid" : {"type" : "string", "index" :
"not_analyzed"}
}
},
"rawLog" : {
"type" : "object",
"dynamic_templates" : [{
"defualt" : {
"match" : "",
"match_mapping_type" : "string",
"mapping" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}],
"properties" : {
"datetime" : {"type" : "date", "format" :
"date_hour_minute_second", "ignore_malformed" : "true"},
"cs(User-Agent)" : {"type" : "string", "index" :
"not_analyzed"},
"cs-ip" : {"type" : "string", "index" :
"not_analyzed"},
"cs-uri" : {"type" : "string"},
"os" : {"type" : "string", "index" : "not_analyzed"},
"browser" : {"type" : "string", "index" :
"not_analyzed"}
}
}
}
}
}
'

On Thu, Mar 20, 2014 at 2:12 PM, Costin Leau <costin.leau@gmail.com<mailto:
costin.leau@gmail.com>> wrote:
Sure - take a look at dynamic_templates - you define one under your
index/type and specify the match for your field.
You can either define the mapping for the fields that you want and
leave the so-called catch-all (*) directive last
or, if you have type of naming convention, use that:
http://www.elasticsearch.org/__guide/en/elasticsearch/__
reference/current/mapping-__root-object-type.html#___dynamic_templates
<http://www.elasticsearch.org/guide/en/elasticsearch/
reference/current/mapping-root-object-type.html#_dynamic_templates>
On 3/20/14 6:32 PM, Brian Stempin wrote:

    That's the problem -- it's a web log that contains a URL that
could have literally anything in it. Anyone could
put a
base64 value as a random query parameter. I could have the M/R
job ignore all fields that I don't explicitly
expect,
but that's not very flexible and prevents me from spotting
possible abuse or user-error. Is there any way for me to
disable ES's type-guessing or to provide a default guess? I'd
rather have ES default to a string than to fail a
M/R job
because its type-guess was wrong.
    Brian


    On Thu, Mar 20, 2014 at 12:26 PM, Costin Leau <
costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>__>
wrote:
         Then what you could do is to minimize the bulk size to say
100 documents, turn on logging and run your data
through.
This way you can catch the 'special' document in the act.
         As for expectations - Elasticsearch tries to guess the field
type by looking at its value - it seems the base64
entry looks like a date, hence the error. You can avoid this
by defining the field (either directly or
through a
template) in your mapping so it always gets mapped to a
string.
As a rule of thumb, whenever you want full control over the
index, mapping is the way to do it.
         On 3/20/14 6:10 PM, Brian Stempin wrote:

             I have unit tests for this MR job, and they show that
the JSON output is a string as I'd expect, so
Gson is most
likely
not the cause.
             I'm hesitant to show more code (owned by the
work-place), but I can describe it a little bit further:
                * The mapper gets a W3C log entry
                * The log entry is broken into its components and put
into document X
* The request URL is then taken and broken down into
its query parameters and the key-value pairs
are put
into document Y
* Some elements are then explicitly filtered from X
and Y
* Those two documents are placed inside of document
Z, which is ultimately what is serialized and
sent to ES
             We do have a base64 encoded param that we expect and
handle appropriately. In this case, someone most
likely
sent it as
the wrong param name, hence why its making its way into
document Y without further processing. Since
its being
sent as
a name that's not listed in the mapping, I expect it to
just be treated as a string.
             The only reason that I chose to go the Gson route vs
building MapWritables is that building
MapWritables is terribly
verbose. Also, it comes with the overhead of having to
wrap each String with a Text type, which just
seems silly.
Using the built-in JSON serializer is just not
convenient in this case.
             Brian


             On Thu, Mar 20, 2014 at 11:18 AM, Costin Leau <
costin.leau@gmail.com mailto:costin.leau@gmail.com
<mailto:costin.leau@gmail.com mailto:costin.leau@gmail.com>
<mailto:costin.leau@gmail.com <mailto:
costin.leau@gmail.com> <mailto:costin.leau@gmail.com
mailto:costin.leau@gmail.com>>> wrote:
                  My guess is that GSON adds the said field in its
result. The base64 suggests that there's some
binary data
in the mix.
                  By the way, can you show up more of your code - any
reason why you create the JSON yourself rather
than
just pass
logEntryMap to Es-Hadoop?
It can create the json for you - which is what I
recommend; unless you have the JSON in HDFS, it's
best to
rely on
es-hadoop to do it instead of an external tool.
                  Cheers,


                  On 3/20/14 4:48 PM, Brian Stempin wrote:

                      Hi,
                      All I'm doing is building a map and passing
that to Gson for serialization. A snippet from my
map method:
                      logEntryMap.put("cs(User-______Agent)",
values[9]);
context.write(NullWritable.get(), new
Text(gson.toJson(logEntryMap)));
                      values[] is a String array.  Everything that
goes into the map that gets serialized is a string.
                      I do have es.input.json set to true.  This
failure doesn't occur until >100,000,000 records
are in the
index, so its
happening late in the load process. The part
that I find strange is that the field in
question isn't in my
mapping, and
I've not touched the default mapping. I'm not
sure why it would try to parse it as anything
other than
a string.
                      I'll turn on TRACE logging and see what happens.

                      Brian


                      On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <
costin.leau@gmail.com
mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com<mailto:
costin.leau@gmail.com>>
<mailto:costin.leau@gmail.com <mailto:
costin.leau@gmail.com> <mailto:costin.leau@gmail.com
mailto:costin.leau@gmail.com>>
<mailto:costin.leau@gmail.com <mailto:
costin.leau@gmail.com> <mailto:costin.leau@gmail.com
mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com<mailto:
costin.leau@gmail.com>
<mailto:costin.leau@gmail.com <mailto:
costin.leau@gmail.com>>>>> wrote:
                           Hi,

                           How do you pass the json to es-hadoop? Do
you have an example? By the way, you can enable
TRACE
logging on
org.elasticsearch.hadoop and see
everything that es-hadoop does, including the data that
goes over
the wire.
My guess is that the conversion of logs to
JSON creates some extra artifacts which are
later on
interpreted as
Writable object (instead of raw JSON) by
ES Hadoop.
Make sure you tell es-hadoop that its
source it's json (through es.input.json set to true).
The logs will likely confirm (or not) the
above
                           Cheers,


                           On 3/19/14 11:14 PM, Brian Stempin wrote:

                               Hi List,
                               I have an ES cluster that takes in
some data from our logs. We use Hadoop to parse the
individual log
entries
into JSON
strings, which does a bulk insert
using ES's output format. For whatever reason, ES
attempts
to parse
base64
strings as
a dates and fails. Here's a line from
one of my Hadoop logs:
                                    java.lang.________IllegalStateException:
Found unrecoverable error [Bad
    Request(400) -



                               MapperParsingException[failed to parse
[csUriParams.d]]; nested:
MapperParsingException[failed
to parse
date
field [REDACTED BASE64 STRING], tried
both date format [dateOptionalTime], and timestamp
number with
locale ];
nested: IllegalArgumentException[________Invalid
format:
"________Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT________Fj..."];
];
Bailing out..
                                        at
             org.elasticsearch.hadoop.rest.________RestClient.______
retryFailedEntries(________RestClient.java:145)
                                        at
    org.elasticsearch.hadoop.rest.________RestClient.bulk(____
RestClient.____java:120)
                                        at
             org.elasticsearch.hadoop.rest.________RestRepository.__
sendBatch(________RestRepository.java:147)
                                        <SNIP>


                               csUriParams.d does not appear in my
mapping, so I never explicitly asked for it to be
treated
as a date.
                               Any idea why ES is trying to treat it
as a date?
                               Thanks,
                               Brian

                               --
                               You received this message because you
are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and
stop receiving emails from it, send an email to
elasticsearch+unsubscribe@go
oglegroups.com http://go____oglegroups.com
http://go__oglegroups.com
http://googlegroups.com
<mailto:elasticsearch%
2Bunsubscribe@googlegroups.com
mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%25__
2Bunsubscribe@googlegroups.com
mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch%252Bunsubscribe@googlegroups.
mailto:elasticsearch%25__252Bunsubscribe@googlegroups.com
<mailto:elasticsearch%25252Bunsubscribe@googlegrou
ps.com
mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com>>>
<mailto:elasticsearch+
unsubscribe@googlegroups.com
mailto:elasticsearch%2B______unsubscribe@googlegroups.com
<mailto:elasticsearch%2B__
unsubscribe@googlegroups.com
mailto:elasticsearch%252B____unsubscribe@googlegroups.com>
<mailto:elasticsearch%2B______
unsubscribe@googlegroups.com
mailto:elasticsearch%252B____unsubscribe@googlegroups.com
<mailto:elasticsearch%252B____
unsubscribe@googlegroups.com
mailto:elasticsearch%25252B__unsubscribe@googlegroups.com>>
<mailto:elasticsearch%______
2Bunsubscribe@googlegroups.com
mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com__>
<mailto:elasticsearch%____
252Bunsubscribe@googlegroups.
mailto:elasticsearch%25__252Bunsubscribe@googlegroups.____com
<mailto:elasticsearch%__25252Bunsubscribe@__googlegrou
ps.com
<mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com

__>>.
                               To view this discussion on the web
visit
https://groups.google.com/d/________msgid/elasticsearch/____
49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com
<https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com>
    <https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
<https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>>
    <https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
<https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>>
    <https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
<https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>>>>
    <https://groups.google.com/d/________msgid/elasticsearch/___
_49e5fe0b-cec3-4914-b8d6-99440dd5fb69%
40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
<https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-cec3-4914-b8d6-99440dd5fb69%
40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players>
    <https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-cec3-4914-b8d6-99440dd5fb69%
40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
<https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer>>
    <https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-cec3-4914-b8d6-99440dd5fb69%
40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
<https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer>>>
    <https://groups.google.com/d/______msgid/elasticsearch/__
49e5fe0b-cec3-4914-b8d6-99440dd5fb69%
40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
<https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer>>
    <https://groups.google.com/d/____msgid/elasticsearch/
49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer>
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-
__cec3-4914-b8d6-99440dd5fb69%_40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-
cec3-4914-b8d6-99440dd5fb69%40GGGROUPS CASINO – Real Slot Casino for 10,000+ Senior Players
email&utm_source=footer>>>>>.
                               For more options, visit
https://groups.google.com/d/________optout
https://groups.google.com/d/______optout
             <https://groups.google.com/d/______optout <
https://groups.google.com/d/____optout>>
<https://groups.google.com/d/______optout <
https://groups.google.com/d/____optout>
             <https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>>>
                      <https://groups.google.com/d/______optout <
https://groups.google.com/d/____optout>
<https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>>
<https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>>.
                           --
                           Costin


                           --
                           You received this message because you are
subscribed to a topic in the Google Groups
"elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/________topic/elasticsearch/____
_____iE0t92CUzA/unsubscribe
<https://groups.google.com/d/______topic/elasticsearch/_____
__iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/_
_____topic/elasticsearch/_______iE0t92CUzA/unsubscribe
<https://groups.google.com/d/____topic/elasticsearch/_____
iE0t92CUzA/unsubscribe>>
                      <https://groups.google.com/d/_
_____topic/elasticsearch/_______iE0t92CUzA/unsubscribe
<https://groups.google.com/d/____topic/elasticsearch/_____
iE0t92CUzA/unsubscribe>
<https://groups.google.com/d/_
___topic/elasticsearch/_____iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>>>
                           <https://groups.google.com/d/_
_____topic/elasticsearch/_______iE0t92CUzA/unsubscribe
<https://groups.google.com/d/____topic/elasticsearch/_____
iE0t92CUzA/unsubscribe>
<https://groups.google.com/d/_
___topic/elasticsearch/_____iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>>
<https://groups.google.com/d/_
topic/elasticsearch/iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>>>>.
To unsubscribe from this group and all its
topics, send an email to
elasticsearch+unsubscribe@go
oglegroups.com http://go____oglegroups.com
http://go__oglegroups.com http://googlegroups.com
<mailto:elasticsearch%
2Bunsubscribe@googlegroups.com
mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%25
2Bunsubscribe@googlegroups.com
mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch%
252Bunsubscribe@googlegroups.
mailto:elasticsearch%25__252Bunsubscribe@googlegroups.____com
<mailto:elasticsearch%__25252Bunsubscribe@googlegrou
ps.com
mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com>>>.
                           To view this discussion on the web visit
    https://groups.google.com/d/________msgid/elasticsearch/____
532A0D9C.____7010401%40gmail.____com
<https://groups.google.com/d/______msgid/elasticsearch/__
532A0D9C.____7010401%40gmail.__com>
             <https://groups.google.com/d/_
_____msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
<https://groups.google.com/d/____msgid/elasticsearch/
532A0D9C.____7010401%40gmail.com>>
                      <https://groups.google.com/d/_
_____msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
<https://groups.google.com/d/____msgid/elasticsearch/
532A0D9C.____7010401%40gmail.com>
<https://groups.google.com/d/____msgid/elasticsearch/
532A0D9C.____7010401%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.
__7010401%40gmail.com>>>
    <https://groups.google.com/d/______msgid/elasticsearch/__
532A0D9C.____7010401%40gmail.__com
<https://groups.google.com/d/____msgid/elasticsearch/
532A0D9C.____7010401%40gmail.com>
<https://groups.google.com/d/____msgid/elasticsearch/
532A0D9C.____7010401%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.
__7010401%40gmail.com>>
<https://groups.google.com/d/_
___msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.
__7010401%40gmail.com>
<https://groups.google.com/d/_
_msgid/elasticsearch/532A0D9C.__7010401%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532A0D9C.
7010401%40gmail.com>>>>.
                           For more options, visit
https://groups.google.com/d/________optout
https://groups.google.com/d/______optout
             <https://groups.google.com/d/______optout <
https://groups.google.com/d/____optout>>
<https://groups.google.com/d/______optout <
https://groups.google.com/d/____optout>
<https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>>>
                      <https://groups.google.com/d/______optout <
https://groups.google.com/d/____optout>
<https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>>
<https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>>.
                      --
                      You received this message because you are
subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop
receiving emails from it, send an email to
elasticsearch+unsubscribe@_go
____oglegroups.com http://go__oglegroups.com
http://googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%252Bunsubscribe@googlegroups.com
mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>
<mailto:elasticsearch+
unsubscribe@googlegroups.com
mailto:elasticsearch%2B____unsubscribe@googlegroups.com
<mailto:elasticsearch%2B____unsubscribe@googlegroups.com
mailto:elasticsearch%252B__unsubscribe@googlegroups.com>
<mailto:elasticsearch%____2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%25__2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch%252Bunsubscribe@googlegroups.
_com
mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>>.
To view this discussion on the web visit
https://groups.google.com/d/______msgid/elasticsearch/______
CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
______pqGp0dSA%40mail.gmail.com
<https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
____pqGp0dSA%40mail.gmail.com>
    <https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
____pqGp0dSA%40mail.gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
____pqGp0dSA%40mail.gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_
68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com>>>
    <https://groups.google.com/d/______msgid/elasticsearch/_____
_CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
______pqGp0dSA%40mail.gmail.com?____utm___medium=email&
utm___source=__footer
<https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer>
    <https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?
utm___medium=email&utm_source=footer>>
    <https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy
____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?
utm___medium=email&utm_source=footer>
    <https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?
utm___medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_
68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm_
medium=email&utm_source=footer>>>__>.
                      For more options, visit
https://groups.google.com/d/______optout
https://groups.google.com/d/____optout <
https://groups.google.com/d/____optout
https://groups.google.com/d/__optout>
<https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>.
                  --
                  Costin

                  --
                  You received this message because you are
subscribed to a topic in the Google Groups
"elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/______topic/elasticsearch/______
_iE0t92CUzA/unsubscribe
<https://groups.google.com/d/____topic/elasticsearch/_____
iE0t92CUzA/unsubscribe>
<https://groups.google.com/d/_
___topic/elasticsearch/_____iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>>
<https://groups.google.com/d/_
___topic/elasticsearch/__iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>>>.
To unsubscribe from this group and all its topics,
send an email to
elasticsearch+unsubscribe@go____oglegroups.com <
http://go__oglegroups.com> http://googlegroups.com
<mailto:elasticsearch%
2Bunsubscribe@googlegroups.com
mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com
<mailto:elasticsearch%252Bunsubscribe@googlegroups.
_com
mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/______msgid/elasticsearch/__
532B06B1.____9010206%40gmail.__com
<https://groups.google.com/d/____msgid/elasticsearch/
532B06B1.____9010206%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/
532B06B1.____9010206%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532B06B1.
__9010206%40gmail.com>>
                  <https://groups.google.com/d/_
___msgid/elasticsearch/532B06B1.____9010206%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532B06B1.
__9010206%40gmail.com>
<https://groups.google.com/d/_
_msgid/elasticsearch/532B06B1.__9010206%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B06B1.
9010206%40gmail.com>>>.
                  For more options, visit
https://groups.google.com/d/______optout
https://groups.google.com/d/____optout <
https://groups.google.com/d/____optout
https://groups.google.com/d/__optout>
<https://groups.google.com/d/____optout <
https://groups.google.com/d/__optout>
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>>.
             --
             You received this message because you are subscribed to
the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails
from it, send an email to
elasticsearch+unsubscribe@go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com <mailto:
elasticsearch%252Bunsubscribe@googlegroups.com>>
<mailto:elasticsearch+unsubscribe@googlegroups.com
mailto:elasticsearch%2B__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%
40mail.gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%__
40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__
40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com>>
    <https://groups.google.com/d/____msgid/elasticsearch/____
CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%__
__40mail.gmail.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%
40mail.gmail.com?utm_medium=__email&utm_source=footer>
    <https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__
40mail.gmail.com?utm_medium=__email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/
CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%
40mail.gmail.com?utm_medium=email&utm_source=footer>>>.
             For more options, visit https://groups.google.com/d/__
__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.
         --
         Costin

         --
         You received this message because you are subscribed to a
topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit
https://groups.google.com/d/____topic/elasticsearch/_____
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe>
<https://groups.google.com/d/__topic/elasticsearch/___
iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>>.
To unsubscribe from this group and all its topics, send an
email to
elasticsearch+unsubscribe@__go__oglegroups.com <
http://googlegroups.com>
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com
mailto:elasticsearch%252Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/____msgid/elasticsearch/
532B16AF.____7030701%40gmail.com
<https://groups.google.com/d/__msgid/elasticsearch/532B16AF.
__7030701%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/532B16AF.
__7030701%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B16AF.
7030701%40gmail.com>>.
         For more options, visit https://groups.google.com/d/__
__optout https://groups.google.com/d/__optout
<https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>>.
    --
    You received this message because you are subscribed to the
Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it,
send an email to
elasticsearch+unsubscribe@__googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>
<mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:
elasticsearch%2Bunsubscribe@googlegroups.com>>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCw1u-__CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb__xujNMiT4A%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-
CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com>
<https://groups.google.com/d/__msgid/elasticsearch/__
CANB1ciCw1u-__CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb
__xujNMiT4A%40mail.gmail.com?__utm_medium=email&utm_source=__footer
<https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-
CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com?
utm_medium=email&utm_source=footer>>.
    For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.
--
Costin

--
You received this message because you are subscribed to a topic in
the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__
topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_
iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@__googlegroups.com
mailto:elasticsearch%2Bunsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532B2F71._
_7060302%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532B2F71.
7060302%40gmail.com>.
For more options, visit https://groups.google.com/d/__optout <
https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an
email to
elasticsearch+unsubscribe@googlegroups.com <mailto:elasticsearch+
unsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/
CANB1ciCgHpf6k67UJ0Bbq1sABPxYdRPczEUrO8P0NAW9RU9wgg%40mail.gmail.com
<https://groups.google.com/d/msgid/elasticsearch/
CANB1ciCgHpf6k67UJ0Bbq1sABPxYdRPczEUrO8P0NAW9RU9wgg%40mail.
gmail.com?utm_medium=email&utm_source=footer>.

For more options, visit https://groups.google.com/d/optout.
--
Costin

--
You received this message because you are subscribed to a topic in the
Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/
topic/elasticsearch/_iE0t92CUzA/unsubscribe.
To unsubscribe from this group and all its topics, send an email to
elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/
msgid/elasticsearch/532C890E.3060404%40gmail.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CANB1ciDVuNO8ArHSOkvPUsDnKt%2BE6dOXbR%2B3QaEEJGvx9ok_Rw%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

costin · March 24, 2014, 4:23pm

That's great to hear!

Cheers,

On 3/24/14 5:17 PM, Brian Stempin wrote:

Hi Costin,
I fixed my typo (I swear that I know how to spell ) and paired it with "date_detection" : false. I haven't tried
them independently, but the pair seems to have resolved my issue.

Thanks for your help,
Brian

On Fri, Mar 21, 2014 at 2:46 PM, Costin Leau <costin.leau@gmail.com mailto:costin.leau@gmail.com> wrote:

You have a type - should be "default" not "defualt"


On 3/21/14 8:37 PM, Brian Stempin wrote:

    Hi Costin,
    Thanks for the response -- that sounds like what I need.  I re-ran my job with a mapping that included
    dynamic_templates, and I'm still having issues.  Here's my mapping...am I using dynamic_templates correctly?

    Thanks for all of your help so far,
    Brian

    curl -XPUT 'http://localhost:9200/dau/__log/_mapping?pretty <http://localhost:9200/dau/log/_mapping?pretty>' -d '
    {
          "log" : {
              "properties" : {
    "csUriParams" : {
                      "type" : "object",
                      "dynamic_templates" : [{
                          "defualt" : {
                              "match" : "*",
                              "match_mapping_type" : "string",
                              "mapping" : {
                                  "type" : "string",
                                  "index" : "not_analyzed"
                              }
                          }
                      }],
                      "properties" : {
             "aoi" : {"type" : "date", "ignore_malformed" : "true"},
                 "zoneid" : {"type" : "long", "ignore_malformed" : "true"},
                 "ftr" : {"type" : "date", "ignore_malformed" : "true"},
                 "pid" : {"type" : "string", "index" : "not_analyzed"},
                 "cid" : {"type" : "string", "index" : "not_analyzed"},
                 "ext" : {"type" : "string", "index" : "not_analyzed"},
                          "systemid" : {"type" : "string", "index" : "not_analyzed"}
         }
                  },
    "rawLog" : {
                      "type" : "object",
                      "dynamic_templates" : [{
                          "defualt" : {
                              "match" : "*",
                              "match_mapping_type" : "string",
                              "mapping" : {
                                  "type" : "string",
                                  "index" : "not_analyzed"
                              }
                          }
                      }],
                      "properties" : {
                          "datetime" : {"type" : "date", "format" : "date_hour_minute_second", "ignore_malformed" :
    "true"},
                     "cs(User-Agent)" : {"type" : "string", "index" : "not_analyzed"},
                          "cs-ip" : {"type" : "string", "index" : "not_analyzed"},
                          "cs-uri" : {"type" : "string"},
                          "os" : {"type" : "string", "index" : "not_analyzed"},
                          "browser" : {"type" : "string", "index" : "not_analyzed"}
         }
                  }
              }
          }
    }
    '


    On Thu, Mar 20, 2014 at 2:12 PM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__> wrote:

         Sure - take a look at dynamic_templates - you define one under your index/type and specify the match for
    your field.
         You can either define the mapping for the fields that you want and leave the so-called catch-all (*)
    directive last
         or, if you have type of naming convention, use that:

    http://www.elasticsearch.org/____guide/en/elasticsearch/____reference/current/mapping-____root-object-type.html#_____dynamic_templates
    <http://www.elasticsearch.org/__guide/en/elasticsearch/__reference/current/mapping-__root-object-type.html#___dynamic_templates>


    <http://www.elasticsearch.org/__guide/en/elasticsearch/__reference/current/mapping-__root-object-type.html#___dynamic_templates
    <http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-root-object-type.html#_dynamic_templates>>



         On 3/20/14 6:32 PM, Brian Stempin wrote:

             That's the problem -- it's a web log that contains a URL that could have literally anything in it.
      Anyone could
             put a
             base64 value as a random query parameter.  I could have the M/R job ignore all fields that I don't
    explicitly
             expect,
             but that's not very flexible and prevents me from spotting possible abuse or user-error.  Is there any
    way for me to
             disable ES's type-guessing or to provide a default guess?  I'd rather have ES default to a string than
    to fail a
             M/R job
             because its type-guess was wrong.

             Brian


             On Thu, Mar 20, 2014 at 12:26 PM, Costin Leau <costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>>__>__> wrote:

                  Then what you could do is to minimize the bulk size to say 100 documents, turn on logging and run
    your data
             through.
                  This way you can catch the 'special' document in the act.

                  As for expectations - Elasticsearch tries to guess the field type by looking at its value - it
    seems the base64
                  entry looks like a date, hence the error. You can avoid this by defining the field (either directly or
             through a
                  template) in your mapping so it always gets mapped to a string.
                  As a rule of thumb, whenever you want full control over the index, mapping is the way to do it.



                  On 3/20/14 6:10 PM, Brian Stempin wrote:

                      I have unit tests for this MR job, and they show that the JSON output is a string as I'd
    expect, so
             Gson is most
                      likely
                      not the cause.

                      I'm hesitant to show more code (owned by the work-place), but I can describe it a little bit
    further:

                         * The mapper gets a W3C log entry
                         * The log entry is broken into its components and put into document X
                         * The request URL is then taken and broken down into its query parameters and the key-value
    pairs
             are put
                      into document Y
                         * Some elements are then explicitly filtered from X and Y
                         * Those two documents are placed inside of document Z, which is ultimately what is
    serialized and
             sent to ES


                      We do have a base64 encoded param that we expect and handle appropriately.  In this case,
    someone most
             likely
                      sent it as
                      the wrong param name, hence why its making its way into document Y without further processing.
      Since
             its being
                      sent as
                      a name that's not listed in the mapping, I expect it to just be treated as a string.

                      The only reason that I chose to go the Gson route vs building MapWritables is that building
             MapWritables is terribly
                      verbose.  Also, it comes with the overhead of having to wrap each String with a Text type,
    which just
             seems silly.
                         Using the built-in JSON serializer is just not convenient in this case.

                      Brian


                      On Thu, Mar 20, 2014 at 11:18 AM, Costin Leau <costin.leau@gmail.com
    <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>>__>
                      <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>> <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__>__>__> wrote:

                           My guess is that GSON adds the said field in its result. The base64 suggests that there's
    some
             binary data
                      in the mix.

                           By the way, can you show up more of your code - any reason why you create the JSON
    yourself rather
             than
                      just pass
                           logEntryMap to Es-Hadoop?
                           It can create the json for you - which is what I recommend; unless you have the JSON in
    HDFS, it's
             best to
                      rely on
                           es-hadoop to do it instead of an external tool.

                           Cheers,


                           On 3/20/14 4:48 PM, Brian Stempin wrote:

                               Hi,
                               All I'm doing is building a map and passing that to Gson for serialization.  A
    snippet from my
             map method:

                               logEntryMap.put("cs(User-________Agent)", values[9]);
                               context.write(NullWritable.________get(), new Text(gson.toJson(logEntryMap))________);




                               values[] is a String array.  Everything that goes into the map that gets serialized
    is a string.

                               I do have es.input.json set to true.  This failure doesn't occur until >100,000,000
    records
             are in the
                      index, so its
                               happening late in the load process.  The part that I find strange is that the field in
             question isn't in my
                               mapping, and
                               I've not touched the default mapping.  I'm not sure why it would try to parse it as
    anything
             other than
                      a string.

                               I'll turn on TRACE logging and see what happens.

                               Brian


                               On Wed, Mar 19, 2014 at 5:35 PM, Costin Leau <costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__>
                      <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>> <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__>__>
                               <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>
    <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>
             <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>__> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com>>
                      <mailto:costin.leau@gmail.com <mailto:costin.leau@gmail.com> <mailto:costin.leau@gmail.com
    <mailto:costin.leau@gmail.com>>__>__>__>__> wrote:

                                    Hi,

                                    How do you pass the json to es-hadoop? Do you have an example? By the way, you
    can enable
             TRACE
                      logging on
                                    org.elasticsearch.hadoop and see everything that es-hadoop does, including the
    data that
             goes over
                      the wire.
                                    My guess is that the conversion of logs to JSON creates some extra artifacts
    which are
             later on
                      interpreted as
                                    Writable object (instead of raw JSON) by ES Hadoop.
                                    Make sure you tell es-hadoop that its source it's json (through es.input.json
    set to true).
                                    The logs will likely confirm (or not) the above :)

                                    Cheers,


                                    On 3/19/14 11:14 PM, Brian Stempin wrote:

                                        Hi List,
                                        I have an ES cluster that takes in some data from our logs.  We use Hadoop
    to parse the
                      individual log
                               entries
                                        into JSON
                                        strings, which does a bulk insert using ES's output format.  For whatever
    reason, ES
             attempts
                      to parse
                               base64
                                        strings as
                                        a dates and fails.  Here's a line from one of my Hadoop logs:

                                             java.lang.__________IllegalStateException: Found unrecoverable error [Bad

             Request(400) -



                                        MapperParsingException[failed to parse [csUriParams.d]]; nested:
             MapperParsingException[failed
                      to parse
                               date
                                        field [REDACTED BASE64 STRING], tried both date format [dateOptionalTime],
    and timestamp
                      number with
                               locale []];
                                        nested: IllegalArgumentException[__________Invalid format:
                      "__________Y2lkPURFJml0ZW1zPWE2NTJjLXgxZT__________Fj..."]; ];
                               Bailing out..

                                                 at

      org.elasticsearch.hadoop.rest.__________RestClient.________retryFailedEntries(__________RestClient.java:145)

                                                 at
             org.elasticsearch.hadoop.rest.__________RestClient.bulk(______RestClient.____java:120)

                                                 at

      org.elasticsearch.hadoop.rest.__________RestRepository.____sendBatch(__________RestRepository.java:147)





                                                 <SNIP>


                                        csUriParams.d does not appear in my mapping, so I never explicitly asked for
    it to be
             treated
                      as a date.

                                        Any idea why ES is trying to treat it as a date?

                                        Thanks,
                                        Brian

                                        --
                                        You received this message because you are subscribed to the Google Groups
             "elasticsearch" group.
                                        To unsubscribe from this group and stop receiving emails from it, send an
    email to
                                        elasticsearch+unsubscribe@__go________oglegroups.com
    <http://go______oglegroups.com> <http://go____oglegroups.com>
             <http://go__oglegroups.com>
                      <http://googlegroups.com>
                               <mailto:elasticsearch%________2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com>__>
                      <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252525__2Bunsubscribe@googlegroups.com>__>__>
                      <mailto:elasticsearch%______252Bunsubscribe@googlegroups
    <mailto:elasticsearch%25____252Bunsubscribe@googlegroups>.
             <mailto:elasticsearch%25____252Bunsubscribe@googlegroups
    <mailto:elasticsearch%2525__252Bunsubscribe@googlegroups>.>______com
                      <mailto:elasticsearch%____25252Bunsubscribe@
    <mailto:elasticsearch%25__25252Bunsubscribe@>__googlegrou__ps.com <http://googlegroups.com>
             <mailto:elasticsearch%__2525252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%252525252Bunsubscribe@googlegroups.com>>>>__>
                                        <mailto:elasticsearch+__________unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B________unsubscribe@googlegroups.com>
             <mailto:elasticsearch%2B________unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B______unsubscribe@googlegroups.com>>
                      <mailto:elasticsearch%2B________unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B______unsubscribe@googlegroups.com>
             <mailto:elasticsearch%252B______unsubscribe@googlegroups.com
    <mailto:elasticsearch%25252B____unsubscribe@googlegroups.com>>>
                               <mailto:elasticsearch%2B________unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B______unsubscribe@googlegroups.com>
             <mailto:elasticsearch%252B______unsubscribe@googlegroups.com
    <mailto:elasticsearch%25252B____unsubscribe@googlegroups.com>>
                      <mailto:elasticsearch%252B______unsubscribe@googlegroups.com
    <mailto:elasticsearch%25252B____unsubscribe@googlegroups.com>
             <mailto:elasticsearch%25252B____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2525252B__unsubscribe@googlegroups.com>>>__>
                      <mailto:elasticsearch%________2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252525__2Bunsubscribe@googlegroups.com>__>__>
                               <mailto:elasticsearch%______252Bunsubscribe@googlegroups
    <mailto:elasticsearch%25____252Bunsubscribe@googlegroups>.
             <mailto:elasticsearch%25____252Bunsubscribe@googlegroups
    <mailto:elasticsearch%2525__252Bunsubscribe@googlegroups>.>______com
                      <mailto:elasticsearch%____25252Bunsubscribe@
    <mailto:elasticsearch%25__25252Bunsubscribe@>__googlegrou__ps.com <http://googlegroups.com>
             <mailto:elasticsearch%__2525252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%252525252Bunsubscribe@googlegroups.com>>>>__>>.





                                        To view this discussion on the web visit
    https://groups.google.com/d/__________msgid/elasticsearch/______49e5fe0b-____cec3-4914-b8d6-______99440dd5fb69%__________40googlegroups.com
    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com>

    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com>>



    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com>

    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>>>




    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com>

    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>>


    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>>>




    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com>

    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>>


    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>>



    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>>


    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com>

    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com>>>>>




    <https://groups.google.com/d/__________msgid/elasticsearch/______49e5fe0b-____cec3-4914-b8d6-______99440dd5fb69%__________40googlegroups.com?utm_________medium=__email&utm_source=________footer
    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com?utm_______medium=__email&utm_source=______footer>

    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com?utm_______medium=__email&utm_source=______footer
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer>>



    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com?utm_______medium=__email&utm_source=______footer
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer>

    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>>>




    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com?utm_______medium=__email&utm_source=______footer
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer>

    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>>


    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>>>>





    <https://groups.google.com/d/________msgid/elasticsearch/____49e5fe0b-____cec3-4914-b8d6-____99440dd5fb69%________40googlegroups.com?utm_______medium=__email&utm_source=______footer
    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer>

    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>>


    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>>>



    <https://groups.google.com/d/______msgid/elasticsearch/__49e5fe0b-____cec3-4914-b8d6-__99440dd5fb69%______40googlegroups.com?utm_____medium=__email&utm_source=____footer
    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>>


    <https://groups.google.com/d/____msgid/elasticsearch/49e5fe0b-____cec3-4914-b8d6-99440dd5fb69%____40googlegroups.com?utm___medium=__email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/49e5fe0b-__cec3-4914-b8d6-99440dd5fb69%__40googlegroups.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/49e5fe0b-cec3-4914-b8d6-99440dd5fb69%40googlegroups.com?utm_medium=email&utm_source=footer>>>>>>.

                                        For more options, visit https://groups.google.com/d/__________optout
    <https://groups.google.com/d/________optout>
             <https://groups.google.com/d/________optout <https://groups.google.com/d/______optout>>

                      <https://groups.google.com/d/________optout <https://groups.google.com/d/______optout>
    <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>>>
             <https://groups.google.com/d/________optout <https://groups.google.com/d/______optout>
    <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>>

                      <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>>>

                               <https://groups.google.com/d/________optout
    <https://groups.google.com/d/______optout> <https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout>>
             <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>>
                      <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>>>.


                                    --
                                    Costin


                                    --
                                    You received this message because you are subscribed to a topic in the Google Groups
                      "elasticsearch" group.
                                    To unsubscribe from this topic, visit
    https://groups.google.com/d/__________topic/elasticsearch/___________iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/________topic/elasticsearch/_________iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/________topic/elasticsearch/_________iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe>>

                      <https://groups.google.com/d/________topic/elasticsearch/_________iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>>>


    <https://groups.google.com/d/________topic/elasticsearch/_________iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>>
                      <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>>>


      <https://groups.google.com/d/________topic/elasticsearch/_________iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>>
                      <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>>
                               <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>
                      <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>>>>.
                                    To unsubscribe from this group and all its topics, send an email to
                               elasticsearch+unsubscribe@__go________oglegroups.com <http://go______oglegroups.com>
    <http://go____oglegroups.com>
             <http://go__oglegroups.com> <http://googlegroups.com>
                                    <mailto:elasticsearch%________2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com>__>
                      <mailto:elasticsearch%25______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%2525____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252525__2Bunsubscribe@googlegroups.com>__>__>
                               <mailto:elasticsearch%______252Bunsubscribe@googlegroups
    <mailto:elasticsearch%25____252Bunsubscribe@googlegroups>.
             <mailto:elasticsearch%25____252Bunsubscribe@googlegroups
    <mailto:elasticsearch%2525__252Bunsubscribe@googlegroups>.>______com
                      <mailto:elasticsearch%____25252Bunsubscribe@
    <mailto:elasticsearch%25__25252Bunsubscribe@>__googlegrou__ps.com <http://googlegroups.com>
             <mailto:elasticsearch%__2525252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%252525252Bunsubscribe@googlegroups.com>>>>__>.




                                    To view this discussion on the web visit
    https://groups.google.com/d/__________msgid/elasticsearch/______532A0D9C.____7010401%40gmail.______com
    <https://groups.google.com/d/________msgid/elasticsearch/____532A0D9C.____7010401%40gmail.____com>
             <https://groups.google.com/d/________msgid/elasticsearch/____532A0D9C.____7010401%40gmail.____com
    <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com>>


      <https://groups.google.com/d/________msgid/elasticsearch/____532A0D9C.____7010401%40gmail.____com
    <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com>
             <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>>>


    <https://groups.google.com/d/________msgid/elasticsearch/____532A0D9C.____7010401%40gmail.____com
    <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com>
             <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>>
                      <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>>>>


             <https://groups.google.com/d/________msgid/elasticsearch/____532A0D9C.____7010401%40gmail.____com
    <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com>
             <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>>
                      <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>>>

    <https://groups.google.com/d/______msgid/elasticsearch/__532A0D9C.____7010401%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>>
                      <https://groups.google.com/d/____msgid/elasticsearch/532A0D9C.____7010401%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com>
             <https://groups.google.com/d/__msgid/elasticsearch/532A0D9C.__7010401%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532A0D9C.7010401%40gmail.com>>>>>.

                                    For more options, visit https://groups.google.com/d/__________optout
    <https://groups.google.com/d/________optout>
             <https://groups.google.com/d/________optout <https://groups.google.com/d/______optout>>

                      <https://groups.google.com/d/________optout <https://groups.google.com/d/______optout>
    <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>>>
             <https://groups.google.com/d/________optout <https://groups.google.com/d/______optout>
    <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>>
                      <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>>>

                               <https://groups.google.com/d/________optout
    <https://groups.google.com/d/______optout> <https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout>>
             <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>>
                      <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>>>.



                               --
                               You received this message because you are subscribed to the Google Groups
    "elasticsearch" group.
                               To unsubscribe from this group and stop receiving emails from it, send an email to
                               elasticsearch+unsubscribe@__go______oglegroups.com <http://go____oglegroups.com>
    <http://go__oglegroups.com>
             <http://googlegroups.com>
                      <mailto:elasticsearch%______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch%____252Bunsubscribe@googlegroups.
    <mailto:elasticsearch%25__252Bunsubscribe@googlegroups.>____com
             <mailto:elasticsearch%__25252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com>>>__>
                               <mailto:elasticsearch+________unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B______unsubscribe@googlegroups.com>
             <mailto:elasticsearch%2B______unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B____unsubscribe@googlegroups.com>>
                      <mailto:elasticsearch%2B______unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B____unsubscribe@googlegroups.com>
             <mailto:elasticsearch%252B____unsubscribe@googlegroups.com
    <mailto:elasticsearch%25252B__unsubscribe@googlegroups.com>>>
             <mailto:elasticsearch%______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com>__>
                      <mailto:elasticsearch%____252Bunsubscribe@googlegroups.
    <mailto:elasticsearch%25__252Bunsubscribe@googlegroups.>____com
             <mailto:elasticsearch%__25252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com>>>__>>.
                               To view this discussion on the web visit
    https://groups.google.com/d/________msgid/elasticsearch/________CANB1ciCdBYj_________68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy________pqGp0dSA%40mail.gmail.__com
    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com>

    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com>>



    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com>>>



    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com>>


    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com>>>>



    <https://groups.google.com/d/________msgid/elasticsearch/________CANB1ciCdBYj_________68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy________pqGp0dSA%40mail.gmail.__com?____utm___medium=email&__utm___source=__footer
    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com?____utm___medium=email&utm___source=__footer>

    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com?____utm___medium=email&utm___source=__footer
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer>>



    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com?____utm___medium=email&utm___source=__footer
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer>>>



    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciCdBYj_______68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy______pqGp0dSA%40mail.gmail.com?____utm___medium=email&utm___source=__footer
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer>>


    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCdBYj_____68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy____pqGp0dSA%40mail.gmail.com?__utm___medium=email&utm_source=__footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCdBYj___68DCxEcDxfYucuyhJ7NPWrmEWtV2Cy__pqGp0dSA%40mail.gmail.com?utm___medium=email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCdBYj_68DCxEcDxfYucuyhJ7NPWrmEWtV2CypqGp0dSA%40mail.gmail.com?utm_medium=email&utm_source=footer>__>__>__>__>.




                               For more options, visit https://groups.google.com/d/________optout
    <https://groups.google.com/d/______optout>
             <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>>
    <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>>
                      <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>>.


                           --
                           Costin

                           --
                           You received this message because you are subscribed to a topic in the Google Groups
             "elasticsearch" group.
                           To unsubscribe from this topic, visit
    https://groups.google.com/d/________topic/elasticsearch/_________iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>>
                      <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>>
                           <https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>
                      <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>>>.
                           To unsubscribe from this group and all its topics, send an email to
                      elasticsearch+unsubscribe@__go______oglegroups.com <http://go____oglegroups.com>
    <http://go__oglegroups.com> <http://googlegroups.com>
                           <mailto:elasticsearch%______2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%25____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%2525__2Bunsubscribe@googlegroups.com>__>
                      <mailto:elasticsearch%____252Bunsubscribe@googlegroups.
    <mailto:elasticsearch%25__252Bunsubscribe@googlegroups.>____com
             <mailto:elasticsearch%__25252Bunsubscribe@__googlegroups.com
    <mailto:elasticsearch%2525252Bunsubscribe@googlegroups.com>>>__>.
                           To view this discussion on the web visit
    https://groups.google.com/d/________msgid/elasticsearch/____532B06B1.____9010206%40gmail.____com
    <https://groups.google.com/d/______msgid/elasticsearch/__532B06B1.____9010206%40gmail.__com>
             <https://groups.google.com/d/______msgid/elasticsearch/__532B06B1.____9010206%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com>>

                      <https://groups.google.com/d/______msgid/elasticsearch/__532B06B1.____9010206%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com>>>


    <https://groups.google.com/d/______msgid/elasticsearch/__532B06B1.____9010206%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com>>
                      <https://groups.google.com/d/____msgid/elasticsearch/532B06B1.____9010206%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com>
             <https://groups.google.com/d/__msgid/elasticsearch/532B06B1.__9010206%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532B06B1.9010206%40gmail.com>>>>.


                           For more options, visit https://groups.google.com/d/________optout
    <https://groups.google.com/d/______optout>
             <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>>
    <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>>
                      <https://groups.google.com/d/______optout <https://groups.google.com/d/____optout>
    <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>>.


                      --
                      You received this message because you are subscribed to the Google Groups "elasticsearch" group.
                      To unsubscribe from this group and stop receiving emails from it, send an email to
                      elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com>
    <http://googlegroups.com>
             <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>
                      <mailto:elasticsearch+______unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B____unsubscribe@googlegroups.com>
             <mailto:elasticsearch%2B____unsubscribe@googlegroups.com
    <mailto:elasticsearch%252B__unsubscribe@googlegroups.com>>
    <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>>.
                      To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciBy6jCC8YVT4FPi03g9TgGkt______-QhB%2BUQKfWvDioYBnRopQ%______40mail.gmail.com
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%____40mail.gmail.com>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%____40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com>>


    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%____40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com>>>


    <https://groups.google.com/d/______msgid/elasticsearch/______CANB1ciBy6jCC8YVT4FPi03g9TgGkt______-QhB%2BUQKfWvDioYBnRopQ%______40mail.gmail.com?utm_medium=______email&utm_source=footer
    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%____40mail.gmail.com?utm_medium=____email&utm_source=footer>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%____40mail.gmail.com?utm_medium=____email&utm_source=footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com?utm_medium=__email&utm_source=footer>>



    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciBy6jCC8YVT4FPi03g9TgGkt____-QhB%2BUQKfWvDioYBnRopQ%____40mail.gmail.com?utm_medium=____email&utm_source=footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com?utm_medium=__email&utm_source=footer>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciBy6jCC8YVT4FPi03g9TgGkt__-QhB%2BUQKfWvDioYBnRopQ%__40mail.gmail.com?utm_medium=__email&utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciBy6jCC8YVT4FPi03g9TgGkt-QhB%2BUQKfWvDioYBnRopQ%40mail.gmail.com?utm_medium=email&utm_source=footer>>>>.

                      For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.


                  --
                  Costin

                  --
                  You received this message because you are subscribed to a topic in the Google Groups
    "elasticsearch" group.
                  To unsubscribe from this topic, visit
    https://groups.google.com/d/______topic/elasticsearch/_______iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>>
                  <https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
             <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>>.
                  To unsubscribe from this group and all its topics, send an email to
             elasticsearch+unsubscribe@__go____oglegroups.com <http://go__oglegroups.com> <http://googlegroups.com>
                  <mailto:elasticsearch%____2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%25__2Bunsubscribe@googlegroups.com>
             <mailto:elasticsearch%__252Bunsubscribe@googlegroups.__com
    <mailto:elasticsearch%25252Bunsubscribe@googlegroups.com>>__>.
                  To view this discussion on the web visit
    https://groups.google.com/d/______msgid/elasticsearch/__532B16AF.____7030701%40gmail.__com
    <https://groups.google.com/d/____msgid/elasticsearch/532B16AF.____7030701%40gmail.com>
             <https://groups.google.com/d/____msgid/elasticsearch/532B16AF.____7030701%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B16AF.__7030701%40gmail.com>>

                  <https://groups.google.com/d/____msgid/elasticsearch/532B16AF.____7030701%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B16AF.__7030701%40gmail.com>
             <https://groups.google.com/d/__msgid/elasticsearch/532B16AF.__7030701%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532B16AF.7030701%40gmail.com>>>.


                  For more options, visit https://groups.google.com/d/______optout
    <https://groups.google.com/d/____optout> <https://groups.google.com/d/____optout
    <https://groups.google.com/d/__optout>>
             <https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>>.


             --
             You received this message because you are subscribed to the Google Groups "elasticsearch" group.
             To unsubscribe from this group and stop receiving emails from it, send an email to
             elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
    <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>
             <mailto:elasticsearch+____unsubscribe@googlegroups.com
    <mailto:elasticsearch%2B__unsubscribe@googlegroups.com> <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>>.
             To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCw1u-____CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb____xujNMiT4A%40mail.gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCw1u-__CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb__xujNMiT4A%40mail.gmail.com>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCw1u-__CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb__xujNMiT4A%40mail.gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com>>

    <https://groups.google.com/d/____msgid/elasticsearch/____CANB1ciCw1u-____CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb____xujNMiT4A%40mail.gmail.com?____utm_medium=email&utm_source=____footer
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCw1u-__CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb__xujNMiT4A%40mail.gmail.com?__utm_medium=email&utm_source=__footer>

    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCw1u-__CpdTcxvXBShuaLNDEAWgyk4Jvq4ifb__xujNMiT4A%40mail.gmail.com?__utm_medium=email&utm_source=__footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCw1u-CpdTcxvXBShuaLNDEAWgyk4Jvq4ifbxujNMiT4A%40mail.gmail.com?utm_medium=email&utm_source=footer>>>.

             For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


         --
         Costin

         --
         You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
         To unsubscribe from this topic, visit
    https://groups.google.com/d/____topic/elasticsearch/_____iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe>
         <https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
    <https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>>.
         To unsubscribe from this group and all its topics, send an email to
    elasticsearch+unsubscribe@__go__oglegroups.com <http://googlegroups.com>
         <mailto:elasticsearch%__2Bunsubscribe@googlegroups.com
    <mailto:elasticsearch%252Bunsubscribe@googlegroups.com>__>.
         To view this discussion on the web visit
    https://groups.google.com/d/____msgid/elasticsearch/532B2F71.____7060302%40gmail.com
    <https://groups.google.com/d/__msgid/elasticsearch/532B2F71.__7060302%40gmail.com>
         <https://groups.google.com/d/__msgid/elasticsearch/532B2F71.__7060302%40gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/532B2F71.7060302%40gmail.com>>.


         For more options, visit https://groups.google.com/d/____optout <https://groups.google.com/d/__optout>
    <https://groups.google.com/d/__optout <https://groups.google.com/d/optout>>.


    --
    You received this message because you are subscribed to the Google Groups "elasticsearch" group.
    To unsubscribe from this group and stop receiving emails from it, send an email to
    elasticsearch+unsubscribe@__googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>
    <mailto:elasticsearch+__unsubscribe@googlegroups.com <mailto:elasticsearch%2Bunsubscribe@googlegroups.com>>.
    To view this discussion on the web visit
    https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCgHpf6k67UJ0Bbq1sABPxYd__RPczEUrO8P0NAW9RU9wgg%40mail.__gmail.com
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCgHpf6k67UJ0Bbq1sABPxYdRPczEUrO8P0NAW9RU9wgg%40mail.gmail.com>
    <https://groups.google.com/d/__msgid/elasticsearch/__CANB1ciCgHpf6k67UJ0Bbq1sABPxYd__RPczEUrO8P0NAW9RU9wgg%40mail.__gmail.com?utm_medium=email&__utm_source=footer
    <https://groups.google.com/d/msgid/elasticsearch/CANB1ciCgHpf6k67UJ0Bbq1sABPxYdRPczEUrO8P0NAW9RU9wgg%40mail.gmail.com?utm_medium=email&utm_source=footer>>.

    For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.


--
Costin

--
You received this message because you are subscribed to a topic in the Google Groups "elasticsearch" group.
To unsubscribe from this topic, visit https://groups.google.com/d/__topic/elasticsearch/___iE0t92CUzA/unsubscribe
<https://groups.google.com/d/topic/elasticsearch/_iE0t92CUzA/unsubscribe>.
To unsubscribe from this group and all its topics, send an email to elasticsearch+unsubscribe@__googlegroups.com
<mailto:elasticsearch%2Bunsubscribe@googlegroups.com>.
To view this discussion on the web visit
https://groups.google.com/d/__msgid/elasticsearch/532C890E.__3060404%40gmail.com
<https://groups.google.com/d/msgid/elasticsearch/532C890E.3060404%40gmail.com>.

For more options, visit https://groups.google.com/d/__optout <https://groups.google.com/d/optout>.

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to
elasticsearch+unsubscribe@googlegroups.com mailto:elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CANB1ciDVuNO8ArHSOkvPUsDnKt%2BE6dOXbR%2B3QaEEJGvx9ok_Rw%40mail.gmail.com
https://groups.google.com/d/msgid/elasticsearch/CANB1ciDVuNO8ArHSOkvPUsDnKt%2BE6dOXbR%2B3QaEEJGvx9ok_Rw%40mail.gmail.com?utm_medium=email&utm_source=footer.
For more options, visit https://groups.google.com/d/optout.

--
Costin

--
You received this message because you are subscribed to the Google Groups "elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/53305C0B.7070206%40gmail.com.
For more options, visit https://groups.google.com/d/optout.

Topic		Replies	Views
Mapping issue about date in elasticsearch 2.3.3 Elasticsearch	8	1742	July 5, 2017
Invalid Date Format Elasticsearch	1	768	March 25, 2020
Mapper Parsing Exception for field with date data that I want to index as a string Elasticsearch	6	10895	February 24, 2017
MapperParsingException after migrating Java client from ES 2.3.3 to 5.5.0 Elasticsearch	9	815	August 9, 2017
Date Mapping Parse Failure after upgrade to 7.1.1 Elasticsearch	4	2671	July 7, 2019

ES attempting to parse dates automatically (possibly Hadoop related)

Related Topics