Interaction between date_histogram and filter on same field

Elastic Stack Elasticsearch
1

Hi,

I'm seeing something slightly odd when using a date_histogram and a filter on the same field. When using both, the query results look fine, but I see errors in the ES log.

Here's the query:

https://gist.github.com/2bdcbf74888b067257c8

Now - the output of the query does look correct. However, I see the following in the ES log:

https://gist.github.com/bc644940577caab88e15

It looks like ES is complaining that interaction.created_at is an invalid field when building the date histogram facet - however, the results look absolutely fine (with some non-exhaustive testing).

If I don't include the range filters, then I don't get any errors in the log.

I have a mapping for this type that looks like this:

        'interaction.created_at': {
            'type': 'date',
            'format': 'EEE, dd MMM yyyy HH:mm:ss +0000'
        }

Can anyone tell me what's going on?

(Incidentally, the reason I've got two range filters rather than just one is that this is an early version of our code, just building a query naively based on parameters passed to it. I'll improve this in due course, assuming it's unrelated.)

This is ElasticSearch 0.19.8 running on OS X Lion (will be Ubuntu 12.04 in production).

Cheers,
Dan

--
Dan Fairs | dan.fairs@gmail.com | @danfairs | www.fezconsulting.com

2

Sorry to reply to my own post.

Here's the query:

Example query · GitHub

Now - the output of the query does look correct. However, I see the following in the ES log:

Traceback · GitHub

It looks like ES is complaining that interaction.created_at is an invalid field when building the date histogram facet - however, the results look absolutely fine (with some non-exhaustive testing).

I've just noticed that these are logged at DEBUG level, therefore feel ignorable. Is that right, or should I be worried about them?

Cheers,
Dan

Dan Fairs | dan.fairs@gmail.com | @danfairs | www.fezconsulting.com

3

Hi Dan,

It looks like you are running this query across all indices in your cluster
and a couple of indices (_river and danfairs_tweets_for_tx_0000032048)
don't have the field interaction.created_at defined in their mappings. You
might improve performance of your query by running it only on the indexes,
that you really want to get data from.

Igor

On Monday, July 9, 2012 8:27:13 AM UTC-4, Dan Fairs wrote:

Sorry to reply to my own post.

Here's the query:

Example query · GitHub

Now - the output of the query does look correct. However, I see the
following in the ES log:

Traceback · GitHub

It looks like ES is complaining that interaction.created_at is an invalid
field when building the date histogram facet - however, the results look
absolutely fine (with some non-exhaustive testing).

I've just noticed that these are logged at DEBUG level, therefore feel
ignorable. Is that right, or should I be worried about them?

Cheers,
Dan

Dan Fairs | dan.fairs@gmail.com | @danfairs | www.fezconsulting.com

4

Hi Igor

It looks like you are running this query across all indices in your cluster and a couple of indices (_river and danfairs_tweets_for_tx_0000032048) don't have the field interaction.created_at defined in their mappings. You might improve performance of your query by running it only on the indexes, that you really want to get data from.

Yes - you're quite right, I noticed this when comparing the shards that were erroring with the shards that were actually hosting the data. As you suggested, I restricted the query to run only on the relevant indices, and I no longer get the log messages.

Thanks for taking a look!

Cheers,
Dan

Dan Fairs | dan.fairs@gmail.com | @danfairs | www.fezconsulting.com