No filter registered for [@timestamp]

Elastic Stack Elasticsearch
1

I have a EFK platform and I trying searchs using the PHP API (https://github.com/elastic/elasticsearch-php)

This is one index item:

{"message":"New request to POST /match.","hostname":"gandalf","severity":"crit","host":"app-base","application":"app-api","environment":"PRO","ip":"80.38.71.125","user_agent":"curl","user_id":"561226f2fef3874e058b4568","device_id":"match_abs","operation_id":"5665cbccbae4e","operation":"find_match","key":"match.post","code":"0","value_float1":"1","value_float2":"0","value_float3":"0","value_string1":"-","value_string2":"-","value_string3":"-","@timestamp":"2015-12-07T19:11:24+01:00"}

This is my test script:

<?php

require 'vendor/autoload.php';

use Monolog\Logger;
use Monolog\Handler\StreamHandler;
use Elasticsearch\ClientBuilder;

$logger = new Logger('name');

$hosts = [
    'localhost:9200', // IP + Port
];

$logger = ClientBuilder::defaultLogger('/tmp/el.log');

$client = ClientBuilder::create() // Instantiate a new ClientBuilder
  ->setHosts($hosts)        // Set the hosts
  ->setLogger($logger)      // Set the logger with a default logger
  ->build();

$params = [
  'index' => 'logstash*',
  'size' => 500,
  'type' => 'fluentd',
  'body' => [
    'sort' => [
      '@timestamp' => ['order' => 'asc'],
     ],
     'query' => [
       'filtered' => [
         'filter' => [
           'bool' => [
             'must' => [
               'query' => [
                 'match' => ['environment' => 'PRO']
               ],
                 'range' => [
                   '@timestamp' => [
                     'gte' => 1449442800000,
                     'lte' => 1449529199999
                   ]
                 ]
               ],
             'must_not' => []
           ]
         ],
         'query' => [
           'match' => ['user_id' => '561226f2fef3874e058b4568'],
           'match' => ['key' => 'match.post']
         ]
       ]
     ]
   ]
];

$response = $client->search($params);
 
echo $response['hits']['total'] . "\n";
foreach($response['hits']['hits'] as $item) {
  echo $item['_source']['key'] . "\n";
  echo $item['_source']['environment'] . "\n";
  echo "\n";
}

After run my script I get this error:

Dec  8 10:05:50 11.0.0.174 ::php-cli::PRO::-: PHP Fatal error:  Uncaught exception 'Elasticsearch\Common\Exceptions\BadRequest400Exception' with message '{"error":"SearchPhaseExecutionException[Failed to execute phase [query], all shards failed; shardFailures {[G9r0YNWZSEG7-tOgMuDNWQ][logstash-2015.12.07][0]: SearchParseException[[logstash-2015.12.07][0]: from[-1],size[-1],sort[<custom:\"@timestamp\": org.elasticsearch.index.fielddata.fieldcomparator.LongValuesComparatorSource@3ff7f796>]: Parse Failure [Failed to parse source [{\"sort\":{\"@timestamp\":{\"order\":\"asc\"}},\"query\":{\"filtered\":{\"filter\":{\"bool\":{\"must\":{\"query\":{\"match\":{\"environment\":\"PRO\"}},\"range\":{\"@timestamp\":{\"gte\":1449442800000,\"lte\":1449529199999}}},\"must_not\":[]}},\"query\":{\"match\":{\"key\":\"match.post\"}}}}}]]]; nested: QueryParsingException[[logstash-2015.12.07] No filter registered for [@timestamp]]; }{[G9r0YNWZSEG7-tOgMuDNWQ][logstash-2015.12.08][0]: SearchParseException[[logstash-2015.12.08][0]: from[-1],size[-1],sort[<custom:\"@timestamp\": org.elasticsearch.i in /root/test/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php on line 644

The main thing seems this:

No filter registered for [@timestamp]

But I have this field in any item.

Besides, this query generated by Kibana 4, works fine.

{"size":500,"sort":{"@timestamp":"desc"},"query":{"filtered":{"query":{"query_string":{"analyze_wildcard":true,"query":"(user_id: 561226f2fef3874e058b4568) and  (key:match.post)"}},"filter":{"bool":{"must":[{"query":{"match":{"environment":{"query":"PRO","type":"phrase"}}}},{"range":{"@timestamp":{"gte":1449442800000,"lte":1449529199999}}}],"must_not":[]}}}},"highlight":{"pre_tags":["@kibana-highlighted-field@"],"post_tags":["@/kibana-highlighted-field@"],"fields":{"*":{}}},"aggs":{"2":{"date_histogram":{"field":"@timestamp","interval":"30m","pre_zone":"+01:00","pre_zone_adjust_large_interval":true,"min_doc_count":0,"extended_bounds":{"min":1449442800000,"max":1449529199999}}}},"fields":["*","_source"],"script_fields":{},"fielddata_fields":["@timestamp"]}

Whats is wrong?

Thanks a lot to all.

2

I found the error.

I made a mistake in the query.

'must' clausule content (inside bool filter) must be an array.

So my php code need to be like this.

<?php

require 'vendor/autoload.php';

use Monolog\Logger;
use Monolog\Handler\StreamHandler;
use Elasticsearch\ClientBuilder;

$logger = new Logger('name');

$hosts = [
    'localhost:9200', // IP + Port
];

$logger = ClientBuilder::defaultLogger('/tmp/el.log');

$client = ClientBuilder::create() // Instantiate a new ClientBuilder
  ->setHosts($hosts)        // Set the hosts
  ->setLogger($logger)      // Set the logger with a default logger
  ->build();

$params = [
  'index' => 'logstash*',
  'size' => 500,
  'type' => 'fluentd',
  'body' => [
    'sort' => [
      '@timestamp' => ['order' => 'asc'],
     ],
     'query' => [
       'filtered' => [
         'filter' => [
           'bool' => [
             'must' => [
               ['query' => [
                 'match' => ['environment' => 'PRO']
               ]],
               ['range' => [
                   '@timestamp' => [
                     'gte' => 1449442800000,
                     'lte' => 1449529199999
                   ]
               ]],
             'must_not' => []
           ]
         ],
         'query' => [
           'match' => ['user_id' => '561226f2fef3874e058b4568'],
           'match' => ['key' => 'match.post']
         ]
       ]
     ]
   ]
];

$response = $client->search($params);
 
echo $response['hits']['total'] . "\n";
foreach($response['hits']['hits'] as $item) {
  echo $item['_source']['key'] . "\n";
  echo $item['_source']['environment'] . "\n";
  echo "\n";
}