How to get Today's log in kibana dev Tools

[Fosiul_Alam]

I am using Kibana dev tools to return all lines by using bellow Query in DEV tools, but instead of giving all matches, its gives only dates from when in installed this kibana.

GET filebeat-*/_search
{
  "query": {
    "match_all": {


    }
  }
}

but its only giving limited lines with date from when i installed elastic search i.e from 2019-07-29T04:57:04.118Z , but not recent.

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 10000,
      "relation" : "gte"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "filebeat-7.2.0-2019.07.27-000001",
        "_type" : "_doc",
        "_id" : "XOQWPGwBUSUoRIJv1eua",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2019-07-29T04:57:04.118Z",
          "log" : {
            "offset" : 1404504,
            "file" : {
              "path" : "/root/root/standalone/log/test.log"
            }
          },
          "input" : {
            "type" : "log"
          },

          "message" : "04:57:03,425 TRACE [com.abc.jca.sockets.test] (default-threads - 5) MessageProcessor - ABC"
        }
      },


 {
        "_index" : "filebeat-7.2.0-2019.07.27-000001",
        "_type" : "_doc",
        "_id" : "XeQWPGwBUSUoRIJv1eua",
        "_score" : 1.0,
        "_source" : {
          "@timestamp" : "2019-07-29T04:57:04.118Z",
          "log" : {
            "offset" : 1404669,
            "file" : {
              "path" : "/root/root/standalone/log/test.log"
            }
          },
          "message" : "04:57:03,425 TRACE [com.ab.jca.sockets.abc] (default-threads - 5) MessageProcessor - ",

      },

Do i need to pass any parameter to get all matches , specially if i want to get from todays log, how shall i pass it ?

Thanks

[rorixrebel (Miguel Valdes)]

if you just want to get everything in that index you can just do

GET filebeat-*/_search

by default it returns 20 results but you can adjust.
if you want specific dates then you need to add them to your query with range, something like:

GET filebeat-*/_search
 {
  "query": {
    "range": {
      "@timestamp": {
        "gte": "now-1d",
        "lte": "1d"
      }
    }
  }
}

[Fosiul_Alam]

Hi
Thanks
How do i change from 20 to All out put ? also that gte formate giving bellow errror

{
  "error": {
    "root_cause": [
      {
        "type": "parse_exception",
        "reason": "failed to parse date field [1d] with format [strict_date_optional_time||epoch_millis]: [Text '1d' could not be parsed, unparsed text found at index 1]"
      }
    ],
    "type": "search_phase_execution_exception",
    "reason": "all shards failed",
    "phase": "query",
    "grouped": true,
    "failed_shards": [
      {
        "shard": 0,
        "index": "filebeat-7.2.0-2019.07.27-000001",
        "node": "U1V34js7Q7KN2HG4Cus8qA",
        "reason": {
          "type": "parse_exception",
          "reason": "failed to parse date field [1d] with format [strict_date_optional_time||epoch_millis]: [Text '1d' could not be parsed, unparsed text found at index 1]",
          "caused_by": {
            "type": "date_time_parse_exception",
            "reason": "Text '1d' could not be parsed, unparsed text found at index 1"
          }
        }
      }
    ]
  },
  "status": 400
}```

[rorixrebel (Miguel Valdes)]

add the parameter size: # at the top of your query to define a size.

    GET filebeat-*/_search
     { 
         "size": 1000,
          "query": { ....
        }
     }

Be mindful that giving it a huge number might be a problem - some might suggest using the scroll api - https://www.elastic.co/guide/en/elasticsearch/reference/6.4/search-request-scroll.html

i may have messed up the format - you can find the right format in here
https://www.elastic.co/guide/en/elasticsearch/reference/6.4/common-options.html#date-math