Elasticsearch query time range issue

But could you share the output of

GET /

?

AND PLEASE FORMAT YOUR CODE ACCORDINGLY TO THE GUIDELINES OF THIS FORUM.

Instead, paste the text and format it with </> icon or pairs of triple backticks (```), and check the preview window to make sure it's properly formatted before posting it.

Hi Dadoonet,

As per reference doc used below
curl -X POST "ip:port/my-index-000001/_pit?keep_alive=1m&pretty"

Output
{"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}

Also tried GET
curl -X GET "ip:port/my-index-000001/_pit?keep_alive=1m&pretty"
Output
{"error":"Incorrect HTTP method for uri [/my-index-000001/_pit] and method [GET], allowed: [POST]","status":405}

So, if I use search_after, will I get memory consumption.

Can you please provide RAW formatted csv download API, (discussed above in same thread ) if possible, then it would be easy to automate the whole process.

I can close this is issue as early as if everything goes fine.

Is there any reason you don't want to answer the questions I'm asking?

It's hard to help you if you are not answering.

Have a look at Response Data Formats | Elasticsearch Guide [8.11] | Elastic

But also look at: Paginating through a large response | Elasticsearch Guide [8.11] | Elastic

Hi Dadonet,

I was formatting before posting every time but I am not sure where it wrong.

If I get PIT ID, then I am sure, will get the results, something I need to do read privileges security at index level

Below I got from one article -> Point in Time should handle security on aliases · Issue #61547 · elastic/elasticsearch · GitHub
we introduced a new feature called PIT that allows to reuse the same context on multiple queries.
We've decided to merge the feature in advance but there is still one thing that we need to fix/decide. The PIT relies on the concrete index names that were resolved when the PIT was created. That allows to keep the scope of a PIT to only indices that existed when the PIT was created but that makes the security on aliases more challenging. Today we allow aliases to have different permissions than their targeted indices. Even though this feature is deprecated in security at the moment, it is unclear if we'll remove it in the future. So for PIT, we've decided to disallow the creation if an alias with a different permission is used in the creation request. This issue is a placeholder to ensure that we implement this protection before 7.10 .
That should be temporary until the @elastic/es-security team revises the plan to un-deprecate or to remove this problematic use case definitely. If the decision is to un-deprecate we'll of course need to support the use case in PIT but that decision can wait after 7.10 is released.

Ok. Last time I'm asking the question.

What is the output of the following command:

GET /

Which you can translate to:

curl -XGET "ip:port/"

Sorry, I really missed it.

curl -XGET "http://ip:port/"

Output

{
  "name" : "172.x.x.x",
  "cluster_name" : "es-cluster",
  "cluster_uuid" : "ssfbdfmdfdmfdfnmd",
  "version" : {
    "number" : "7.9.0",
    "build_flavor" : "default",
    "build_type" : "rpm",
    "build_hash" : "a479a2a7fce0389512d6a9361301708b92dff667",
    "build_date" : "2020-08-11T21:36:48.204330Z",
    "build_snapshot" : false,
    "lucene_version" : "8.6.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

I edited your post so you can see the difference.

From

Capture d’écran 2021-03-31 à 17.42.34

To

Capture d’écran 2021-03-31 à 17.42.52

So please next time, do not ident the "normal" text.

Back to your question:

"number" : "7.9.0",

Point In Time API has been introduced in 7.10. So you need to upgrade.
7.12.0 is now available.

Thank you so much, will try to upgrade to latest version

Do not indent your text.

```
Hi David,

I have upgraded Elastic Search version to 7.12.0.
Now I am able to generate PIT ID, but getting below error.

curl -X GET "http://ip:port/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "size": 10000,
  "query": {
    "match" : {
      "message" : "exception"
    }
  },
  "pit": {
            "id":  "48myAwcaZGV2MS1hcHBsb2dzLWJlLTIwMjEuMDQuMDUWdF9TcGV1Y0FTbXE2Z3U3NXZ6cG5zdwAWRWpnRU0waGhSMldEczBfQ25HMnBRUQAAAAAAAACKKhZBX1U1ci0yTlN2U29iV3RXV1d0VnR3ABpkZXYxLWFwcGxvZ3MtYmUtMjAyMS4wMy4zMBZBRUNzbWxrWFNOcXdHUjJmUlBfMXFBABZjUDZQYl9zcVJsQ0h1Sk5JS2RaRERnAAAAAAAAAMmZFnBZUWdlNDZEUVhhMXhEcXRxSUVDNHcAGmRldjEtYXBwbG9ncy1iZS0yMDIxLjAzLjMxFk9lQXF6YVNhU1MybTJ4Sm9fd2ZNVUEAFkVqZ0VNMGhoUjJXRHMwX0NuRzJwUVEAAAAAAAAAiikWQV9VNXItMk5TdlNvYld0V1dXdFZ0dwAaZGV2MS1hcHBsb2dzLWJlLTIwMjEuMDQuMDIWcEd5MGxlX2RRV1duOWJERXZ5cVZidwAWWWNmSml6aVdRR21yNFJpTWZ5U3FCdwAAAAAAAAOWqRZMSG9ka2Jsc1FoT3B6WmRVaFhwS193ABpkZXYxLWFwcGxvZ3MtYmUtMjAyMS4wNC4wMRZvOGlCU0I0a1FTeUp3Y1VjUDdTVm9nABZjUDZQYl9zcVJsQ0h1Sk5JS2RaRERnAAAAAAAAAMmaFnBZUWdlNDZEUVhhMXhEcXRxSUVDNHcAGmRldjEtYXBwbG9ncy1iZS0yMDIxLjA0LjAzFmNwMEt2TVhBUTFXNnE2eTBZWFRqa0EAFlljZkppemlXUUdtcjRSaU1meVNxQncAAAAAAAADlqoWTEhvZGtibHNRaE9welpkVWhYcEtfdwAaZGV2MS1hcHBsb2dzLWJlLTIwMjEuMDQuMDQWb1c3UVJfYmFSZml3YjZ6T0dMYUdaZwAWWWNmSml6aVdRR21yNFJpTWZ5U3FCdwAAAAAAAAOWqxZMSG9ka2Jsc1FoT3B6WmRVaFhwS193AAcWcEd5MGxlX2RRV1duOWJERXZ5cVZidwAAFm84aUJTQjRrUVN5SndjVWNQN1NWb2cAABZjcDBLdk1YQVExVzZxNnkwWVhUamtBAAAWdF9TcGV1Y0FTbXE2Z3U3NXZ6cG5zdwAAFm9XN1FSX2JhUmZpd2I2ek9HTGFHWmcAABZPZUFxemFTYVNTMm0yeEpvX3dmTVVBAAAWQUVDc21sa1hTTnF3R1IyZlJQXzFxQQAA",
            "keep_alive": "5m"
  },
  "sort": [
    {"@timestamp": "asc"}
  ]
}
'
**Output** 

{
  "error" : {
    "root_cause" : [
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [51609]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [35369]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [51610]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [235177]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [235178]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [235179]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [35370]"
      }
    ],
    "type" : "search_phase_execution_exception",
    "reason" : "all shards failed",
    "phase" : "query",
    "grouped" : true,
    "failed_shards" : [
      {
        "shard" : 0,
        "index" : "dev-applogs-be-2021.03.30",
        "node" : "cP6Pb_sqRlCHuJNIKdZDDg",
      "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [35369]"
        }
      },
      {
        "shard" : 0,
        "index" : "dev-applogs-be-2021.04.01",
        "node" : "cP6Pb_sqRlCHuJNIKdZDDg",
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [51610]"
        }
      },
      {
        "shard" : 0,
        "index" : "dev-applogs-be-2021.04.02",
        "node" : "YcfJiziWQGmr4RiMfySqBw",
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [235177]"
        }
      },
      {
        "shard" : 0,
        "index" : "dev-applogs-be-2021.04.03",
        "node" : "YcfJiziWQGmr4RiMfySqBw",
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [235178]"
        }
      },
      ```
**Elastisearch logs** 


[2021-04-05T03:03:01,751][WARN ][c.a.j.SdkMBeanRegistrySupport] [172.x.x.x]
java.security.AccessControlException: access denied ("javax.management.MBeanServerPermission" "findMBeanServer")
        at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472) ~[?:?]
        at java.security.AccessController.checkPermission(AccessController.java:1036) ~[?:?]
        at java.lang.SecurityManager.checkPermission(SecurityManager.java:408) ~[?:?]
        at javax.management.MBeanServerFactory.checkPermission(MBeanServerFactory.java:413) ~[?:?]
        at javax.management.MBeanServerFactory.findMBeanServer(MBeanServerFactory.java:361) ~[?:?]
        at com.amazonaws.jmx.MBeans.getMBeanServer(MBeans.java:111) ~[aws-java-sdk-core-1.11.749.jar:?]
        at com.amazonaws.jmx.MBeans.registerMBean(MBeans.java:50) ~[aws-java-sdk-core-1.11.749.jar:?]
I have resolved java.security.AccessControlException: access denied ("javax.management.MBeanServerPermission" "findMBeanServer") 
by installing discovery plugin in cluster. But getting earlier issue is 

 "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [36544]"

What is the full output of the cluster stats API?

Output 
{
  "_nodes" : {
    "total" : 3,
    "successful" : 3,
    "failed" : 0
  },
  "cluster_name" : "es-cluster",
  "cluster_uuid" : "rMdbBBhVSIiMLIu34RVHKg",
  "timestamp" : 1617616250511,
  "status" : "green",
  "indices" : {
    "count" : 300,
    "shards" : {
      "total" : 600,
      "primaries" : 300,
      "replication" : 1.0,
      "index" : {
        "shards" : {
          "min" : 2,
          "max" : 2,
          "avg" : 2.0
        },
        "primaries" : {
          "min" : 1,
          "max" : 1,
          "avg" : 1.0
        },
        "replication" : {
          "min" : 1.0,
          "max" : 1.0,
          "avg" : 1.0
        }
      }
    },
    "docs" : {
      "count" : 710964173,
      "deleted" : 5661746
    },
    "store" : {
      "size" : "356.5gb",
      "size_in_bytes" : 382824864415,
      "reserved" : "0b",
      "reserved_in_bytes" : 0
    },
    "fielddata" : {
      "memory_size" : "0b",
      "memory_size_in_bytes" : 0,
      "evictions" : 0
    },
    "query_cache" : {
      "memory_size" : "48kb",
      "memory_size_in_bytes" : 49200,
      "total_count" : 4043,
      "hit_count" : 816,
      "miss_count" : 3227,
      "cache_size" : 60,
      "cache_count" : 60,
      "evictions" : 0
    },
    "completion" : {
      "size" : "0b",
      "size_in_bytes" : 0
    },
    "segments" : {
      "count" : 6838,
      "memory" : "158.7mb",
      "memory_in_bytes" : 166508156,
      "terms_memory" : "119.6mb",
      "terms_memory_in_bytes" : 125460912,
      "stored_fields_memory" : "6.2mb",
      "stored_fields_memory_in_bytes" : 6602528,
      "term_vectors_memory" : "0b",
      "term_vectors_memory_in_bytes" : 0,
      "norms_memory" : "15.3mb",
      "norms_memory_in_bytes" : 16059840,
      "points_memory" : "0b",
      "points_memory_in_bytes" : 0,
      "doc_values_memory" : "17.5mb",
      "doc_values_memory_in_bytes" : 18384876,
      "index_writer_memory" : "1.9gb",
      "index_writer_memory_in_bytes" : 2087335820,
      "version_map_memory" : "20.7mb",
      "version_map_memory_in_bytes" : 21710999,
      "fixed_bit_set" : "4.8mb",
      "fixed_bit_set_memory_in_bytes" : 5099520,
      "max_unsafe_auto_id_timestamp" : 1617612493206,
      "file_sizes" : { }
    },
    "mappings" : {
      "field_types" : [
        {
          "name" : "boolean",
          "count" : 1025,
          "index_count" : 278
        },
        {
          "name" : "byte",
          "count" : 32,
          "index_count" : 32
        },
        {
          "name" : "date",
          "count" : 941,
          "index_count" : 289
        },
        {
          "name" : "float",
          "count" : 584,
          "index_count" : 142
        },
        {
          "name" : "geo_point",
          "count" : 227,
          "index_count" : 35
        },
        {
          "name" : "half_float",
          "count" : 62,
          "index_count" : 17
        },
        {
          "name" : "histogram",
          "count" : 32,
          "index_count" : 32
        },
        {
          "name" : "integer",
          "count" : 154,
          "index_count" : 7
        },
        {
          "name" : "ip",
          "count" : 419,
          "index_count" : 35
        },
        {
          "name" : "keyword",
          "count" : 27956,
          "index_count" : 290
        },
        {
          "name" : "long",
          "count" : 5254,
          "index_count" : 281
        },
        {
          "name" : "nested",
          "count" : 25,
          "index_count" : 11
        },
        {
          "name" : "object",
          "count" : 12140,
          "index_count" : 287
        },
        {
          "name" : "scaled_float",
          "count" : 64,
          "index_count" : 32
        },
        {
          "name" : "text",
          "count" : 12364,
          "index_count" : 276
        }
      ]
    },
    "analysis" : {
      "char_filter_types" : [ ],
      "tokenizer_types" : [ ],
      "filter_types" : [ ],
      "analyzer_types" : [ ],
      "built_in_char_filters" : [ ],
      "built_in_tokenizers" : [ ],
      "built_in_filters" : [ ],
      "built_in_analyzers" : [ ]
    },
    "versions" : [
      {
        "version" : "7.9.0",
        "index_count" : 238,
        "primary_shard_count" : 238,
        "total_primary_size" : "173.7gb",
        "total_primary_bytes" : 186554870137
      },
      {
        "version" : "7.12.0",
        "index_count" : 62,
        "primary_shard_count" : 62,
        "total_primary_size" : "4.4gb",
        "total_primary_bytes" : 4757646956
      }
    ]
  },
  "nodes" : {
    "count" : {
      "total" : 3,
      "coordinating_only" : 0,
      "data" : 3,
      "data_cold" : 3,
      "data_content" : 3,
      "data_frozen" : 3,
      "data_hot" : 3,
      "data_warm" : 3,
      "ingest" : 3,
      "master" : 3,
      "ml" : 3,
      "remote_cluster_client" : 3,
      "transform" : 3,
      "voting_only" : 0
    },
    "versions" : [
      "7.12.0"
    ],
    "os" : {
      "available_processors" : 12,
      "allocated_processors" : 12,
      "names" : [
        {
          "name" : "Linux",
          "count" : 3
        }
      ],
      "pretty_names" : [
        {
          "pretty_name" : "Amazon Linux 2",
          "count" : 3
        }
      ],
      "architectures" : [
        {
          "arch" : "amd64",
          "count" : 3
        }
      ],
      "mem" : {
        "total" : "46.4gb",
        "total_in_bytes" : 49873268736,
        "free" : "453.6mb",
        "free_in_bytes" : 475656192,
        "used" : "46gb",
        "used_in_bytes" : 49397612544,
        "free_percent" : 1,
        "used_percent" : 99
      }
    },
    "process" : {
      "cpu" : {
        "percent" : 10
      },
      "open_file_descriptors" : {
        "min" : 1959,
        "max" : 2065,
        "avg" : 2018
      }
    },
    "jvm" : {
      "max_uptime" : "1.4h",
      "max_uptime_in_millis" : 5269275,
      "versions" : [
        {
          "version" : "15.0.1",
          "vm_name" : "OpenJDK 64-Bit Server VM",
          "vm_version" : "15.0.1+9",
          "vm_vendor" : "AdoptOpenJDK",
          "bundled_jdk" : true,
          "using_bundled_jdk" : true,
          "count" : 3
        }
      ],
      "mem" : {
        "heap_used" : "9.8gb",
        "heap_used_in_bytes" : 10523690496,
        "heap_max" : "23.2gb",
        "heap_max_in_bytes" : 24939331584
      },
      "threads" : 213
    },
    "fs" : {
      "total" : "1.4tb",
      "total_in_bytes" : 1610574925824,
      "free" : "1tb",
      "free_in_bytes" : 1194139471872,
      "available" : "1tb",
      "available_in_bytes" : 1194139471872
    },
    "plugins" : [
      {
        "name" : "discovery-ec2",
        "version" : "7.12.0",
        "elasticsearch_version" : "7.12.0",
        "java_version" : "1.8",
        "description" : "The EC2 discovery plugin allows to use AWS API for the unicast discovery mechanism.",
        "classname" : "org.elasticsearch.discovery.ec2.Ec2DiscoveryPlugin",
        "extended_plugins" : [ ],
        "has_native_controller" : false,
        "licensed" : false,
        "type" : "isolated"
      },
      {
        "name" : "repository-s3",
        "version" : "7.12.0",
        "elasticsearch_version" : "7.12.0",
        "java_version" : "1.8",
        "description" : "The S3 repository plugin adds S3 repositories",
        "classname" : "org.elasticsearch.repositories.s3.S3RepositoryPlugin",
        "extended_plugins" : [ ],
        "has_native_controller" : false,
        "licensed" : false,
        "type" : "isolated"
      }
    ],
    "network_types" : {
      "transport_types" : {
        "security4" : 3
      },
      "http_types" : {
        "security4" : 3
      }
    },
    "discovery_types" : {
      "zen" : 3
    },
    "packaging_types" : [
      {
        "flavor" : "default",
        "type" : "rpm",
        "count" : 3
      }
    ],
    "ingest" : {
      "number_of_pipelines" : 7,
      "processor_stats" : {
        "conditional" : {
          "count" : 1475794,
          "failed" : 0,
          "current" : 0,
          "time" : "20.3s",
          "time_in_millis" : 20378
        },
        "geoip" : {
          "count" : 1475794,
          "failed" : 0,
          "current" : 0,
          "time" : "11.5s",
          "time_in_millis" : 11532
        },
        "gsub" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "pipeline" : {
          "count" : 5903176,
          "failed" : 0,
          "current" : 0,
          "time" : "52s",
          "time_in_millis" : 52040
        },
        "script" : {
          "count" : 0,
          "failed" : 0,
          "current" : 0,
          "time" : "0s",
          "time_in_millis" : 0
        },
        "user_agent" : {
          "count" : 1475794,
          "failed" : 0,
          "current" : 0,
          "time" : "7.9s",
          "time_in_millis" : 7950
        }
      }
    }
  }
}

Hi Christian,

I did not get it. 

I am trying to get more than 10,000 records  using PIT ID so that I have upgraded to latest version.

But getting same error.

"reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [235178]"
        }
   

I am not getting what is this.
(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

Hi David / Stephen,

My requirement is changed.
I will create a new thread with exact requirement.

Thank you so much for supporting.

Regards,
Suresh

Hi David,

I am re-opening this thread as I need to get some fields using PIT.

Below is the error, after adding PIT 

{
  "error" : {
    "root_cause" : [
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [832668]"
      },
      {
        "type" : "search_context_missing_exception",
        "reason" : "No search context found for id [150759]"
      },
     {
        "shard" : 0,
        "index" : "dev2-applogs-be-2021.04.08",
        "node" : "cP6Pb_sqRlCHuJNIKdZDDg",
        "reason" : {
          "type" : "search_context_missing_exception",
          "reason" : "No search context found for id [150761]"
        }


Query

 curl -X GET "http://ip:port_search?pretty" -H 'Content-Type: application/json' -d'
{
  "size": 10000,
  "_source": [ "kubernetes.container.name", "kubernetes.namespace", "message", "@timestamp" ],
  "query": {
        "bool": {
            "must": {
                "match": {
                    "message": "exception"
                }
            },
            "filter": {
                "range": {
                    "@timestamp": {
                        "gte": "2021-04-06T01:00:00",
                        "lte": "2021-04-06T18:00:00"
                    }
                }
            }
          }
         },

  "pit": {
            "id":  "48myAwcaZGV2Mi1hcHBsb2dzLWJlLTIwMjEuMDQuMDgWY1d0MzhzSERRZnF6eGNNLVJubko0QQAWY1A2UGJfc3FSbENIdUpOSUtkWkREZwAAAAAAAAJM6RZvRHNRTkY0MlF2aWpuVWEweHpIdmN3ABpkZXYyLWFwcGxvZ3MtYmUtMjAyMS4wNC4wNRZqWlAyYXdwSVRTS1hqRUh4bGI2Uk93ABZZY2ZKaXppV1FHbXI0UmlNZnlTcUJ3AAAAAAAADLSdFm95VWZUVFQzUnBXMU5xem9IX0hNbkEAGmRldjItYXBwbG9ncy1iZS0yMDIxLjA0LjA3FmFEQWotRURYU2pHbjVBdm1mTWtSNGcAFlljZkppemlXUUdtcjRSaU1meVNxQncAAAAAAAAMtJ4Wb3lVZlRUVDNScFcxTnF6b0hfSE1uQQAaZGV2Mi1hcHBsb2dzLWJlLTIwMjEuMDQuMDIWTmNudXJCdS1UMldVQWtHcnhyR3NDZwAWWWNmSml6aVdRR21yNFJpTWZ5U3FCdwAAAAAAAAy0nBZveVVmVFRUM1JwVzFOcXpvSF9ITW5BABpkZXYyLWFwcGxvZ3MtYmUtMjAyMS4wNC4wNhZDUGRfUF9RMVJEdVMxYmpLNlViSWJBABZFamdFTTBoaFIyV0RzMF9DbkcycFFRAAAAAAAAAeJMFnVVazE1dXRWVDJDTmdKWFUtekNRMmcAGmRldjItYXBwbG9ncy1iZS0yMDIxLjA0LjA0FkRNUUpHd1l3VHlLbjVJM0JOSlZRSGcAFmNQNlBiX3NxUmxDSHVKTklLZFpERGcAAAAAAAACTOgWb0RzUU5GNDJRdmlqblVhMHh6SHZjdwAaZGV2Mi1hcHBsb2dzLWJlLTIwMjEuMDQuMDMWZlhWT2FjR3hRRktFSDlhekNZNlBZQQAWY1A2UGJfc3FSbENIdUpOSUtkWkREZwAAAAAAAAJM5xZvRHNRTkY0MlF2aWpuVWEweHpIdmN3AAcWRE1RSkd3WXdUeUtuNUkzQk5KVlFIZwAAFmNXdDM4c0hEUWZxenhjTS1Sbm5KNEEAABZmWFZPYWNHeFFGS0VIOWF6Q1k2UFlBAAAWQ1BkX1BfUTFSRHVTMWJqSzZVYkliQQAAFmFEQWotRURYU2pHbjVBdm1mTWtSNGcAABZqWlAyYXdwSVRTS1hqRUh4bGI2Uk93AAAWTmNudXJCdS1UMldVQWtHcnhyR3NDZwAA",
            "keep_alive": "1m"
  },
  "sort": [
    {"@timestamp": "asc"}
  ]
}
'

Stop formatting as code the text part. Please format only the code part.

Is there anything you don't understand in this sentence that I need to clarify?

1 Like
curl -X GET "http://ip:port_search?pretty" -H 'Content-Type: application/json' -d'
{
  "size": 10000,
  "_source": [ "kubernetes.container.name", "kubernetes.namespace", "message", "@timestamp" ],
  "query": {
        "bool": {
            "must": {
                "match": {
                    "message": "exception"
                }
            },
            "filter": {
                "range": {
                    "@timestamp": {
                        "gte": "2021-04-06T01:00:00",
                        "lte": "2021-04-06T18:00:00"
                    }
                }
            }
          }
         },

  "pit": {
            "id":  "48myAwcaZGV2Mi1hcHBsb2dzLWJlLTIwMjEuMDQuMDgWY1d0MzhzSERRZnF6eGNNLVJubko0QQAWY1A2UGJfc3FSbENIdUpOSUtkWkREZwAAAAAAAAJM6RZvRHNRTkY0MlF2aWpuVWEweHpIdmN3ABpkZXYyLWFwcGxvZ3MtYmUtMjAyMS4wNC4wNRZqWlAyYXdwSVRTS1hqRUh4bGI2Uk93ABZZY2ZKaXppV1FHbXI0UmlNZnlTcUJ3AAAAAAAADLSdFm95VWZUVFQzUnBXMU5xem9IX0hNbkEAGmRldjItYXBwbG9ncy1iZS0yMDIxLjA0LjA3FmFEQWotRURYU2pHbjVBdm1mTWtSNGcAFlljZkppemlXUUdtcjRSaU1meVNxQncAAAAAAAAMtJ4Wb3lVZlRUVDNScFcxTnF6b0hfSE1uQQAaZGV2Mi1hcHBsb2dzLWJlLTIwMjEuMDQuMDIWTmNudXJCdS1UMldVQWtHcnhyR3NDZwAWWWNmSml6aVdRR21yNFJpTWZ5U3FCdwAAAAAAAAy0nBZveVVmVFRUM1JwVzFOcXpvSF9ITW5BABpkZXYyLWFwcGxvZ3MtYmUtMjAyMS4wNC4wNhZDUGRfUF9RMVJEdVMxYmpLNlViSWJBABZFamdFTTBoaFIyV0RzMF9DbkcycFFRAAAAAAAAAeJMFnVVazE1dXRWVDJDTmdKWFUtekNRMmcAGmRldjItYXBwbG9ncy1iZS0yMDIxLjA0LjA0FkRNUUpHd1l3VHlLbjVJM0JOSlZRSGcAFmNQNlBiX3NxUmxDSHVKTklLZFpERGcAAAAAAAACTOgWb0RzUU5GNDJRdmlqblVhMHh6SHZjdwAaZGV2Mi1hcHBsb2dzLWJlLTIwMjEuMDQuMDMWZlhWT2FjR3hRRktFSDlhekNZNlBZQQAWY1A2UGJfc3FSbENIdUpOSUtkWkREZwAAAAAAAAJM5xZvRHNRTkY0MlF2aWpuVWEweHpIdmN3AAcWRE1RSkd3WXdUeUtuNUkzQk5KVlFIZwAAFmNXdDM4c0hEUWZxenhjTS1Sbm5KNEEAABZmWFZPYWNHeFFGS0VIOWF6Q1k2UFlBAAAWQ1BkX1BfUTFSRHVTMWJqSzZVYkliQQAAFmFEQWotRURYU2pHbjVBdm1mTWtSNGcAABZqWlAyYXdwSVRTS1hqRUh4bGI2Uk93AAAWTmNudXJCdS1UMldVQWtHcnhyR3NDZwAA",
            "keep_alive": "1m"
  },
  "sort": [
    {"@timestamp": "asc"}
  ]
}
'