How to download data from an elasticsearch index

Juan_David_Jaramillo · April 6, 2021, 4:11pm

Hi, guy's
How can i download all the data of an index in elasticsearch?
it downloads the CSV but it does not show all the data, I need to download the whole index, please help me?

dadoonet · April 6, 2021, 4:24pm

You can use:

the size and from parameters to display by default up to 10000 records to your users. If you want to change this limit, you can change index.max_result_window setting but be aware of the consequences (ie memory).
the search after feature to do deep pagination.
the Scroll API if you want to extract a resultset to be consumed by another tool later.

Juan_David_Jaramillo · April 7, 2021, 2:46pm

I was looking, but it did not let me download all the information, what else can I do to download a whole index, thanks.

dadoonet · April 7, 2021, 3:49pm

The scroll api as I said.

Juan_David_Jaramillo · April 15, 2021, 4:30pm

I have already done the steps you ask me and I got the "scroll_id", then what do I do to download the index? The documentation does not explain how to do it, thank you.

POST /up-time*/_search?scroll=1m
{
  "size": 100,
  "query":  {
    "match": {
      "message": "foo"
    }
  }
}

and this is the result

{
  "_scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFld4UExiOFlhVHo2RHpwd0VRLV9TNmcAAAAAAAADORZ6SUlmc3FiM1JMNmlDMS0xcVpEeFlR",
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

then it asks me to do this

POST /_search/scroll
{
  "scroll": "1m",
  "scroll_id": "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFld4UExiOFlhVHo2RHpwd0VRLV9TNmcAAAAAAAAJUBZ6SUlmc3FiM1JMNmlDMS0xcVpEeFlR"
}

and I get this result as it says in the documentation

{
  "_scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFld4UExiOFlhVHo2RHpwd0VRLV9TNmcAAAAAAAAOphZ6SUlmc3FiM1JMNmlDMS0xcVpEeFlR",
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

finally it asks me to obtain the results, by obtaining the "scroll_id".

GET /_search?scroll=1m
{
  "sort": [
    "_doc"
  ]
}

So I get this result of all the "scroll_id" in the index

#! this request accesses system indices: [.apm-agent-configuration, .apm-custom-link, .async-search, .kibana_1, .kibana_7.12.0_001, .kibana_task_manager_1, .kibana_task_manager_7.12.0_001, .ml-config, .security-7, .tasks, .triggered_watches, .watches], but in a future major version, direct access to system indices will be prevented by default
{
  "_scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDnF1ZXJ5VGhlbkZldGNoKRZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAgWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAkWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAcWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEA8WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEA0WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAoWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAsWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAwWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBAWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEA4WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBEWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBIWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBMWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBQWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBUWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBgWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBYWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBcWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBkWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEC4WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBoWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBsWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEB8WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBwWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEB0WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEB4WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECAWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAED4WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECEWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECIWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECMWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECQWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECcWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECUWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECYWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECoWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECgWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECkWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECsWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECwWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEC0WeklJZnNxYjNSTDZpQzEtMXFaRHhZUQ==",
  "took" : 664,
  "timed_out" : false,
  "_shards" : {
    "total" : 41,
    "successful" : 41,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 415280,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : ".async-search",
        "_type" : "_doc",
        "_id" : "lIy7gwquQJGPY_95V_L2pQ",
        "_score" : null,
        "_source" : {
          "result" : "48myAwFERm14SmVUZG5kM0YxVVVwSFVGbGZPVFZXWDB3eWNGRWJla2xKWm5OeFlqTlNURFpwUXpFdE1YRmFSSGhaVVRvek9ERXkAAQEAAH/AAAAAAAAAAQEOZGF0ZV9oaXN0b2dyYW0KdGltZXNlcmllcwoFCXRpbWVGaWVsZAAJdGltZXN0YW1wBWluZGV4ABpraWJhbmFfc2FtcGxlX2RhdGFfZmxpZ2h0cw5pbnRlcnZhbFN0cmluZwACMWgKYnVja2V0U2l6ZQEAAA4QCHNlcmllc0lkACQ2MWNhNTdmMS00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTcEAAEGDkFtZXJpY2EvQm9nb3RhAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAEBAAABeLJtIoABAAABeLeTfoAAAAAAAAAAAAAACWRhdGVfdGltZSdzdHJpY3RfZGF0ZV9vcHRpb25hbF90aW1lfHxlcG9jaF9taWxsaXMOQW1lcmljYS9Cb2dvdGEAAAAZAAABeLJtIoAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXiypBEAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4str/gAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLMR7gAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXizSNyAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4s3/LAAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLO2uYAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXiz7agAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4tCSWgAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLRbhQAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi0knOAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4tMliAAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLUAUIAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi1Nz8AAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4tW4tgAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLWlHAAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi13AqAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4thL5AAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLZJ54AAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi2gNYAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4trfEgAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLbuswAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi3JaGAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4t1yQAAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLeTfoAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAgABAQEAAAAAAHcAAAAAAAABeLeXlDUAAAF426QYNQ==",
          "headers" : {
            "_xpack_security_authentication" : "48myAwAHZWxhc3RpYwEJc3VwZXJ1c2VyCgEJX3Jlc2VydmVkBQEAAAEAFGVsa3NlcnZlci1WaXJ0dWFsQm94CHJlc2VydmVkCHJlc2VydmVkAAAKAA=="
          },
          "expiration_time" : 1618592667701,
          "response_headers" : { }
        },
        "sort" : [
          0
        ]
      },
      {
        "_index" : ".kibana-event-log-7.11.1-000001",
        "_type" : "_doc",
        "_id" : "lUk_y3cBlIpB_Q5C4UEL",
        "_score" : null,
        "_source" : {
          "@timestamp" : "2021-02-22T19:38:15.035Z",
          "event" : {
            "provider" : "eventLog",
            "action" : "starting"
          },
          "message" : "eventLog starting",
          "ecs" : {
            "version" : "1.6.0"
          },
          "kibana" : {
            "server_uuid" : "14ded7a1-87e4-4ba2-882b-45f9c7a05ba8"
          }
        },
        "sort" : [
          0
        ]
      },
      {
        "_index" : ".kibana-event-log-7.11.1-000002",
        "_type" : "_doc",
        "_id" : "zCNgangBpucHuqdGVm9r",
        "_score" : null,
        "_source" : {
          "@timestamp" : "2021-03-25T17:13:21.756Z",
          "event" : {
            "provider" : "eventLog",
            "action" : "stopping"
          },
          "message" : "eventLog stopping",
          "ecs" : {
            "version" : "1.6.0"
          },
          "kibana" : {
            "server_uuid" : "14ded7a1-87e4-4ba2-882b-45f9c7a05ba8"
          }
        },
        "sort" : [
          0
        ]
      },

dadoonet · April 15, 2021, 5:14pm

I believe that the problem is that

POST /up-time*/_search?scroll=1m
{
  "size": 100,
  "query":  {
    "match": {
      "message": "foo"
    }
  }
}

Gives absolutely no hits.
So there is nothing to scroll here as the resultset is empty.

If you want to get the full index, run instead:

POST /up-time*/_search?scroll=1m
{
  "size": 100
}

Then get the 100 hits, do whatever you want with that.
Get also the scroll_id and use it for the next calls.

POST /_search/scroll                                                               
{
  "scroll" : "1m",                                                                 
  "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==" 
}

Until the resultset is empty again.

Sandeep_Raju · April 15, 2021, 7:50pm

If your index is small, you can just go to discover,select your index.
Then save your search with name.
Then share -> download Report as CSV-> management->report-> Download report.

Juan_David_Jaramillo · April 15, 2021, 8:26pm

Es un indice con informacion de 3 meses o mas , por lo tanto no se puede descargar de esa forma.

Sandeep_Raju · April 15, 2021, 8:34pm

One other way is by using Elasticdump Tool to download an index data.

Juan_David_Jaramillo · April 15, 2021, 8:37pm

Bajo el paquete en este sitio? elasticdump - npm

Sandeep_Raju · April 15, 2021, 9:49pm

Yes you can use the above mentioned website.
I used the instructions from below website:

There is also a github link:

Juan_David_Jaramillo · April 16, 2021, 6:46pm

I am also trying with this one so that it compresses the data from the elastic server to my documents and I get the same error and it does not let me download the data, which I can do now and tried in several ways, also the cluster is active

elasticdump \ --input=http://localhost:9200/up-time* \ --output=$ \ | gzip > /home/elk-server/up-time*.json.gz
Fri, 16 Apr 2021 18:45:44 GMT | Error Emitted => {"errors":["`input` is a required input","`output` is a required input"]}

Juan_David_Jaramillo · April 16, 2021, 8:29pm

it ready worked for me in the following way

npm install elasticdump -g

sudo apt-get update
sudo apt-get upgrade
shutdown -r 0

elasticdump \
> --input=http://localhost:9200/up-time*/ \
> --output=up-time.json \
> --type=data
Fri, 16 Apr 2021 20:18:33 GMT | starting dump
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 0)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 100)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 200)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 300)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 400)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 500)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 600)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 700)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 800)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 900)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 1000)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1100)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1200)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1300)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1400)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1500)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 1600)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 1700)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 1800)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 1900)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 2000)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2100)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2200)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2300)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2400)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2500)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:58 GMT | got 100 objects from source elasticsearch (offset: 2600)
Fri, 16 Apr 2021 20:18:58 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:58 GMT | got 100 objects from source elasticsearch (offset: 2700)
Fri, 16 Apr 2021 20:18:58 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:58 GMT | got 100 objects from source elasticsearch (offset: 2800)
Fri, 16 Apr 2021 20:18:58 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:58 GMT | got 43 objects from source elasticsearch (offset: 2900)
Fri, 16 Apr 2021 20:18:58 GMT | sent 43 objects to destination file, wrote 43
Fri, 16 Apr 2021 20:18:58 GMT | got 0 objects from source elasticsearch (offset: 2943)
Fri, 16 Apr 2021 20:18:58 GMT | Total Writes: 2943
Fri, 16 Apr 2021 20:18:58 GMT | dump complete
root@elkserver-VirtualBox:~# ls
go  up-time.json

system · May 14, 2021, 8:29pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
[es/search] failed: [search_phase_execution_exception] all shards failed Elasticsearch language-clients	7	13347	July 10, 2023
Python ElasticSearch - How to increase query size limit Elasticsearch language-clients	5	6407	March 20, 2021
Elasticsearch input plugin size Logstash	8	1391	August 31, 2021
Aggregation of more than 10000 records Elasticsearch	5	10711	September 20, 2018
Get query is showing only 10 results while searching for a particular data Elasticsearch	3	38851	July 3, 2017

How to download data from an elasticsearch index

Related Topics