How to download data from an elasticsearch index

Juan_David_Jaramillo · April 6, 2021, 4:11pm

Hi, guy's
How can i download all the data of an index in elasticsearch?
it downloads the CSV but it does not show all the data, I need to download the whole index, please help me?

dadoonet · April 6, 2021, 4:24pm

You can use:

the size and from parameters to display by default up to 10000 records to your users. If you want to change this limit, you can change index.max_result_window setting but be aware of the consequences (ie memory).
the search after feature to do deep pagination.
the Scroll API if you want to extract a resultset to be consumed by another tool later.

Juan_David_Jaramillo · April 7, 2021, 2:46pm

I was looking, but it did not let me download all the information, what else can I do to download a whole index, thanks.

dadoonet · April 7, 2021, 3:49pm

The scroll api as I said.

Juan_David_Jaramillo · April 15, 2021, 4:30pm

I have already done the steps you ask me and I got the "scroll_id", then what do I do to download the index? The documentation does not explain how to do it, thank you.

POST /up-time*/_search?scroll=1m
{
  "size": 100,
  "query":  {
    "match": {
      "message": "foo"
    }
  }
}

and this is the result

{
  "_scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFld4UExiOFlhVHo2RHpwd0VRLV9TNmcAAAAAAAADORZ6SUlmc3FiM1JMNmlDMS0xcVpEeFlR",
  "took" : 0,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

then it asks me to do this

POST /_search/scroll
{
  "scroll": "1m",
  "scroll_id": "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFld4UExiOFlhVHo2RHpwd0VRLV9TNmcAAAAAAAAJUBZ6SUlmc3FiM1JMNmlDMS0xcVpEeFlR"
}

and I get this result as it says in the documentation

{
  "_scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDXF1ZXJ5QW5kRmV0Y2gBFld4UExiOFlhVHo2RHpwd0VRLV9TNmcAAAAAAAAOphZ6SUlmc3FiM1JMNmlDMS0xcVpEeFlR",
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 0,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [ ]
  }
}

finally it asks me to obtain the results, by obtaining the "scroll_id".

GET /_search?scroll=1m
{
  "sort": [
    "_doc"
  ]
}

So I get this result of all the "scroll_id" in the index

#! this request accesses system indices: [.apm-agent-configuration, .apm-custom-link, .async-search, .kibana_1, .kibana_7.12.0_001, .kibana_task_manager_1, .kibana_task_manager_7.12.0_001, .ml-config, .security-7, .tasks, .triggered_watches, .watches], but in a future major version, direct access to system indices will be prevented by default
{
  "_scroll_id" : "FGluY2x1ZGVfY29udGV4dF91dWlkDnF1ZXJ5VGhlbkZldGNoKRZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAgWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAkWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAcWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEA8WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEA0WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAoWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAsWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEAwWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBAWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEA4WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBEWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBIWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBMWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBQWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBUWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBgWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBYWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBcWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBkWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEC4WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBoWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBsWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEB8WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEBwWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEB0WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEB4WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECAWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAED4WeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECEWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECIWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECMWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECQWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECcWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECUWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECYWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECoWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECgWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECkWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECsWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAECwWeklJZnNxYjNSTDZpQzEtMXFaRHhZURZXeFBMYjhZYVR6NkR6cHdFUS1fUzZnAAAAAAAAEC0WeklJZnNxYjNSTDZpQzEtMXFaRHhZUQ==",
  "took" : 664,
  "timed_out" : false,
  "_shards" : {
    "total" : 41,
    "successful" : 41,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 415280,
      "relation" : "eq"
    },
    "max_score" : null,
    "hits" : [
      {
        "_index" : ".async-search",
        "_type" : "_doc",
        "_id" : "lIy7gwquQJGPY_95V_L2pQ",
        "_score" : null,
        "_source" : {
          "result" : "48myAwFERm14SmVUZG5kM0YxVVVwSFVGbGZPVFZXWDB3eWNGRWJla2xKWm5OeFlqTlNURFpwUXpFdE1YRmFSSGhaVVRvek9ERXkAAQEAAH/AAAAAAAAAAQEOZGF0ZV9oaXN0b2dyYW0KdGltZXNlcmllcwoFCXRpbWVGaWVsZAAJdGltZXN0YW1wBWluZGV4ABpraWJhbmFfc2FtcGxlX2RhdGFfZmxpZ2h0cw5pbnRlcnZhbFN0cmluZwACMWgKYnVja2V0U2l6ZQEAAA4QCHNlcmllc0lkACQ2MWNhNTdmMS00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTcEAAEGDkFtZXJpY2EvQm9nb3RhAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAEBAAABeLJtIoABAAABeLeTfoAAAAAAAAAAAAAACWRhdGVfdGltZSdzdHJpY3RfZGF0ZV9vcHRpb25hbF90aW1lfHxlcG9jaF9taWxsaXMOQW1lcmljYS9Cb2dvdGEAAAAZAAABeLJtIoAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXiypBEAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4str/gAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLMR7gAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXizSNyAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4s3/LAAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLO2uYAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXiz7agAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4tCSWgAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLRbhQAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi0knOAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4tMliAAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLUAUIAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi1Nz8AAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4tW4tgAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLWlHAAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi13AqAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4thL5AAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLZJ54AAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi2gNYAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4trfEgAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLbuswAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAXi3JaGAAAIGZmlsdGVyLjYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1udW1lcmF0b3L/AAAGZmlsdGVyMDYxY2E1N2YyLTQ2OWQtMTFlNy1hZjAyLTY5ZTQ3MGFmNzQxNy1kZW5vbWluYXRvcgoAAAAAAAF4t1yQAAACBmZpbHRlci42MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctbnVtZXJhdG9y/wAABmZpbHRlcjA2MWNhNTdmMi00NjlkLTExZTctYWYwMi02OWU0NzBhZjc0MTctZGVub21pbmF0b3IKAAAAAAABeLeTfoAAAgZmaWx0ZXIuNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LW51bWVyYXRvcv8AAAZmaWx0ZXIwNjFjYTU3ZjItNDY5ZC0xMWU3LWFmMDItNjllNDcwYWY3NDE3LWRlbm9taW5hdG9yCgAAAAAAAgABAQEAAAAAAHcAAAAAAAABeLeXlDUAAAF426QYNQ==",
          "headers" : {
            "_xpack_security_authentication" : "48myAwAHZWxhc3RpYwEJc3VwZXJ1c2VyCgEJX3Jlc2VydmVkBQEAAAEAFGVsa3NlcnZlci1WaXJ0dWFsQm94CHJlc2VydmVkCHJlc2VydmVkAAAKAA=="
          },
          "expiration_time" : 1618592667701,
          "response_headers" : { }
        },
        "sort" : [
          0
        ]
      },
      {
        "_index" : ".kibana-event-log-7.11.1-000001",
        "_type" : "_doc",
        "_id" : "lUk_y3cBlIpB_Q5C4UEL",
        "_score" : null,
        "_source" : {
          "@timestamp" : "2021-02-22T19:38:15.035Z",
          "event" : {
            "provider" : "eventLog",
            "action" : "starting"
          },
          "message" : "eventLog starting",
          "ecs" : {
            "version" : "1.6.0"
          },
          "kibana" : {
            "server_uuid" : "14ded7a1-87e4-4ba2-882b-45f9c7a05ba8"
          }
        },
        "sort" : [
          0
        ]
      },
      {
        "_index" : ".kibana-event-log-7.11.1-000002",
        "_type" : "_doc",
        "_id" : "zCNgangBpucHuqdGVm9r",
        "_score" : null,
        "_source" : {
          "@timestamp" : "2021-03-25T17:13:21.756Z",
          "event" : {
            "provider" : "eventLog",
            "action" : "stopping"
          },
          "message" : "eventLog stopping",
          "ecs" : {
            "version" : "1.6.0"
          },
          "kibana" : {
            "server_uuid" : "14ded7a1-87e4-4ba2-882b-45f9c7a05ba8"
          }
        },
        "sort" : [
          0
        ]
      },

dadoonet · April 15, 2021, 5:14pm

I believe that the problem is that

POST /up-time*/_search?scroll=1m
{
  "size": 100,
  "query":  {
    "match": {
      "message": "foo"
    }
  }
}

Gives absolutely no hits.
So there is nothing to scroll here as the resultset is empty.

If you want to get the full index, run instead:

POST /up-time*/_search?scroll=1m
{
  "size": 100
}

Then get the 100 hits, do whatever you want with that.
Get also the scroll_id and use it for the next calls.

POST /_search/scroll                                                               
{
  "scroll" : "1m",                                                                 
  "scroll_id" : "DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMNjU1QQ==" 
}

Until the resultset is empty again.

Sandeep_Raju · April 15, 2021, 7:50pm

If your index is small, you can just go to discover,select your index.
Then save your search with name.
Then share -> download Report as CSV-> management->report-> Download report.

Juan_David_Jaramillo · April 15, 2021, 8:26pm

Es un indice con informacion de 3 meses o mas , por lo tanto no se puede descargar de esa forma.

Sandeep_Raju · April 15, 2021, 8:34pm

One other way is by using Elasticdump Tool to download an index data.

Juan_David_Jaramillo · April 15, 2021, 8:37pm

Bajo el paquete en este sitio? elasticdump - npm

Sandeep_Raju · April 15, 2021, 9:49pm

Yes you can use the above mentioned website.
I used the instructions from below website:

There is also a github link:

Juan_David_Jaramillo · April 16, 2021, 6:46pm

I am also trying with this one so that it compresses the data from the elastic server to my documents and I get the same error and it does not let me download the data, which I can do now and tried in several ways, also the cluster is active

elasticdump \ --input=http://localhost:9200/up-time* \ --output=$ \ | gzip > /home/elk-server/up-time*.json.gz
Fri, 16 Apr 2021 18:45:44 GMT | Error Emitted => {"errors":["`input` is a required input","`output` is a required input"]}

Juan_David_Jaramillo · April 16, 2021, 8:29pm

it ready worked for me in the following way

npm install elasticdump -g

sudo apt-get update
sudo apt-get upgrade
shutdown -r 0

elasticdump \
> --input=http://localhost:9200/up-time*/ \
> --output=up-time.json \
> --type=data
Fri, 16 Apr 2021 20:18:33 GMT | starting dump
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 0)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 100)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 200)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 300)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 400)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:33 GMT | got 100 objects from source elasticsearch (offset: 500)
Fri, 16 Apr 2021 20:18:33 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 600)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 700)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 800)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 900)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:38 GMT | got 100 objects from source elasticsearch (offset: 1000)
Fri, 16 Apr 2021 20:18:38 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1100)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1200)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1300)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1400)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:43 GMT | got 100 objects from source elasticsearch (offset: 1500)
Fri, 16 Apr 2021 20:18:43 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 1600)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 1700)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 1800)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 1900)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:48 GMT | got 100 objects from source elasticsearch (offset: 2000)
Fri, 16 Apr 2021 20:18:48 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2100)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2200)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2300)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2400)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:53 GMT | got 100 objects from source elasticsearch (offset: 2500)
Fri, 16 Apr 2021 20:18:53 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:58 GMT | got 100 objects from source elasticsearch (offset: 2600)
Fri, 16 Apr 2021 20:18:58 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:58 GMT | got 100 objects from source elasticsearch (offset: 2700)
Fri, 16 Apr 2021 20:18:58 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:58 GMT | got 100 objects from source elasticsearch (offset: 2800)
Fri, 16 Apr 2021 20:18:58 GMT | sent 100 objects to destination file, wrote 100
Fri, 16 Apr 2021 20:18:58 GMT | got 43 objects from source elasticsearch (offset: 2900)
Fri, 16 Apr 2021 20:18:58 GMT | sent 43 objects to destination file, wrote 43
Fri, 16 Apr 2021 20:18:58 GMT | got 0 objects from source elasticsearch (offset: 2943)
Fri, 16 Apr 2021 20:18:58 GMT | Total Writes: 2943
Fri, 16 Apr 2021 20:18:58 GMT | dump complete
root@elkserver-VirtualBox:~# ls
go  up-time.json

system · May 14, 2021, 8:29pm

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Get all documents from an index Elasticsearch	10	107522	June 21, 2017
Get All Data Through Elasticsearch Elasticsearch	2	360	February 28, 2020
How to retrieve all records from index Elasticsearch	5	3392	December 13, 2018
How to get all results from Elastic Search Index using Powershell and save them locally Elasticsearch	4	546	July 14, 2020
Pulling more than 10000 records from elasticsearch query Elasticsearch	2	31345	June 11, 2019

How to download data from an elasticsearch index

Related topics