Search Exports under Kibana (10 million documents)

Elastic Stack Kibana
1

Hi there,
in an elk stack with basic licence for now (version 8.17.x), I would like to export about 10 million syslog documents out of Kibana. Is there a better way then 'export as a csv file'? Or how can I config the environment to make this task easier?
Thanks for any input!!

2

You can use Logstash I think for this.
Otherwise, there are some community tools like

HTH

3

Thanks! I also found this approach: Just using good old logstash.

input {
elasticsearch {
hosts => ["http://elasticsearch:9200"]
index => "syslog" # Replace with your index name
query => '{"query": {"match_all": {}}}'
size => 10000
scroll => "5m"
}
}

output {
csv {
path => "/usr/share/logstash/output/syslog_export.csv"
fields => ["@timestamp", "host", "message", "severity"] # Adjust to your syslog fields
}
}

1 Like