Sorting @timestamp while exporting from elasticsearch and writing into csv

Hi,
Problem Statement: To export the logs from elasticsearch and write into csv in timewise order(either increasing or decreasing)

Solution: I am using logstash , configuration is given below,

input {
 elasticsearch {
    hosts => "localhost:9200"
    index => "mergnginxinfo"
	schedule => "*/1 * * * *"
    query => '
    {
	    "query": {
			"bool": {
				"must": [
					{
					  "match_all": {}
					},
					{
					  "exists": {
						"field": "transaction.id.keyword"
					  }
					} ,
						{
						"range":{
						"@timestamp":
						{
							"gte": "now-4m", 
							"lte": "now-3m"
						}
						}
						}
						
				]
			
	  
	}
	
  },
	  "sort": [
				{
					"@timestamp": {
					 "order": "asc"
					}			
				}
  ]}'	
 } 
 
}

output {
  csv {
    # elastic field name
    fields =>  ["@timestamp","requestid","ngnix.cpu.usage","ngnix.responsebytes","ngnix.memory.total.free","ngnix.memory.total.test","ngnix.transaction.duration","ngnix.urlpath","ngnix.Error_ID","ngnix.Exception","ngnix.ipaddress","ngnix.useragent","app1.user.id","app1.firstname","app1.lastname","app1.cpu.usage","app1.memory.total.free","app1.memory.total.test","app1.responsecode","app1.user.email","app1.transaction.duration.us","app1.methodname","app1.http.Port","app1.error.id","app1.error.exception.type","app1.transaction.result","app1.uripath","app2.user.id","app2.firstname","app2.lastname","app2.cpu.usage","app2.memory.total.free","app2.memory.total.test","app2.methodname","app2.responsecode","app2.error.id","app2.error.exception.type","app2.user.email","app2.http.Port","app2.transaction.duration.us","app2.transaction.result","app2.uripath"]
	
    # This is path where we store output.   
    path => "C:/Users/M1056317/ELK/csv/try2/csv-export-%{+YYYY-MM-dd_hh.mm}.txt"
		
  }
 
}

Problem: I am not getting the timestamp in particular order, I checked the same with GET API, it is coming fine there but not in csv file. Seems like there is some issue while writing.

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.