# Duration Time Calculation: missing started time

**URL:** <https://discuss.elastic.co/t/duration-time-calculation-missing-started-time/188999>\
**Category:** Logstash\
**Created:** [July 5, 2019, 4:05am UTC](https://discuss.elastic.co/t/duration-time-calculation-missing-started-time/188999 "2019-07-05T04:05:17Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![yanly011](https://avatars.discourse-cdn.com/v4/letter/y/a698b9/32.png) [@yanly011](https://discuss.elastic.co/u/yanly011)\
**Post date:** [July 5, 2019, 4:05am UTC](https://discuss.elastic.co/t/duration-time-calculation-missing-started-time/188999/1 "2019-07-05T04:05:17Z")

</div>

Hi, I just created a pipeline and transfer .csv log file to index. I want to find the started time and calculate the duration time. I used the code from official document:  
filter {  
csv {  
separator =\> ","  
columns =\> ["Timestamp", "SeverityName", "EventName", "EntityName", "Application", "Operation", "TransactionId", "SessionId", "OperationId", "PartyName", "ClientId", "Host", "LogId"]  
}

```
	mutate {convert => ["SeverityName", "string"]}
	mutate {convert => ["EventName", "string"]}
	mutate {convert => ["EntityName", "string"]}
	mutate {convert => ["Application", "string"]}
	mutate {convert => ["Operation", "string"]}
	mutate {convert => ["TransactionId", "string"]}
	mutate {convert => ["SessionId", "string"]}
	mutate {convert => ["OperationId", "string"]}
	mutate {convert => ["PartyName", "string"]}
	mutate {convert => ["ClientId", "string"]}
	mutate {convert => ["Host", "string"]}
	mutate {convert => ["LogId", "integer"]}
		
	date {
		  match => ["Timestamp", "YYYY-MM-dd HH:mm:ss.SSS"]
		  target => "@timestamp"
		  remove_field => ["Timestamp"]
	}
		
	if [EventName] == "Sending" and [EntityName] == "Response" {
	    
		elasticsearch {
		    index => "prodlogssmall_10"
			query => "EventName:Received AND EntityName:Request AND SessionId:%{[SessionId]}"
			fields => { "@timestamp" => "started" }
		}
		 
		date {
			  match => ["[started]", "YYYY-MM-dd HH:mm:ss.SSS"]
			  target => "[started]"
		}			 	
    
	    ruby {
			  code => "
			        event.set('duration', (event.get('@timestamp') - event.get('started')) * 1000) 				
					"
		}	
	}		

```

When I run logstash, the started field can be created successfully and I can see it in ES. But duration filed will be failed to creat because it seems there is no filed of 'started' when it is calculated. The error message is:  
[2019-07-05T13:32:21,186][ERROR][logstash.filters.ruby] Ruby exception occurred: can't convert nil into an exact number

I am confused that my code is almost same as the code from official document:  
[https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html](https://www.elastic.co/guide/en/logstash/current/plugins-filters-elasticsearch.html)

Could you please help me? Many thanks.

---

<div class="post-metadata">

**Author:** ![yanly011](https://avatars.discourse-cdn.com/v4/letter/y/a698b9/32.png) [@yanly011](https://discuss.elastic.co/u/yanly011)\
**Post date:** [July 5, 2019, 5:40am UTC](https://discuss.elastic.co/t/duration-time-calculation-missing-started-time/188999/2 "2019-07-05T05:40:08Z")

</div>

I found the problem is that I have to set batch size as 1. Hence, the elasticsearch filter plugin can find the started time of a session in ES. However, it is very slow if the size cannot be set larger. Is there any other solution to calculate duration time?

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 2, 2019, 5:40am UTC](https://discuss.elastic.co/t/duration-time-calculation-missing-started-time/188999/3 "2019-08-02T05:40:14Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.