# Avoid duplicates code in Logstash
**URL:** https://discuss.elastic.co/t/avoid-duplicates-code-in-logstash/358695
**Category:** Logstash
**Created:** [May 3, 2024, 9:57am UTC](https://discuss.elastic.co/t/avoid-duplicates-code-in-logstash/358695 "2024-05-03T09:57:31Z")
**Posts on this page:** 5
**Page:** 1
### Author:  [@Leo23](https://discuss.elastic.co/u/Leo23)
#### Post date: [May 3, 2024, 9:57am UTC](https://discuss.elastic.co/t/avoid-duplicates-code-in-logstash/358695/1 "2024-05-03T09:57:31Z")
I have a problem I want to measure the execution time between each step of a log. So I use Elapsed Filter but I can have hundred step in a logs how can I delete duplicate code like this :
```auto
elapsed {
unique_id_field => "id"
start_tag => "Step 1"
end_tag => "Step 2"
new_event_on_match => true
add_tag => "1->2"
}
elapsed {
unique_id_field => "id"
start_tag => "Step 2"
end_tag => "Step 3"
new_event_on_match => true
add_tag => "2->3"
}
...
elapsed {
unique_id_field => "id"
start_tag => "Step 99"
end_tag => "Step 100"
new_event_on_match => true
add_tag => "99->100"
}
```
Do there is a way to generate this code with variable or loop ?
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 3, 2024, 12:05pm UTC](https://discuss.elastic.co/t/avoid-duplicates-code-in-logstash/358695/2 "2024-05-03T12:05:16Z")
The only way I can think of to do this would be to use a ruby filter and re-purpose much of the [code](https://github.com/logstash-plugins/logstash-filter-elapsed/blob/main/lib/logstash/filters/elapsed.rb) from the elapsed filter. Instead of just storing a single timestamp for the id you would have to rewrite it to store a hash of step start/end times.
---
### Author:  [@Leo23](https://discuss.elastic.co/u/Leo23)
#### Post date: [May 3, 2024, 12:20pm UTC](https://discuss.elastic.co/t/avoid-duplicates-code-in-logstash/358695/3 "2024-05-03T12:20:42Z")
This seems difficult, especially since I'm not familiar with Ruby. The first thing to do is change start\_tag end end\_tag to an array with [Step 1, Step 2 ...] for start\_tag end [Step 2, Step 3...] for the end\_tag ?
```auto
config :start_tag, :validate => :string, :required => true
# The name of the tag identifying the "end event"
config :end_tag, :validate => :string, :required => true
```
---
### Author:  [@Leo23](https://discuss.elastic.co/u/Leo23)
#### Post date: [May 3, 2024, 12:27pm UTC](https://discuss.elastic.co/t/avoid-duplicates-code-in-logstash/358695/4 "2024-05-03T12:27:36Z")
I will change to that and after I don't know what can I do
```auto
config :start_tags, :validate => :array, :required => true
config :end_tags, :validate => :array, :required => true
```
---
### Author:  [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [May 4, 2024, 4:41pm UTC](https://discuss.elastic.co/t/avoid-duplicates-code-in-logstash/358695/5 "2024-05-04T16:41:14Z")
I had some time to look at this and think you can do it with an aggregate filter. Note that pipeline.ordered must evaluate to true and pipeline.workers must be set to 1.
Consider the input file
```auto
{ "id": 1, "tags": ["step_1", "pest_1", "foo"], "@timestamp": "2024-05-04T11:40:38.132Z" }
{ "id": 2, "tags": ["step_1"], "@timestamp": "2024-05-04T11:40:38.132Z" }
{ "id": 2, "tags": ["step_2"], "@timestamp": "2024-05-04T11:40:38.444Z" }
{ "id": 1, "tags": ["step_2"], "@timestamp": "2024-05-04T11:40:38.555Z" }
{ "id": 1, "tags": ["step_3"], "@timestamp": "2024-05-04T11:40:42.232Z" }
{ "id": 2, "tags": ["step_4"], "@timestamp": "2024-05-04T11:40:39.333Z" }
{ "id": 2, "tags": ["step_5"], "@timestamp": "2024-05-04T11:40:44.123Z" }
```
That can be processed using
```
aggregate {
task_id => "%{id}"
code => '
begin
step = event.get("tags").select { |tag| tag =~ /^step_\d+/ }
stepNo = step[0].sub(/^step_/, "")
map["steps"] ||= {}
map["steps"][stepNo.to_s] = event.get("@timestamp")
rescue StandardError => e
puts e.to_s
end
'
push_map_as_event_on_timeout => true
timeout_task_id_field => "user_id"
timeout => 10
timeout_code => '
begin
steps = event.get("steps")
steps = steps.sort_by { |k, v| k}
lastValue ||= []
steps.each { |k,v|
if lastValue != [] # Skip first item
event.set("step_#{k}_#{lastValue[0]}", v - lastValue[1])
end
lastValue = [k, v]
}
rescue StandardError => e
puts e.to_s
end
'
}
```
to produce (in addition to the source events) two events like this
```
{
"@timestamp" => 2024-05-04T16:36:49.678604952Z,
"steps" => {
"2" => 2024-05-04T11:40:38.555Z,
"3" => 2024-05-04T11:40:42.232Z,
"1" => 2024-05-04T11:40:38.132Z
},
"@version" => "1",
"step_3_2" => 3.677,
"user_id" => "1",
"step_2_1" => 0.423
}
{
"@timestamp" => 2024-05-04T16:36:49.679253516Z,
"steps" => {
"2" => 2024-05-04T11:40:38.444Z,
"1" => 2024-05-04T11:40:38.132Z,
"4" => 2024-05-04T11:40:39.333Z,
"5" => 2024-05-04T11:40:44.123Z
},
"@version" => "1",
"step_4_2" => 0.889,
"step_5_4" => 4.79,
"user_id" => "2",
"step_2_1" => 0.312
}
```
Obviously this can be adjusted in many, many ways.