# Pipeline - Create a delta field

**URL:** <https://discuss.elastic.co/t/pipeline-create-a-delta-field/309618>\
**Category:** Logstash\
**Created:** [July 14, 2022, 8:29am UTC](https://discuss.elastic.co/t/pipeline-create-a-delta-field/309618 "2022-07-14T08:29:50Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ely\_96](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/ely_96/32/103816_2.png) [@Ely\_96](https://discuss.elastic.co/u/Ely_96)\
**Post date:** [July 14, 2022, 8:29am UTC](https://discuss.elastic.co/t/pipeline-create-a-delta-field/309618/1 "2022-07-14T08:29:50Z")

</div>

Hi Guys,

I'm trying to study how to create a delta field: I mean a field that should contains a difference between 2 fields inside an existing index.  
So, I have 2 examples of CSVs:

```auto
ID;Month;Date;Date_string;Name;Surname;Score
1;202112;2021-12-01;DEC-2021;Mario;Rossi;15
2;202112;2021-12-01;DEC-2021;Francesca;Bianchi;12

```

The second CSV updates my data:

```auto
ID;Month;Date;Date_string;Name;Surname;Score
3;202202;2022-02-01;FEB-2022;Mario;Rossi;18
4;202202;2022-02-01;FEB-2022;Francesca;Bianchi;10

```

As you can see, the score of Mario Bianchi changes (from 15 to 18) and also the score of Francesca Bianchi (from 12 to 10).

The pipeline that ingest the data:

```auto
input {
  file {
    path => "C:/elastic_d/logstash/bin/data/lookup/data_2.txt"
    start_position => "beginning"
    sincedb_path => "NULL"
    type => "csv"
  }
}

filter {
  csv {
    separator => ";"
    skip_header => "true"
    columns => ["ID","Month","Date","Date_string","Name","Surname","Score"]
  }

  mutate {convert => {"Score" => "integer"}}

}

output {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "lookup"
  }
}

```

Then I created a new pipeline in order to create a new index that should contains a new field "Stage\_1" with the difference of score (the variation of them):

```auto
input {
  elasticsearch {
      hosts => ["localhost:9200"]
      index => "lookup"
  }
}

filter {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "lookup"
    query => "Name:%{Name} AND Surname:%{Surname} AND ID:>%{ID}"
    add_tag => "event_benchmark"
  }

  if "event_benchmark" not in [tags] {
    mutate {add_field => { "stage_1" => 0 }}
    mutate {convert => {"stage_1" => "integer"}}
  }
}

output {
    elasticsearch {
    hosts => ["localhost:9200"]
    index => "lookup_clean"
  }
}

```

But I really don't know how to:

1. Get the last event (date desc order) with "event\_benchmark" tag
2. Calculate the delta, as difference between Score (without tag "event\_benchmark" so the last one) and the last event (in date desc order) with the tag "event\_becnhmark". For each key Name + Surname.
3. Insert the calculated value inside a field called "Stage\_1"

Could you please help me? Any suggestions... pls 🙂

Thanks in advance  
Ely

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 11, 2022, 8:30am UTC](https://discuss.elastic.co/t/pipeline-create-a-delta-field/309618/2 "2022-08-11T08:30:35Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.