# Increment counter if document already exists

**URL:** <https://discuss.elastic.co/t/increment-counter-if-document-already-exists/138604>\
**Category:** Logstash\
**Created:** [July 4, 2018, 7:01pm UTC](https://discuss.elastic.co/t/increment-counter-if-document-already-exists/138604 "2018-07-04T19:01:39Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![yodog](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yodog/32/4822_2.png) [@yodog](https://discuss.elastic.co/u/yodog)\
**Post date:** [July 4, 2018, 7:01pm UTC](https://discuss.elastic.co/t/increment-counter-if-document-already-exists/138604/1 "2018-07-04T19:01:40Z")

</div>

lets say i have an index `iptables-%{+YYYY.MM}`, which holds iptables kernel generated logs.

i would like to:

1- create the doc if the log line doesn't exists  
or  
2- increment a field called `counter` if it already exists

every doc would be saved for the first time with a counter, incrementing on every insert of the same key.

```
input {
    beats {
        add_field => { "counter" => 1 }
        port => "5044"
    }
}

```

the `document_id` would be based on some fields from `message`;  
this would allow me to always have only one of each message type.

```
filter {
    fingerprint {
        add_tag => ["alreadyseen"]
        concatenate_sources => true
        key => "alreadyseen"
        source => ["SRC", "SPT", "DST", "DPT"]
        target => "[@metadata][fingerprint]"
    }
}

output {
    elasticsearch {
        action => "update"
        doc_as_upsert => true
        document_id => "%{[@metadata][fingerprint]}"
        index => "iptables-%{+YYYY.MM}"
        sniffing => true
        template_overwrite => true
    }
}

```

but i have no idea on how to increment my counter.  
any help?

---

<div class="post-metadata">

**Author:** ![yodog](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yodog/32/4822_2.png) [@yodog](https://discuss.elastic.co/u/yodog)\
**Post date:** [July 5, 2018, 7:05pm UTC](https://discuss.elastic.co/t/increment-counter-if-document-already-exists/138604/2 "2018-07-05T19:05:54Z")

</div>

so, after reading [https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-update.html#\_scripted\_updates](https://www.elastic.co/guide/en/elasticsearch/reference/6.2/docs-update.html#_scripted_updates) i tried the following

```
elasticsearch {
    action => "update"
    doc_as_upsert => true
    document_id => "%{[@metadata][fingerprint]}"
    manage_template => false
    script => "ctx._source.counter++"
}

```

but the `counter` field always concatenate the number `1` instead of adding

```
on 1st execution counter: 1
on 2nd execution counter: 11
on 3rd execution counter: 111

```

and so on

i tried all lines below

```
script => "ctx._source.counter++"
script => "ctx._source.counter += 1"
script => "ctx._source.counter = ctx._source.counter + 1"
script => "ctx._source.counter = ctx._source.counter++"
```

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [July 5, 2018, 8:08pm UTC](https://discuss.elastic.co/t/increment-counter-if-document-already-exists/138604/3 "2018-07-05T20:08:27Z")

</div>

```
add_field => { "counter" => 1 }

```

Unless you have an index template counter will be a string in elasticsearch. You could mutate it to be an integer (and create a new index, since the old index already has a mapping).

```
mutate { convert => { "counter" => "integer" } }

```

However, you will then hit another problem: a null pointer exception when you do the initial insert 🙂

```
script => "if (ctx._source.counter != null) {ctx._source.counter++}"
```

---

<div class="post-metadata">

**Author:** ![yodog](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yodog/32/4822_2.png) [@yodog](https://discuss.elastic.co/u/yodog)\
**Post date:** [July 9, 2018, 2:38pm UTC](https://discuss.elastic.co/t/increment-counter-if-document-already-exists/138604/4 "2018-07-09T14:38:17Z")

</div>

the strange thing is that i actually do have a index template `mapping`

```
"properties": {
  "@timestamp": {
    "type": "date"
  },
  "counter": {
    "type": "long"
  },

```

and kibana mapped it as `number`, so it seems right

anyway, got it to work with

`script => "if (ctx._source['counter'] == null) { ctx._source['counter'] = 1 } else { ctx._source.counter++ }"`

and removing `add_field` from the input

---

<div class="post-metadata">

**Author:** ![yodog](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yodog/32/4822_2.png) [@yodog](https://discuss.elastic.co/u/yodog)\
**Post date:** [July 11, 2018, 2:34pm UTC](https://discuss.elastic.co/t/increment-counter-if-document-already-exists/138604/5 "2018-07-11T14:34:59Z")

</div>

also,

> [@How to assign a new field a certain type and value?](https://discuss.elastic.co/t/how-to-assign-a-new-field-a-certain-type-and-value/34148/2):
>
> Perhaps surprisingly, mutate { add\_field =\> { somefield =\> 1 } } doesn't actually add an integer field: $ cat test.config input { stdin { } } output { stdout { codec =\> rubydebug } } filter { mutate { add\_field =\> { "somefield" =\> 1 } } } $ echo 'foo' | /opt/logstash/bin/logstash -f test.config Logstash startup completed { "message" =\> "foo", "@version" =\> "1", "@timestamp" =\> "2015-11-09T19:45:06.364Z", "host" =\> "hallonet", "somefield" =\> "1" } Logstash…

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [August 8, 2018, 2:35pm UTC](https://discuss.elastic.co/t/increment-counter-if-document-already-exists/138604/6 "2018-08-08T14:35:09Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.