# Add\_field and copy data

**URL:** <https://discuss.elastic.co/t/add-field-and-copy-data/109773>\
**Category:** Logstash\
**Created:** [November 30, 2017, 2:36pm UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773 "2017-11-30T14:36:11Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![mathurin68](https://avatars.discourse-cdn.com/v4/letter/m/6bbea6/32.png) [@mathurin68](https://discuss.elastic.co/u/mathurin68)\
**Post date:** [November 30, 2017, 2:36pm UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/1 "2017-11-30T14:36:11Z")

</div>

Alright, I think I just about have what I'm looking for ...

Any activity that comes over event\_data.CommandLine the doesn't have " " quotations around it. I want to rename the field and copy the data from the field into the new one.

This almost seems to work, it creates the new field but it doesn't copy the info over...

filter {  
if "(.\*?)" not in [event\_data.CommandLine] {  
mutate {  
add\_field =\> {"event\_data.Suspicious" =\> "Suspicious Activity"}  
copy =\> {"event\_data.CommandLine" =\> "event\_data.Suspicious" }  
}  
}  
}

This does create the field, event\_data.Suspicious but it doesn't copy the data over.

As always I appreciate the help!!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 30, 2017, 3:11pm UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/2 "2017-11-30T15:11:30Z")

</div>

copy =\> {"[event\_data][CommandLine]" =\> "event\_data.Suspicious" }

is what you want. Assuming you want a dot in the field name.

---

<div class="post-metadata">

**Author:** ![mathurin68](https://avatars.discourse-cdn.com/v4/letter/m/6bbea6/32.png) [@mathurin68](https://discuss.elastic.co/u/mathurin68)\
**Post date:** [November 30, 2017, 5:04pm UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/3 "2017-11-30T17:04:46Z")

</div>

Thanks Badger! Why do I have to go 'down the tree' on one and not the other?

Why not this -  
copy =\> {"[event\_data][CommandLine]" =\> "[event\_data].[Suspicious]" }

Appreciate it, this looks promising!

---

<div class="post-metadata">

**Author:** ![mathurin68](https://avatars.discourse-cdn.com/v4/letter/m/6bbea6/32.png) [@mathurin68](https://discuss.elastic.co/u/mathurin68)\
**Post date:** [November 30, 2017, 5:09pm UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/4 "2017-11-30T17:09:26Z")

</div>

Also, and this is just a pet peeve thing but the add field puts a , at the end of it.

add\_field =\> {"event\_data.Suspicious" =\> ""}

...a little thing but I was curious.

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [November 30, 2017, 5:29pm UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/5 "2017-11-30T17:29:39Z")

</div>

If you input

```
{ "Foo": 1, "Bar":"2" }
```

into logstash with this config

```
input { stdin { } }
output { stdout { codec => rubydebug } }

filter {
  json {
    source => "message"
    target => "event_data"
  }
  mutate {
  add_field => {"event_data.Suspicious" => "Suspicious Activity"}
  }
  mutate {
  copy => {"[event_data][Foo]" => "event_data.withdot" }
  }
  mutate {
  copy => {"[event_data][Foo]" => "[event_data][withbracket]" }
  }
  mutate {
  }
}
```

You will see the difference.

```
{
               "@timestamp" => 2017-11-30T17:25:46.727Z,
                 "@version" => "1",
                     "host" => "[...]",
               "event_data" => {
        "withbracket" => 1,
                "Bar" => "2",
                "Foo" => 1
    },
                  "message" => "{ \"Foo\": 1, \"Bar\":\"2\" }",
    "event_data.Suspicious" => "Suspicious Activity",
       "event_data.withdot" => 1
}
```

---

<div class="post-metadata">

**Author:** ![mathurin68](https://avatars.discourse-cdn.com/v4/letter/m/6bbea6/32.png) [@mathurin68](https://discuss.elastic.co/u/mathurin68)\
**Post date:** [December 1, 2017, 3:10pm UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/6 "2017-12-01T15:10:43Z")

</div>

Alright! I think I'm staring to get this, if I can trouble you for one last suggestion, as a final step I need to convert the field to keyword(end goal being to use this in machine learning)

From other discussions on here I put this together but can't get it to work and tried all kinds of combos -  
filter {  
if [event\_id] == 4688 and [event\_data][CommandLine] !~ /"(.\*?)"/ {  
mutate {  
add\_field =\> {"event\_data.Suspicious" =\> ""}  
}  
mutate {  
copy =\> {"[event\_data][CommandLine]" =\> "event\_data.Suspicious" }  
}  
mutate {  
convert =\> ["[event\_data][Suspicious]", "keyword"]  
}  
}  
}

... keep getting the dreaded pipeline terminated error.

I appreciate it, especially the explanations!

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [December 1, 2017, 5:28pm UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/7 "2017-12-01T17:28:03Z")

</div>

Whether something is a keyword is determined by the template field mapping in Elasticsearch, it is not a logstash thing. If you are using the default logstash-\* template pretty much every string field has a .keyword field added to it.

You should ask a new question over in the Elasticsearch forum.

---

<div class="post-metadata">

**Author:** ![mathurin68](https://avatars.discourse-cdn.com/v4/letter/m/6bbea6/32.png) [@mathurin68](https://discuss.elastic.co/u/mathurin68)\
**Post date:** [December 2, 2017, 5:22am UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/8 "2017-12-02T05:22:34Z")

</div>

Thanks dude, I found it the dev tools, it was already a keyword! Thanks so much for your help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [December 30, 2017, 5:22am UTC](https://discuss.elastic.co/t/add-field-and-copy-data/109773/9 "2017-12-30T05:22:54Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.