# Field munging

**URL:** <https://discuss.elastic.co/t/field-munging/63128>\
**Category:** Logstash\
**Created:** [October 16, 2016, 1:03pm UTC](https://discuss.elastic.co/t/field-munging/63128 "2016-10-16T13:03:45Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [October 16, 2016, 1:03pm UTC](https://discuss.elastic.co/t/field-munging/63128/1 "2016-10-16T13:03:45Z")

</div>

So here's what I got. I am monitoring two files via logstash, each has a field "service". Now, I have the below combinations of results in Kibana, the dashes show no result:

```
ntp,-
-,ntp
ntp,ntp_google
-,-

```

I'd like to do....something, but I'm not sure what. In plain English I'd really like "if you have any results, just show that one result" which would address the first two. For the second, "if you have a result with an underscore, just show that one". Lastly, "if you have two dashes, rename that to 'Unknown'". I don't even know if it's possible, and truth be told if it's a boatton of work in logstash then meh I'll just deal with it as it is. Thank you.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 16, 2016, 3:38pm UTC](https://discuss.elastic.co/t/field-munging/63128/2 "2016-10-16T15:38:24Z")

</div>

> the below combinations of results in Kibana

It's very hard to understand what you mean. What do the documents contain? What query are you using in Kibana? Are you seeing this in the Discover tab? A screenshot might help.

---

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [October 16, 2016, 4:56pm UTC](https://discuss.elastic.co/t/field-munging/63128/3 "2016-10-16T16:56:09Z")

</div>

Yea I guess I could have made this clearer. Ok so here's the logstash bits:

```
input {

	file {
		type => "connlog"
		path => "/opt/bro/spool/bro/conn.log"
		sincedb_path => "/var/lib/logstash/.sincedbconn"
	}

```

This has a grok match (snips added for clarity):

`match => ["message", "(<snip>\t(?<service>(.*?)) <snip> \t(?<service>(.*))"]`

I've named them both service since a) they ARE services (ssl for the first and for example ssl\_facebook on the second). This is how Kibana displays them:

 ![](https://us1.discourse-cdn.com/elastic/original/2X/a/ae5cde2f24147583a8916ba02e5370b95ed6de00.png)

So again...I can live with this..the service has at least been identified with one or the other field. If it hasn't been, then I know it's something unknown and I can dig deeper into it. My question is can I clean this up so where I only have either a) a single entry, the service with a "\_" if it exists, b) remove the "-," or ",-" if they exist, and lastly c) change the "-,-" to "Unknown". Thanks...sorry I wasn't very clear here.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 16, 2016, 5:14pm UTC](https://discuss.elastic.co/t/field-munging/63128/4 "2016-10-16T17:14:41Z")

</div>

- So the `service` field should really never be an array then? Since you want to remove all hyphen entries and in the last example want to keep "ssl\_google" it seems `service` should always be a string.
- Please show some example log entries. The simplest solution might (depending on the answer to the previous bullet) be to adjust the grok expression.

---

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [October 16, 2016, 8:45pm UTC](https://discuss.elastic.co/t/field-munging/63128/5 "2016-10-16T20:45:20Z")

</div>

Thanks for looking at this Magnus. Here's a sample:

```
1476643999.554721	CCds0e2aAntQPGzZn2	192.168.1.2	13062	192.168.1.253	53	udp	dns	0.045913	37	213	SF	T	T	0	Dd	1	65	1	241	(empty)	dns_google

```

I'm basically cheating here....the last "service" entry is actually called "protosigs", but I called it service so they'd end up in the same place. The goal and intent is to be able to histogram by date using the service Term, since currently we can only use one Term in Kibana (besides TimeLion which continues to mystify me). Case in point:  
 ![](https://us1.discourse-cdn.com/elastic/original/2X/6/6c01f7d9115f8b2ebe6335b255b45320ff0eb2ef.png)

Thanks again Magnus...I tried using `join => { "service" => "," }`, but I think that only applies to arrays.

---

<div class="post-metadata">

**Author:** ![magnusbaeck](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/magnusbaeck/32/44943_2.png) [@magnusbaeck](https://discuss.elastic.co/u/magnusbaeck)\
**Post date:** [October 18, 2016, 5:50am UTC](https://discuss.elastic.co/t/field-munging/63128/6 "2016-10-18T05:50:40Z")

</div>

Maybe capture the last field into a separate temporary field that you overwrite the `service` value with if set?

---

<div class="post-metadata">

**Author:** ![DigiAngel](https://avatars.discourse-cdn.com/v4/letter/d/977dab/32.png) [@DigiAngel](https://discuss.elastic.co/u/DigiAngel)\
**Post date:** [October 18, 2016, 10:58am UTC](https://discuss.elastic.co/t/field-munging/63128/7 "2016-10-18T10:58:58Z")

</div>

That's a neat idea...not sure how'd I would do that in logstash...I'll do some research..thank you Magnus!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [July 6, 2017, 4:33am UTC](https://discuss.elastic.co/t/field-munging/63128/8 "2017-07-06T04:33:43Z")

</div>

Closed on July 6, 2017, 4:33am UTC


