# Logstash KV filter time field

**URL:** https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183
**Category:** Logstash
**Tags:** elastic-stack-monitoring, elastic-stack-security
**Created:** [July 20, 2021, 3:44pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183 "2021-07-20T15:44:21Z")
**Posts on this page:** 7
**Page:** 1

<div class="post-metadata">

### Author: ![RaonyO](https://avatars.discourse-cdn.com/v4/letter/r/5fc32e/32.png) [@RaonyO](https://discuss.elastic.co/u/RaonyO)
#### Post date: [July 20, 2021, 3:44pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183/1 "2021-07-20T15:44:22Z")

</div>

hello guys, i'm using kv filter to filter syslogs in logstash to be sent to elasticsearch. but the kv is creating a field with time and data instead of being in value, how to fix this?  
is reaching the field limit because one is being created every time

![image](https://us1.discourse-cdn.com/elastic/original/3X/5/a/5a21b63c8eea89b1ae808d0eda3a96161b46648c.png)  
 ![image](https://us1.discourse-cdn.com/elastic/original/3X/9/b/9b1b98f398abbcdbebfeeccef6576fe51f439e8a.png)

here is my settings:

```auto
filter {
      kv {
        field_split => "||"
        value_split => "="
        allow_duplicate_values => false
        source => "message"
      }

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 20, 2021, 4:51pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183/2 "2021-07-20T16:51:46Z")

</div>

If your data is not key-value pairs (and it is obviously not) then use grok or dissect to extract the key-value data from the message. See [this](https://discuss.elastic.co/t/parsing-firewall-logs-in-logstash/212786/2) example.

---

<div class="post-metadata">

### Author: ![RaonyO](https://avatars.discourse-cdn.com/v4/letter/r/5fc32e/32.png) [@RaonyO](https://discuss.elastic.co/u/RaonyO)
#### Post date: [July 20, 2021, 6:01pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183/3 "2021-07-20T18:01:15Z")

</div>

hello @Badger , thanks for the help, I saw the link to the example you sent but I couldn't implement it, can you help me with the filter?  
anyway, thanks!

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 20, 2021, 6:15pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183/4 "2021-07-20T18:15:39Z")

</div>

You have not given any indication of what your data looks like, so I cannot possibly suggest a pattern to match it.

---

<div class="post-metadata">

### Author: ![RaonyO](https://avatars.discourse-cdn.com/v4/letter/r/5fc32e/32.png) [@RaonyO](https://discuss.elastic.co/u/RaonyO)
#### Post date: [July 20, 2021, 6:54pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183/5 "2021-07-20T18:54:04Z")

</div>

sorry, i just forgot, here it is:

```auto
"<13>Jul 20 15:27:30 10.241.20.6 time=1626801326|hostname=MGMT-XX|severity=Informational|confidence_level=Medium|product=IPS|action=Detect|ifdir=inbound|ifname=lo|loguid={9x10f3e5yy,0x1c,0x6475cf72,0xeec2e5bc}|origin=xx.xxx.131.3|originsicname=CN\\=FW01-Out,O\\=MGJ.br.atootc|sequencenum=42|time=1626801326|version=5|attack=SSL Enforcement Violation|attack_info=OpenSSL ChaCha20_Poly1305 Cipher Suites|description_url=CVE_2016_7054_help.html|dst=xxx.xx.35.20|https_inspection_action=Inspect|industry_reference=CVE-2016-7054|lastupdatetime=1626805676|log_id=2|malware_rule_id={13B84A4D-2280-4C37-A24E-6FD1377AE144}|performance_impact=3|policy=FRA-POLICY|policy_time=1625860876|protection_id=asm_dynamic_prop_CVE_2016_7054|protection_name=OpenSSL ChaCha20_Poly1305 Cipher Suites|protection_type=IPS|proto=6|received_bytes=3000|rule_name=Webmail|rule_uid=584d05b2-3722-4729-XXXX-XXXX1f7a7b4|s_port=591XX|sent_bytes=1947|service=XXX|session_id={0x60f6e642e,0x3c,0x6475cf72,0xeec2e5bc}|smartdefense_profile=XXXX-IDS|src=xxx.xxx.249.124|suppressed_logs=30|layer_name=IPS|layer_name=F-POLICY Threat Prevention|layer_name=IPS|layer_name=FRA-POLICY Threat Prevention|layer_name=IPS|layer_name=FRA-POLICY Threat Prevention|layer_name=IPS|layer_name=FRA-POLICY Threat Prevention|layer_name=IPS|layer_name=FRA-POLICY Threat Prevention|layer_uuid={364B9452-D032-4D02-8358-XXXXXXX}

```

and here is my output conf:

```auto
output {
  elasticsearch { hosts => ["https://XX.XX.XX.160:9200"]
  codec => cef
  user => "xxxxxx"
  password => "XXXXXXXXX"
  ssl_certificate_verification => false
  index => "syslog" }
  stdout { codec => rubydebug }

```

---

<div class="post-metadata">

### Author: ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)
#### Post date: [July 20, 2021, 7:21pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183/6 "2021-07-20T19:21:12Z")

</div>

Please do not start multiple threads for the same request.

---

<div class="post-metadata">

### Author: ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)
#### Post date: [August 17, 2021, 7:22pm UTC](https://discuss.elastic.co/t/logstash-kv-filter-time-field/279183/7 "2021-08-17T19:22:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
