# Trying to understand json parsing of Winlogbeats logs

**URL:** <https://discuss.elastic.co/t/trying-to-understand-json-parsing-of-winlogbeats-logs/219821>\
**Category:** Logstash\
**Created:** [February 18, 2020, 3:21pm UTC](https://discuss.elastic.co/t/trying-to-understand-json-parsing-of-winlogbeats-logs/219821 "2020-02-18T15:21:44Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [February 18, 2020, 3:21pm UTC](https://discuss.elastic.co/t/trying-to-understand-json-parsing-of-winlogbeats-logs/219821/1 "2020-02-18T15:21:44Z")

</div>

Hello there !

I've read lots of doc and forum topics but can't succeed to find a solution

Winlogbeats are pushed to Kafka, then Logstash consume these logs

I have the following (valid) json pulled on kafka by Logstash :

> {"@timestamp":"2020-02-18T15:14:02.352Z","@metadata":{"beat":"winlogbeat","type":"\_doc","version":"7.5.2"},"agent":{"ephemeral\_id":"363b6102-b019-482b-a868-df0455cb707b","hostname":"WIN7SPLUNKPC","id":"89c9d4d6-5c59-4db1-9cdc-5cdfc0521a02","version":"7.5.2","type":"winlogbeat"},"ecs":{"version":"1.1.0"},"message":"Le service Parental Controls est entré dans l’état : en cours d’exécution.","winlog":{"record\_id":13383,"task":"","computer\_name":"WIN7SPLUNKPC","event\_id":7036,"event\_data":{"param2":"en cours d’exécution","Binary":"5700500043005300760063002F0034000000","param1":"Parental Controls"},"provider\_guid":"{555908d1-a6d7-4695-8e1e-26931d2012f4}","channel":"System","provider\_name":"Service Control Manager","api":"wineventlog","process":{"thread":{"id":3388},"pid":436},"keywords":["Classique"]},"event":{"kind":"event","code":7036,"provider":"Service Control Manager","created":"2020-02-18T15:14:04.338Z"},"log":{"level":"information"},"host":{"hostname":"WIN7SPLUNKPC","architecture":"x86\_64","os":{"build":"7601.24544","platform":"windows","version":"6.1","family":"windows","name":"Windows 7 Professional","kernel":"6.1.7601.24545 (win7sp1\_ldr\_escrow.200102-1707)"},"id":"a931f168-a8ce-4411-a3a3-6077d2830b48","name":"WIN7SPLUNKPC"}}

Here is the **logstash conf** :

```
input {
  kafka {
    bootstrap_servers => "192.168.250.102:9092"
    topics => "winlogbeat"
    codec => "json"
    type => kafka
    }
}

output {
  elasticsearch {
      ilm_enabled => true
      hosts => ["http://coord01:9200","http://coord02:9200"]
      index => "ims_linux_logs"
  }
  file {
   path => "/usr/share/logstash/logs/logstash.log"
   codec => rubydebug
}
}

```

I have the following error in logstash logs :

> logstash01 | [WARN] 2020-02-18 15:14:06.419 [[winlogbeat\_kafka]\>worker1] elasticsearch - Could not index event to Elasticsearch. {:status=\>400, :action=\>["index", {:\_id=\>nil, :\_index=\>"ims\_linux\_logs", :\_type=\>"\_doc", :routing=\>nil}, #LogStash::Event:0x4795a2e1], :response=\>{"index"=\>{"\_index"=\>"ims\_linux\_logs-000001", "\_type"=\>"\_doc", "\_id"=\>"givdWHABmYntWF\_FC0jK", "status"=\>400, "error"=\>{"type"=\>"mapper\_parsing\_exception", "reason"=\>"failed to parse field [host] of type [text] in document with id 'givdWHABmYntWF\_FC0jK'. Preview of field's value: '{hostname=WIN7SPLUNKPC, os={build=7601.24544, kernel=6.1.7601.24545 (win7sp1\_ldr\_escrow.200102-1707), name=Windows 7 Professional, family=windows, version=6.1, platform=windows}, name=WIN7SPLUNKPC, id=a931f168-a8ce-4411-a3a3-6077d2830b48, architecture=x86\_64}'", "caused\_by"=\>{"type"=\>"illegal\_state\_exception", "reason"=\>"Can't get text on a START\_OBJECT at 1:24"}}}}}

Ruby debug is like this :

```
{
      "@version" => "1",
          "host" => {
                  "id" => "a931f168-a8ce-4411-a3a3-6077d2830b48",
                "name" => "WIN7SPLUNKPC",
        "architecture" => "x86_64",
                  "os" => {
            "platform" => "windows",
              "family" => "windows",
             "version" => "6.1",
              "kernel" => "6.1.7601.24545 (win7sp1_ldr_escrow.200102-1707)",
               "build" => "7601.24544",
                "name" => "Windows 7 Professional"
        },
            "hostname" => "WIN7SPLUNKPC"
    },
          "type" => "kafka",
         "event" => {
            "kind" => "event",
         "created" => "2020-02-18T15:14:04.338Z",
        "provider" => "Service Control Manager",
            "code" => 7036
    },
           "ecs" => {
        "version" => "1.1.0"
    },
        "winlog" => {
             "event_id" => 7036,
        "computer_name" => "WIN7SPLUNKPC",
              "channel" => "System",
        "provider_name" => "Service Control Manager",
                  "api" => "wineventlog",
             "keywords" => [
            [0] "Classique"
        ],
        "provider_guid" => "{555908d1-a6d7-4695-8e1e-26931d2012f4}",
            "record_id" => 13383,
           "event_data" => {
            "param2" => "en cours d’exécution",
            "Binary" => "5700500043005300760063002F0034000000",
            "param1" => "Parental Controls"
        },
              "process" => {
               "pid" => 436,
            "thread" => {
                "id" => 3388
            }
        },
                 "task" => ""
    },
    "@timestamp" => 2020-02-18T15:14:02.352Z,
         "agent" => {
                  "id" => "89c9d4d6-5c59-4db1-9cdc-5cdfc0521a02",
             "version" => "7.5.2",
                "type" => "winlogbeat",
        "ephemeral_id" => "363b6102-b019-482b-a868-df0455cb707b",
            "hostname" => "WIN7SPLUNKPC"
    },
       "message" => "Le service Parental Controls est entré dans l’état : en cours d’exécution.",
           "log" => {
        "level" => "information"
    }
}

```

Thanks for your help ! 🙂

---

<div class="post-metadata">

**Author:** ![Badger](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/badger/32/25190_2.png) [@Badger](https://discuss.elastic.co/u/Badger)\
**Post date:** [February 18, 2020, 3:59pm UTC](https://discuss.elastic.co/t/trying-to-understand-json-parsing-of-winlogbeats-logs/219821/2 "2020-02-18T15:59:36Z")

</div>

> [@Travis](#):
>
> failed to parse field [host] of type [text]

elasticsearch expects the [host] field to be text, but in your event it is an object with fields nested inside it.

---

<div class="post-metadata">

**Author:** ![Travis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/travis/32/54079_2.png) [@Travis](https://discuss.elastic.co/u/Travis)\
**Post date:** [February 18, 2020, 4:24pm UTC](https://discuss.elastic.co/t/trying-to-understand-json-parsing-of-winlogbeats-logs/219821/3 "2020-02-18T16:24:29Z")

</div>

Hey @Badger !

Indeed. And I understood why : I didn't applied the good template (I suppose the default one was used - or maybe no one ? ) ...

As I tested before to push Winlogbeats to Logstash directly, I remembered have pushed the template manually (like described [here](https://www.elastic.co/guide/en/beats/winlogbeat/current/winlogbeat-template.html#load-template-manually)).

So, I adapted this template to feet my needs (index settings, ilm policy..) and magic : no more issue 😊

Thanks !

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [March 17, 2020, 4:24pm UTC](https://discuss.elastic.co/t/trying-to-understand-json-parsing-of-winlogbeats-logs/219821/4 "2020-03-17T16:24:33Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.