Split message field into multiple fields in kibana

Elastic Stack Kibana
1

I've the following data for the message field which is being shipped by filebeat to elasticseatch. I am not using Logstash here

2020-09-20 15:44:23 ::1 get / - 80 - ::1 mozilla/5.0+(windows+nt+10.0;+win64;+x64)+windows/537.36+(khtml,+like+gecko)+chrome/85.0.4183.102+chrome/537.36 - 200 0 0 10

I want to split the above data at every space and assign them to different fields and the new fields should get reflect in the kibana discovery portal.

How can we do that?

I've tried to use scripted field in kibana, but I am unable to achieve it as I am not aware of scripted field querying.

2

Welcome to our community! :smiley:

The best path is to do the processing before it is indexed. Those logs look like HTTP access logs, what is the source of them?

3

@warkolm These are IIS logs getting shipped by Filebeat. The last field '10' is time-taken is somehow the not getting shipped. So I decided to split and assign the last value to a new field so that it gets showed in Kibana discovery portal.

4

Are you using the IIS module? https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-module-iis.html

5

@warkolm Yes, I am using IIS module but it is not shipping time-taken field.

FYI: Below are my filebeat and IIS module config files

filebeat.yml

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - c:\inetpub\logs\Logfiles\*\*

filebeat.config.modules:
  path: C:\Program Files\filebeat\modules.d\*.yml
  reload.enabled: true
  reload.period: 10s

setup.template.settings:
  index.number_of_shards: 1
setup.kibana:
  host: "<Kibana_IP>:5601"

output.elasticsearch:
  hosts: ["<Elasticsearch Master IP>:9200"]

processors:
  - add_host_metadata: ~
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

iis.yml

- module: iis
 access:
   enabled: true
   #var.paths:

 error:
   enabled: true
   #var.paths:

I've also specified the logs path in iis.yml config file at var.paths but it is also not exporting the time-taken field

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.