Can't creat a Template

Elastic Stack Beats
1

Hi,
After two days trying basic stuff y can' t do that Filebeat and Kibana read my data log and split in my JSON fields to use the Kibana visualization tools, using the 5.6.4 version (last x32 bits version)

Our logs are in JSON with the following fields:
{"@timestamp":"2017-10-03T23:50:15.000Z","EMP":"BAL","INST":"PRI","TIPO":"T2","MSG":"FPS STATS COV : E=0/M=946/A=0 ","MOD":"AbdFPS","FNT":"fpsCbTimerVaciado.c","LIN":"83","ID1":0,"ID2":946,"ID3":0,"STR1":"","STR2":"","STR3":""}

So, in Filebeat folder I've created a file prueba.template.json like the original one. I've edited it with the information of my fields. I think this file is wrong, but I think file isn't loaded in Kibana.
prueba.template.json - https://pastebin.com/uXxrQD9i

In filebeat.yml I've tried to load the template file
filebeat.yml - https://pastebin.com/jQb9R30d -

Running Filebeat, ElasticSearch and Kibana and I've detected that log files are loaded but not the template.
Arranco Filebeat, ElasticSearch y Kibana y detecto que si que se cargan los archivos de Log pero no el template. Let's go to Kibana and load the index pattern "filebeat-*" and

1
1.png946×358 32 KB
2
2.png1090×664 50.5 KB

There are apache fields that I've deleted on prueba.template.json and my fields are not shown. If I go to discover can see that the content in message (the interesting one) is not processed.

3
3.png1164×412 26.9 KB

Help a newbie :frowning:

2

There are two things: index template which is for Elasticsearch and the index pattern which is for Kibana. The index template that you linked above is for elasticsearch and does not have a direct affect on Kibana. The config options you are looking for under 5.6 are under dashboard.*.

Having said all the above, I'm curious what you are trying to do. If you have json logs, you need to tell filebeat to decode the json. So I think your issue is more in the filebeat config then with the template and index pattern. Can you share your filebeat config and how you start filebeat?

3

Hi, thanks for you response.

Yes, we have JSON logs and we want to tell filebeat to decode the JSON.
We can't install filebeat service with PowerShell because we have not permissions to install in the corporate machine that filebeat is running (long history...) . Does that have something to do with our problem?

The config
filebeat.yml --> https://pastebin.com/EpuUAyc8
filebeat.template.json ---> https://pastebin.com/FyGzeUJV

To open filebeat
Command Line in Filebeat folder and >filebeat -c filebeat.yml -e
Anything else?

4

Can you use the file output and read in some log files and share the output? I wonder if the decoding worked as expected.

Please also share the filebeat log output.

Three things from the config:

  • Indentation on setup.kibana is wrong
  • Template config should be under setup but seems to be under output now?
  • You still have the default path (/var/log/*) in your config. Is that on purpose?

It seems you copied together the config but missed some important parts.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.