Indexing simple json at filebeats 7.2

Elastic Stack Beats
1

I'd like to store some JSON log file to Elasticsearch and index log easily.

filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/sample/*.log
  json.keys_under_root: true
  json.add_error_key: true

setup.template.name: "sample"
setup.template.fields: "fields.yml"
setup.template.pattern: "sample-*"
setup.template.overwrite: "true"

output.elasticsearch:
  hosts: ["localhost:9200"]
  enabled: true
  index: "sample"

example log

{"time":"2019-07-02T16:52:29.62+0900","remote_addr":"10.0.12.5","uri":"/list_account"}

- key: sample
  title: "sample"
  fields:
    - name: "time"
      type: "date"
    - name: remote_addr
      type: "ip"
    - name: uri
      type: "keyword"

I expect that Iog is stored with index, but there are some problems

(1) Index is not created as expected name

filebeat export template
{
  "index_patterns": [
    "filebeat-7.2.0-*"
  ],
...

(2) log is parsed and stored as _source in the default index(filebeat-7.2.0-*)

How should I do to store the above simple JSON with index?

2

Welcome! Sorry, could you clarify -- are you trying to save your logs to sample and it's getting put in filebeat-* instead, or are you not seeing the right logs at all? It sounds like the former but I wanted to make sure. As for your second question, saving the original input in _source is usually the right thing to do, is your problem that you want to remove _source or that the fields are not otherwise being indexed?

3

Thank you for your reply.

are you trying to save your logs to sample and it's getting put in filebeat-* instead, or are you not seeing the right logs at all?

I'd like to store logs to sample.
Logs seems to be stored into default index filebeat-*

saving the original input in _source is usually the right thing to do, is your problem that you want to remove _source or that the fields are not otherwise being indexed?

maybe 'the fields are not otherwise being indexed'
I'd like to fields get indexed, but I can't understand whether my output index setting is wrong or fields.yml is wrong.

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.