Need some help and hopefully its something simple I have overlooked. I am trying to import a JSON file, single line records and when I go into Kibana, to view the data in discover, I dont see any of the fields if I use a custom index. However, if I use the default (filebeat) format, the fields are displayed. however, if I go into visualizations and create a date table, I see the fields. Here is my filebeat yml file. I only included my changes I made. Everything else is default or commented out.
-
type: log
Change to true to enable this input configuration.
enabled: true
json.keys_under_root: true
json.add_error_key: true
setup.template.settings:
index.number_of_shards: 1
#index.codec: best_compression
#_source.enabled: true
setup.ilm.enabled: false
output.elasticsearch:
Array of hosts to connect to.
enabled: true
hosts: ["servername.domain:9200"]
index: "indexname-%{+yyyy.MM.dd}"
setup.template.name: "indexname"
setup.template.pattern: "indexname-*"
processors:
#- add_host_metadata: ~
#- add_cloud_metadata: ~
- timestamp:
field: EventDate
layouts:
- '2006-01-02T15:04:05'
- '2006-01-02T15:04:05.999'
- drop_fields:
fields: EventDate