Filebeat to elasticsearch using ingest pipeline

Elastic Stack Elasticsearch
1

Dear Team,

need a advise we are ingesting data from filebeat to ES using the ingest pipeline in ES
I have created an ingest pipeline and also created an index template in that defined the ingest pipeline

"PUT _ingest/pipeline/beat_test
{
"description": "THN Test",
"processors": [{
"json": {
"field": "message",
"add_to_root": false,
"target_field": "fields",
"ignore_failure" : true
}
}
]
}

PUT _component_template/logs-my_app-settings
{
"template": {
"settings": {
"index.default_pipeline": "beat_test"
}
}
}

PUT _index_template/logs-my_app-template
{
"index_patterns": ["logs_json"],
"composed_of": ["logs-my_app-settings"]
}"

and the output of my filebeat is

'
output.Elasticsearch:
hosts: ["https://ESIP:PORT"]
username: "test_data"
password: "*****"
pipeline: "beat_test"
'

Still not getting data in the ES defined index

a help would be highly appreicated

Regards

2

@Badger any idea for this

3

Please don't ping people that aren't already taking part in your topic.

Please also format your code/logs/config using the </> button, or markdown style back ticks. It helps to make things easy to read which helps us help you

What does this mean? Where does the data go?

4

Hi,

thankyou for the response
my data is going in the default filebeat index this is the main concern

even after definig the index.

though figured out a different way
i have set the _index field to the index name.

Warm Regards

5

Hi,

did you solve it ?

6

Yes, I solved it.
I created an ingestpipeline config

7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.