Filebeat Index

[Vog93 (Namad)]

Hello, I have installed filebeat and uploaded a CSV to obtain the filebeat configuration for the installation, creating a new index and pipeline. the yml file looks like this:

filebeat.inputs:
- type: log
  paths:
  - C:\Users\victo\Downloads\archive\*.csv
  exclude_lines: ['^"?General_Health"?,"?Checkup"?,"?Exercise"?,"?Heart_Disease"?,"?Skin_Cancer"?,"?Other_Cancer"?,"?Depression"?,"?Diabetes"?,"?Arthritis"?,"?Sex"?,"?Age_Category"?,"?Height_\(cm\)"?,"?Weight_\(kg\)"?,"?BMI"?,"?Smoking_History"?,"?Alcohol_Consumption"?,"?Fruit_Consumption"?,"?Green_Vegetables_Consumption"?,"?FriedPotato_Consumption"?']

cloud.id: "MyID:****"
cloud.auth: "elastic:****"
 
index: "prueba3"
pipeline: "prueba3-pipeline"

setup:
  template.enabled: false
  ilm.enabled: false

After that I go to powershell and execute ./filebeat -c filebeat.yml setup when finished ./filebeat -e -c filebeat.yml

I can see that the data is going to the filebeat-* index but not to the created one "prueba3"

I want to have the data in my index prueba3, what is the problem what am i doing wrong?

[Vog93 (Namad)]

Here is the filebeat-* index
and the prueba3 index that has the data that i uploaded manually, I think that the number of hits of the prueba3 should increase because I have 2 files on the directory

This is just a test, because I have to upload a bunch of files with the same structure to elastic and that is why i am using filebeat and obtaining the structure of the fiiles by just uploading one and copying the yml.

[Vog93 (Namad)]

I see that the data is in the message part how can I have it on my index?

[stephenb (Stephen Brown)]

Think you need this to be

@Vog93

output.elasticsearch:
  index: "prueba3"
  pipeline: "prueba3-pipeline"

What version as what you need to do is bit different in 7.x and 8.x.

Did you set up a template for the mappings? Otherwise you are going to get a default mapping which is not what you want... let me know the version and I think there are some post here with the correct steps / syntax

[Vog93 (Namad)]

That solved it thank you!!