I'm a new elasticsearch user. What I'd like to do is capture a csv file of system metrics. I want to use this data in parallel with captured application data to analyze and then optimize system configuration and make application tweaks.
I'm trying to use metricbeat. To get that running I installed and setup elasticsearch, kibana, and metricbeat. Initially the metricbeat output was going to elasticsearch and I could use the kibana canned system dashboards.
I changed output to file, and it is writing json output. Since output is now to file instead of elasticsearch, I thought I would not have to run elasticsearch and kibana. But if I do not, metricbeat aborts with an error that it cannot connect to kibana. Must I run eleacticsearch and kibana, or am I doing something wrong? The output json file has more fields than I want. I tried editing the fields.yml file, but that didn't seem to work. Is the better approach to output to logstash and use mutate to remove fields? I'm also new to logstash. I'm assuming I can write json out of logstash. Once I have the json I was planning to use jq to convert it to cvs.
I'm also open to easier approaches if there is one.
Thanks.