Sending Data from Metricbeat to Logstash not working

Elastic Stack Logstash
1

Hello All,
In current implementation the data is going well from metricbeat to elastic but now while changing
it from metricbeat->logstash->elastic it dosent works,data is not received by logstash.Can someone help where I am going wrong.I have defined ILM policy and rollover and with this I'm giving data to logstash.

metricbeat.yml

metricbeat.config.modules:
  path: ${path.config}/modules.d/*.yml
  reload.enabled: false
setup.ilm.enabled: true
setup.ilm.check_exists: true
setup.ilm.rollover_alias: mis-metric
setup.ilm.pattern: '{now/d}-000001'
setup.ilm.overwrite: false
setup.kibana:
  host: http://abc06appl002.dev.ppdm01.group.vw:5601
output.logstash:
   hosts: ["abc06appl002.dev.ppdm01.group.vw:5044"]
processors:
- add_fields:
    fields:
        configfile: ${DM_SHARED}/MIS/globalconfiguration.properties
- rename:
    fields:
    - from: agent.hostname
      to: host.name
    ignore_missing: true
    fail_on_error: false
- drop_fields:
    fields:
 #  - host.mac
 #  - system.process.cmdline
    - ecs
    - agent.ephemeral_id
    - agent.type
    - agent.version
    - agent.hostname
    - event
monitoring.enabled: true
monitoring.elasticsearch.hosts: true
#logging.level: debug

type or paste code here

logstash config file:

input {
  beats {
    port => 5044
  }
}


output {
  
    elasticsearch {
      hosts => "http://abc06appl002.dev.ppdm01.group.vw:9200"
      
   }  
  
}

I'm confused if alreaddy ILM policy is defined should i use in metricbeat.yml or in logstash
output plugin filter...Can anyone help in this.

Many Thanks,
PM

2

ILM configuration in metricbeat applies to data sent directly from metricbeat to Elasticsearch. If the data goes through logstash then those setting have no effect and ILM must be configured in the logstash output.

3

@Badger Thanks for your response ! So just need to define the ILM policy in output filter
of Elasticsearch for metricbeat and would other processor and filter would work fine
in metricbeat.yml?..will try this.
what would be the right way to this,I want the data to come in "mis-metric" index with
rollover and policy defined.
If I get u right ,I just need to define this in output filter of logstash?..other config seems fine?
Thanx again!

4

I do not run either metricbeat or Elasticsearch so I cannot answer your other questions.

5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.