Beats (Metricbeat) unable to change settings via commandline

Elastic Stack Beats
1

Hi,

We need to dynamically change the value of a field. Following this pattern does not work:

in beats 5.0 one can also overwrite any setting from command line via -E flag. For example:

./countbeat -c countbeat.yml --configtest -E name=otherhost

Get an error pertaining to being unable to find template file. To get round this I added -path-config /etc/metricbeat but now it just hangs (a basic config test returns OK though). I also need to change the working directory to the metricbeats bin directory. Any suggestions?

2

Can you share your config file? Any maybe add some logs.

Keyword "template": Beats will by default looks for <beatname>.template.json to load the elasticsearch index template. See template settings. You can configure an alternative path or disable template loading.

3 
    ###################### Metricbeat Configuration Example #######################
    # This file is an example configuration file highlighting only the most common
    # options. The metricbeat.full.yml file from the same directory contains all the
    # supported options with more comments. You can use it as a reference.
    #
    # You can find the full configuration reference here:
    # https://www.elastic.co/guide/en/beats/metricbeat/index.html

    #==========================  Modules configuration ============================
    metricbeat.modules:

    #------------------------------- System Module -------------------------------
    - module: system
      metricsets:
        # CPU stats
        - cpu

        # System Load stats
        - load

        # Per CPU core stats
        #- core

        # IO stats
        #- diskio

        # Per filesystem stats
        - filesystem

        # File system summary stats
        - fsstat

        # Memory stats
        - memory

        # Network stats
        - network

        # Per process stats
        - process
      enabled: true
      period: 10s
      processes: ['.*']



    #================================ General =====================================

    # The name of the shipper that publishes the network data. It can be used to group
    # all the transactions sent by a single shipper in the web interface.
    #name: gs_guid

    # The tags of the shipper are included in their own field with each
    # transaction published.
    tags: ["GameServer"]

    # Optional fields that you can specify to add additional information to the
    # output.
    fields_under_root: true
    fields:
      gs_guid: "guid_tbc"

    #================================ Outputs =====================================

    # Configure what outputs to use when sending the data collected by the beat.
    # Multiple outputs may be used.

    #-------------------------- Elasticsearch output ------------------------------
    output.elasticsearch:
      # Array of hosts to connect to.
      hosts: ["myserver.com:9200"]

      # Optional protocol and basic auth credentials.
      #protocol: "https"
      #username: "elastic"
      #password: "changeme"

    #----------------------------- Logstash output --------------------------------
    #output.logstash:
      # The Logstash hosts
      #hosts: ["localhost:5044"]

      # Optional SSL. By default is off.
      # List of root certificates for HTTPS server verifications
      #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

      # Certificate for SSL client authentication
      #ssl.certificate: "/etc/pki/client/cert.pem"

      # Client Certificate Key
      #ssl.key: "/etc/pki/client/cert.key"

    #================================ Logging =====================================

    # Sets log level. The default log level is info.
    # Available log levels are: critical, error, warning, info, debug
    #logging.level: debug

    # At debug level, you can selectively enable logging only for some components.
    # To enable all selectors use ["*"]. Examples of other selectors are "beat",
    # "publish", "service".
    #logging.selectors: ["*"]
4

Hi, here is the config file. There is indeed a template file in etc/metricbeat called metricbeat.template.json

5

so you got it working? Can you share debug logs (run with -e -v -d '*').

6

The debug logs show that it is working: I changed the name to a different string. Couple of things.

I subsequently checked the default configuration file /etc/metricbeat/metricbeat.yml and the configuration file has not been changed to reflect my changes above.

Is this process designed to persist to the configuration file or is is just a temporary change whilst the command is running?

Also how can you change a field name (or another nested key/value)?

7

The -E ... CLI option only overwrites settings within the current running process. No changes on CLI will be written to the configuration file. Instead of -E you can also pass multiple configuration files via -c.

For more information see Config File Format docs. You can overwrite any setting using it's complete name. e.g. -E output.elasticsearch.hosts=otherhost:9200

8

This topic was automatically closed after 21 days. New replies are no longer allowed.