Use Heartbeat to read process status from a file

[abhishek13 (Abhishek Sharma)]

I have a process that writes it status in a file, single line stating "RUNNING/STOPPED".
I can do a curl file:///mylocation/status.txt and in output i get RUNNING if process is running and stopped otherwise.
Is there any way i can have heartbeat to curl this file and check the response body for "RUNNING or STOPPED " value.
What i have tried and failed.

- type: http # monitor type `http`. Connect via HTTP an optionally verify response

  # Monitor name used for job name and document type
  name: Server167

  # Configure task schedule
  schedule: '@every 30s' # every 5 seconds from start of beat

  # Configure URLs to ping
  urls: ["file:///root/status.txt"]

  check.request:
  # Configure HTTP method to use. Only 'HEAD', 'GET' and 'POST' methods are allowed.
    method: "GET"

  # Expected response settings
  check.response:
  # Required response contents.
    body: "RUNNING"

[martinr_ubi (Martin R.)]

No, heartbeat can't fetch things from a file, only over the network, tcp (http/s), udp, icmp, etc.

You need to find another way, put a web server in front of that file (serve it) or ingest that file with filebeat either continuously or through --once maybe.
There is more than one way to ingest single line files that are contiously rewritten to by an app but it's not directly supported in filebeat, you currently have to resort to trickey/hackery/systemd units or-and timers/cron jobs etc.
If the line in the file doesn't end with some kind of new_line character more trickey is involved because filebeat doesn't ingest a line unless it ends with a new_line character.

The path of least resistance depends on your skillset with the different tools and methods that can be employed to ingest such a file at regular interval even though it's not really a log file and thus not currently directly supported by filebeat. For some it will be easier to install any simple web server that will serve the file on localhost for heartbeat or metricbeat to fetch it. Metricbeat can only ingest json so in your case that's not a good fix. But heartbeat could fetch it if it was made available on localhost:xxx/status.txt.

Filebeat can ingest stdin so in some use cases you make it do that and then you feed stuff (your file) into the stdin of filebeat at some regular interval. Just make an inventory of all the ways you can somehow stuff that content in ES and then choose the best for your context while keeping as simple as possible and using stuff you know or can test.

You can also open a github request to describe your use case and document what a feature to solve it would look like. Filebeat could have a type of input where the harvested file is always read from the beginning, a kind of harvester for continuously rewritten files or written-to once files (like jobs outputs or reports).