# Is it possible to format Logstash output in a way that it shows in Uptime?

**URL:** <https://discuss.elastic.co/t/is-it-possible-to-format-logstash-output-in-a-way-that-it-shows-in-uptime/314883>\
**Category:** Logstash\
**Created:** [September 21, 2022, 3:47pm UTC](https://discuss.elastic.co/t/is-it-possible-to-format-logstash-output-in-a-way-that-it-shows-in-uptime/314883 "2022-09-21T15:47:45Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![SamuelSMendes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samuelsmendes/32/104246_2.png) [@SamuelSMendes](https://discuss.elastic.co/u/SamuelSMendes)\
**Post date:** [September 21, 2022, 3:47pm UTC](https://discuss.elastic.co/t/is-it-possible-to-format-logstash-output-in-a-way-that-it-shows-in-uptime/314883/1 "2022-09-21T15:47:45Z")

</div>

I was thinking if, by using mutate in the output of my pipeline on logstash in a way that "mimics" the heartbeat output, I could have an uptime application to see the return of my pseudo-heartbeat (generated by the logstash on the act of changing the content of the output).

Once placed at Uptime settings UI where you can choose the index which the monitors receives data, that index would have the same specifications of the heartbeat generated output (like monitor.status, url, id, name and etc). This way, would I be able to see my "made up" monitor in Uptime?

The context that leads this question is:

I have to monitor an array of objects that returns a lot of devices, each one with an unique ID, name and a "connected" status. By changing the lines in mutate I could easily make up so that the unique ID stands up by a monitor ID, the name by it's name, the connected could easily go into an "up" or "down" status and so on.

Of course, nothing stops me from using the Dashboard or Canvas since I already have the info at hand, but I was wondering if by making the necessary changes I would be able to have it at Uptime, or if there is anything essential to Uptime that only responds to heartbeat/synthetics.

Not a everyday case but would be glad if anyone could help me set this straight,

best regards!

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 21, 2022, 4:36pm UTC](https://discuss.elastic.co/t/is-it-possible-to-format-logstash-output-in-a-way-that-it-shows-in-uptime/314883/2 "2022-09-21T16:36:57Z")

</div>

You probably can do that.

You just need to make your output looks like it is coming from heartbeat and save it in a custom index that the Uptime app will read.

I'm not sure how this works in version 8.X, but in version 7 you could add more indices in the Uptime app.

 ![Screenshot from 2022-09-21 13-35-44](https://us1.discourse-cdn.com/elastic/original/3X/e/1/e167b502b8691c354677fdd74354ca7dfbdf6409.png)

So, you just need to do what you are already planning, create a custom index with the same naming convention for the fields and use the same mapping as the uptime indices use and it should populate the uptime app in kibana.

---

<div class="post-metadata">

**Author:** ![SamuelSMendes](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samuelsmendes/32/104246_2.png) [@SamuelSMendes](https://discuss.elastic.co/u/SamuelSMendes)\
**Post date:** [September 21, 2022, 8:19pm UTC](https://discuss.elastic.co/t/is-it-possible-to-format-logstash-output-in-a-way-that-it-shows-in-uptime/314883/3 "2022-09-21T20:19:30Z")

</div>

I tried out and then got stucked on this error...

```auto
[2022-09-21T17:09:14.585-03:00][ERROR][http] ResponseError: search_phase_execution_exception: [script_exception] Reason: runtime error; [script_exception] Reason: runtime error
    at KibanaTransport.request (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\node_modules\@elastic\transport\lib\Transport.js:455:27)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at KibanaTransport.request (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\src\core\server\elasticsearch\client\create_transport.js:63:16)
    at Client.SearchApi [as search] (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\node_modules\@elastic\elasticsearch\lib\api\api\search.js:60:12)
    at Object.search (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\lib.js:51:15)
    at statusCount (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\get_snapshot_counts.js:43:7)
    at Object.getSnapshotCount (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\get_snapshot_counts.js:28:17)
    at handler (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\rest_api\snapshot\get_snapshot_count.js:40:12)
    at handler (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\rest_api\uptime_route_wrapper.js:57:17)
    at Router.handle (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\src\core\server\http\router\router.js:163:30)
    at handler (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\src\core\server\http\router\router.js:124:50)
    at exports.Manager.execute (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\node_modules\@hapi\hapi\lib\toolkit.js:60:28)
    at Object.internals.handler (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\node_modules\@hapi\hapi\lib\handler.js:46:20)
    at exports.execute (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\node_modules\@hapi\hapi\lib\handler.js:31:20)
    at Request._lifecycle (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\node_modules\@hapi\hapi\lib\request.js:371:32)
[2022-09-21T17:09:14.613-03:00][ERROR][http] TypeError: Cannot read properties of undefined (reading 'id')
    at summaryPingsToSummary (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\search\refine_potential_matches.js:92:32)
    at fullyMatchingIds (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\search\refine_potential_matches.js:77:22)
    at refinePotentialMatches (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\search\refine_potential_matches.js:31:16)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at MonitorSummaryIterator.fetchChunk [as chunkFetcher] (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\search\fetch_chunk.js:34:20)
    at MonitorSummaryIterator.attemptBufferMore (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\search\monitor_summary_iterator.js:147:21)
    at MonitorSummaryIterator.bufferNext (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\search\monitor_summary_iterator.js:121:22)
    at MonitorSummaryIterator.next (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\search\monitor_summary_iterator.js:55:5)
    at MonitorSummaryIterator.nextPage (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\search\monitor_summary_iterator.js:71:23)
    at Object.getMonitorStates (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\lib\requests\get_monitor_states.js:44:16)
    at handler (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\rest_api\monitors\monitor_list.js:53:22)
    at handler (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\x-pack\plugins\uptime\server\rest_api\uptime_route_wrapper.js:57:17)
    at Router.handle (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\src\core\server\http\router\router.js:163:30)
    at handler (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\src\core\server\http\router\router.js:124:50)
    at exports.Manager.execute (C:\Users\samuel.soares\Desktop\elk\elk8.0\kibana-8.0.0\node_modules\@hapi\hapi\lib\toolkit.js:60:28)

```

Here is how the mimic of the monitor looks like:

```auto
    mutate {
        
        add_field => {"agent.ephemeral_id" => "xxxxx"}
        add_field => {"agent.hostname" => "xxxxx"}
        add_field => {"agent.id" => "xxxxxx"}
        add_field => {"agent.name" => "xxxxxx"}
        add_field => {"agent.type" => "heartbeat"}
        add_field => {"agent.version" => "7.17.3"}
        add_field => {"ecs.version" => "1.12.0"}
        add_field => {"event.dataset" => "http"}
        add_field => {"http.response.body.bytes" => 0}
        add_field => {"http.response.body.hash" => "-"}
        add_field => {"http.response.headers.Cache-Control" => "-"}
        add_field => {"http.response.headers.Content-Length" => 0}
        add_field => {"http.response.headers.Date" => "%{[headers][date]}"}
        add_field => {"http.response.headers.Kbn-License-Sig" => "-"}
        add_field => {"http.response.headers.Kbn-Name" => "xxxxxxxx"}
        add_field => {"http.response.headers.Location" => "/spaces/ender"}
        add_field => {"http.response.headers.Referrer-Policy" => "no-referrer-when-downgrade"}
        add_field => {"http.response.headers.X-Content-Type-Options" => "nosniff"}
        add_field => {"http.response.mime_type" => "text/plain; charset=utf-8"}
        add_field => {"monitor.id" => "%{[body][udid]}"}
        add_field => {"monitor.name" => "%{[body][deviceName]}"}
        add_field => {"monitor.status" => "%{[body][connected]}"}
        add_field => {"http.url" => "https://xxxxxx.com"}
        add_field => {"http.response.status_code" => "-"}
        add_field => {"monitor.type" => "http"}
        add_field => {"url.domain" => "localhost"}
        add_field => {"url.full" => "https://xxxxxx/rest/deviceContent"}
        add_field => {"url.scheme" => "http"}
        add_field => {"summary.up" => 0}
        add_field => {"summary.down" => 0}
    

        }
    
    if "true" in [monitor.status]{
        mutate {
            replace => {"monitor.status" => "up"}
            replace => {"http.response.status_code" => 200}
            replace => {"summary.up" => 1}
        }
    } else {
        mutate {
            replace => {"monitor.status" => "down"}
            replace => {"summary.down" => 1}
        }
    }

```

so far everything goes to the cluster and I can see everything just right in the discover. But once I put the index in Uptime and try to see it send back this:

 ![Captura de Tela (1197)](https://us1.discourse-cdn.com/elastic/original/3X/1/5/153cfd92136936c07a16f128ce553227c2145180.png)

---

<div class="post-metadata">

**Author:** ![leandrojmp](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/leandrojmp/32/107231_2.png) [@leandrojmp](https://discuss.elastic.co/u/leandrojmp)\
**Post date:** [September 21, 2022, 8:50pm UTC](https://discuss.elastic.co/t/is-it-possible-to-format-logstash-output-in-a-way-that-it-shows-in-uptime/314883/4 "2022-09-21T20:50:25Z")

</div>

The fields names and mappings needs to be exactly the same.

Your mutates are wrong, in logstash using `agent.id` in a mutate filter means that it will create a field with a literal dot in the name.

You wil have `{ "agent.id": "value" }` and not `{ "agent": { "id": "value" } }`.

You need to use the format `[field][nested]`, for example `[agent][id]` , for every field in your mutate.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 19, 2022, 8:50pm UTC](https://discuss.elastic.co/t/is-it-possible-to-format-logstash-output-in-a-way-that-it-shows-in-uptime/314883/5 "2022-10-19T20:50:29Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.