# Elastic-agent osquery through logstash doesn't work

**URL:** <https://discuss.elastic.co/t/elastic-agent-osquery-through-logstash-doesnt-work/312783>\
**Category:** Elastic Agent\
**Tags:** fleet\
**Created:** [August 24, 2022, 9:32am UTC](https://discuss.elastic.co/t/elastic-agent-osquery-through-logstash-doesnt-work/312783 "2022-08-24T09:32:54Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![heikis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heikis/32/80983_2.png) [@heikis](https://discuss.elastic.co/u/heikis)\
**Post date:** [August 24, 2022, 9:32am UTC](https://discuss.elastic.co/t/elastic-agent-osquery-through-logstash-doesnt-work/312783/1 "2022-08-24T09:32:54Z")

</div>

Hello. We are managing Elastic Agents in Fleet. Entire Elastic Stack is on version 8.3.3.

We have problems doing osqueries in Kibana- the query seems to be successful but we get no results and no indexes are created:  
`"2 agents have responded, no osquery data has been reported"`.

In Fleet we have configured agent's output to logstash. In logs we see errors:  
`"Failed to publish events caused by: write tcp x.x.x.x:51787 (agent) -> y.y.y.y:5046 (logstash): wsasend: An existing connection was forcibly closed by the remote host"`.

Is osquery supposed to work with logstash as output on agents? I can confirm that osquery is working when agent output is set to elasticsearch, but we need Logstash to work.

Any ideas/suggestions/things to try?

Thanks!

---

<div class="post-metadata">

**Author:** ![BenB196](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/benb196/32/83401_2.png) [@BenB196](https://discuss.elastic.co/u/BenB196)\
**Post date:** [August 24, 2022, 11:43am UTC](https://discuss.elastic.co/t/elastic-agent-osquery-through-logstash-doesnt-work/312783/2 "2022-08-24T11:43:31Z")

</div>

Looks like this is a bug that should be fixed in the next release ([Osquerybeat: Fix osquerybeat is not running with logstash output by aleksmaus · Pull Request #674 · elastic/elastic-agent · GitHub](https://github.com/elastic/elastic-agent/pull/674))

---

<div class="post-metadata">

**Author:** ![aleksmaus](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/aleksmaus/32/56530_2.png) [@aleksmaus](https://discuss.elastic.co/u/aleksmaus)\
**Post date:** [August 24, 2022, 3:19pm UTC](https://discuss.elastic.co/t/elastic-agent-osquery-through-logstash-doesnt-work/312783/3 "2022-08-24T15:19:53Z")

</div>

@BenB196 That PR fixes the issue with the agent not running osquerybeat for any other outputs besides elasticsearch. It was merged into 8.3 here [[8.3](backport #674) Osquerybeat: Fix osquerybeat is not running with logstash output by mergify[bot] · Pull Request #681 · elastic/elastic-agent · GitHub](https://github.com/elastic/elastic-agent/pull/681) and was released as a part of 8.3.3, so the agent should run osquerybeat.

There was another issue where the data was not routed/saved properly into the elasticsearch datastream because the events where missing the datastream properties.

> <https://github.com/elastic/beats/pull/32543>
>
> \## What does this PR do?
> 
> Adds the \`\`\`add\_data\_stream\`\`\` processors that appen…ds the \`\`\`data\_stream\`\`\` fields to the events. This enabled better compatibility with the logstash configuration allowing it to figure out the destination datastream for the results of the osquery.
> 
> \## Why is it important?
> 
> Better support osquery with logstash as the output.
> 
> \## Checklist
> 
> \- \[x\] My code follows the style guidelines of this project
> \- \[x\] I have commented my code, particularly in hard-to-understand areas
> 
> 
> \## How to test this PR locally
> 
> Here is an example logstash pipeline configuration that was not working before this change:
> 
> \`\`\`
> input {
> elastic\_agent {
> port =\> 5044
> ssl =\> true
> ssl\_certificate\_authorities =\> \["certs/ca/ca.crt"\]
> ssl\_certificate =\> "certs/logstash.crt"
> ssl\_key =\> "certs/logstash.pkcs8.key"
> ssl\_verify\_mode =\> "force\_peer"
> }
> }
> 
> filter {
> mutate {
> rename =\> \["\_host", "host" \]
> }
> }
> 
> output {
> elasticsearch {
> hosts =\> "https://\<redacted\>.us-west2.gcp.elastic-cloud.com:443"
> data\_stream =\> true
> ssl =\> true
> user =\> elastic
> password =\> \<redacted\>
> }
> }
> \`\`\`
> 
> After the change the event has additional datastream attributes
> \`\`\`
> "data\_stream" =\> {
> "namespace" =\> "default",
> "type" =\> "logs",
> "dataset" =\> "osquery\_manager.result"
> }, 
> \`\`\`
> 
> \## Screenshots
> 
> Verified the result are now reaching the osquery\_manager datastream:
> 
> \<img width="1282" alt="Screen Shot 2022-07-28 at 6 08 15 PM" src="https://user-images.githubusercontent.com/872351/181648483-fa8fea43-4732-4924-9eb7-2e6995ed28f8.png"\>

This was merged into 8.4 and should be a part of 8.4.0 release.

---

<div class="post-metadata">

**Author:** ![heikis](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/heikis/32/80983_2.png) [@heikis](https://discuss.elastic.co/u/heikis)\
**Post date:** [August 25, 2022, 10:31am UTC](https://discuss.elastic.co/t/elastic-agent-osquery-through-logstash-doesnt-work/312783/4 "2022-08-25T10:31:05Z")

</div>

We got osquery working in our environment.  
Upgraded the elasticsearch and logstash nodes to 8.4.0 but that did not resolve the problem.  
Then upgraded osquerymanager integration from 1.3.1 to 1.4.1 - still nothing.  
Then upgraded Fleet Server agent and all endpoint Elastic-agents to 8.4.0 and osqueries are now working. Thank you for your help!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [September 22, 2022, 10:31am UTC](https://discuss.elastic.co/t/elastic-agent-osquery-through-logstash-doesnt-work/312783/5 "2022-09-22T10:31:47Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
