# Elastic-agent 8.17.7: logs are not being harvested from .log files on Windows host **URL:** \ **Category:** Elastic Observability\ **Created:** [June 24, 2025, 6:18pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447 "2025-06-24T18:18:34Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![kiedis](https://avatars.discourse-cdn.com/v4/letter/k/ecc23a/32.png) [@kiedis](https://discuss.elastic.co/u/kiedis)\ **Post date:** [June 24, 2025, 6:18pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/1 "2025-06-24T18:18:34Z")
first i would like to point out that there is no official tutorial for Custom Logs (Filestream) integration in elastic docs, except [this piece](https://www.elastic.co/docs/reference/integrations/filestream) touting its numerous improvements when compared to its deprecated predecessor. Apparently, this Custom Logs (Filestream) integration is so simple and intuitive to implement that there is no need to drain one's precious time down the toilet for writing a step-by-step guide. Therefore, i already feel kinda bad for posting this question here but anyway - how the hell do i make this thing start harvesting data from .log files on a Windows host? I already: 1. gave this integration a name 2. provided a path to .log files 3. gave a name for a Dataset ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/d/5d32e2ae164097851f2d46b50d5ef8bbd226375d.png) Supposedly thats all it takes for Elastic Agent to start harvesting data but i dont see any logs coming in. What i do see is a bunch of trash logs & metrics, most probably being pushed to these indices (among others): ```auto .ds-logs-elastic_agent.filebeat-default-2025.06.23-000001 .ds-logs-elastic_agent.metricbeat-default-2025.06.23-000001 .ds-logs-elastic_agent-default-2025.06.23-000001 .ds-logs-system.auth-default-2025.06.23-000001 .ds-logs-elastic_agent.fleet_server-default-2025.06.23-000001 .ds-logs-system.syslog-default-2025.06.23-000001 ``` Thats about all the indexes that i see in elastic that contain "logs" substring in their names, and none of them pertain to my app's logs. Would the index for my app logs contain the name of the Dataset that i provided in the wizard? What should i look for when analyzing Elastic Agent's .ndjson files (i dont even see my app's name or path to log files mentioned there)? Are there some built-in tools which would help to debug this kind of problem? Anyone? ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 24, 2025, 7:58pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/2 "2025-06-24T19:58:23Z")
Hi @kiedis Welcome to the community. Can you show a listing of the directory of the logs? ![1000005731](https://us1.discourse-cdn.com/elastic/original/3X/f/3/f3b57f673256440697430314d9731113670e773c.png) Curious why `.logs.*` ? Also assuming on the agent you left the defaults and the agent is shipping agent logs.. Go to Discover dataview `logs-*` KQL `data_stream.dataset : "elastic_agent.filebeat"` Add the `message` and `host.name` ![Screenshot 2025-06-24 at 12.47.55 PM](https://us1.discourse-cdn.com/elastic/original/3X/3/9/397b8d96e27020131b17684ec9faf5db7bc026bd.jpeg) ---
**Author:** ![kiedis](https://avatars.discourse-cdn.com/v4/letter/k/ecc23a/32.png) [@kiedis](https://discuss.elastic.co/u/kiedis)\ **Post date:** [June 26, 2025, 8:05pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/3 "2025-06-26T20:05:45Z")
good evening Mr @stephenb, I use `.logs.*` because my files have these extensions due to the log rotation thing: ```auto dmskrnl.log dmskrnl.log.1 dmskrnl.log.2 ``` And when i issue `data_stream.dataset : "elastic_agent.filebeat"` i see this output in Discover: | Jun 26, 2025 @ 22:55:09.082 | Harvester started for paths: [/var/log/auth.log\* /var/log/secure\*] | obsv-linux | | --- | --- | --- | | Jun 26, 2025 @ 22:50:13.932 | File is inactive. Closing because close\_inactive of 5m0s reached. | obsv-linux | | Jun 26, 2025 @ 22:45:08.908 | Harvester started for paths: [/var/log/auth.log\* /var/log/secure\*] | obsv-linux | So, the Elastic Agent apparently is trying to harvest logs from `/var...` directory on my Linux box which is of course wrong. In my original post i already provided the printscreen which shows Windows path to the log files, integration name and dataset name. In addition to that i can confirm that Elastic Agent is properly deployed on the Windows machine and the `my-logs` integration is officially Healthy: ![image](https://us1.discourse-cdn.com/elastic/original/3X/8/1/8119b59f97189cb7725652b6acc7d41430e1d6c8.png) Supposedly that should be enough for the fruitful harvesting from the Windows machine? And speaking of any other available options, i did not touch anything and left to their defaults, like for example Namespace, Output and Agent policy. P.S. I wanted to paste more printscreens for more convincing impressions of my righteous configuration choices, but apparently new members of this community are assigned a meager ration of just 1 printscreen per post, so i am a bit apprehensive that this post might not be entirely clear and more questions might arise (which i'd be happy to answer) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 26, 2025, 10:14pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/4 "2025-06-26T22:14:23Z")
Hi @kiedis > [@kiedis](#): > > `/var...` directory on my Linux box which is of course wrong. You need to filter discover on the window hostname because all the agent logs are in the same file stream so try this... add the `host.name` as a column and filter on it in the KQL bar `data_stream.dataset : "elastic_agent.filebeat" and host.name: "yourwinhostname"` Also you can go the the windows machine and run `.\elastic-agent inspect` and / or `.\elastic-agent status` > [@kiedis](#): > > I use `.logs.*` because my files have these extensions due to the log rotation thing: > > ```auto > dmskrnl.log <<<< THIS WILL NOT GET HARVESTED > dmskrnl.log.1 > dmskrnl.log.2 > > ``` You will need to add a row with out the trailing `.*` ---
**Author:** ![kiedis](https://avatars.discourse-cdn.com/v4/letter/k/ecc23a/32.png) [@kiedis](https://discuss.elastic.co/u/kiedis)\ **Post date:** [June 27, 2025, 8:56pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/5 "2025-06-27T20:56:11Z")
dear Mr @stephenb, When i issue `data_stream.dataset : "elastic_agent.filebeat" and host.name : "WIN-GGI5VI06VT1"` i simply get `"no results match criteria"`. I went ahead and downloaded latest diagnostics file and verified that the Elastic Agent actually does successfully harvest my log files - i do see some actual log entries from my app in "message" field. Here is a sample containing a Warning-level log entry from my app about server connection failure, also indicating that this message supposedly should be delivered to `logs-my.app-default` index in Elasticsearch as per the `raw_index` field: ```auto {"log.level":"debug","@timestamp":"2025-06-27T23:08:39.146+0300","message":"Publish event: {\n \"@timestamp\": \"2025-06-27T20:08:39.142Z\",\n \"@metadata\": {\n \"beat\": \"filebeat\",\n \"type\": \"_doc\",\n \"version\": \"8.17.7\",\n \"raw_index\": \"logs-my.app-default\",\n \"input_id\": \"filestream-filestream-edd8d5e0-ded1-4f27-bc1e-a5abc8094855\",\n \"stream_id\": \"filestream-filestream.generic-edd8d5e0-ded1-4f27-bc1e-a5abc8094855\"\n },\n \"data_stream\": {\n \"dataset\": \"my.app\",\n \"namespace\": \"default\",\n \"type\": \"logs\"\n },\n \"elastic_agent\": {\n \"snapshot\": false,\n \"version\": \"8.17.7\",\n \"id\": \"8eca444a-01c2-44ad-bbb4-3a19e6481aa0\"\n },\n \"agent\": {\n \"ephemeral_id\": \"546f44a4-3932-40d1-bf80-8db2591a6b79\",\n \"name\": \"WIN-GGI5VI06VT1\",\n \"id\": \"8eca444a-01c2-44ad-bbb4-3a19e6481aa0\",\n \"type\": \"filebeat\",\n \"version\": \"8.17.7\"\n },\n \"ecs\": {\n \"version\": \"8.0.0\"\n },\n \"host\": {\n \"hostname\": \"WIN-GGI5VI06VT1\",\n \"name\": \"win-ggi5vi06vt1\",\n \"architecture\": \"x86_64\",\n \"os\": {\n \"family\": \"windows\",\n \"name\": \"Windows Server 2022 Standard\",\n \"kernel\": \"10.0.20348.3207 (WinBuild.160101.0800)\",\n \"build\": \"20348.3207\",\n \"type\": \"windows\",\n \"platform\": \"windows\",\n \"version\": \"10.0\"\n },\n \"id\": \"9f9ec748-4fe1-4f65-9e3a-45c020f20bf2\",\n \"ip\": [\n \"fe80::2cb8:ab28:f449:9e06\",\n \"192.168.1.111\"\n],\n \"mac\": [\n \"00-15-5D-01-B7-02\"\n]\n },\n \"log\": {\n \"offset\": 0,\n \"file\": {\n \"path\": \"C:\\\\Users\\\\Public\\\\Company\\\\Logs\\\\dmskrnl.log\",\n \"idxhi\": \"28508160\",\n \"idxlo\": \"20882\",\n \"vol\": \"2389561806\",\n \"fingerprint\": \"a654f81b7a09cc6e64d103f660f8194b21736403f94e569361d6bb5098eee7c7\"\n }\n },\n \"message\": \"2025-06-27 22:52:48,950 WARN [0x0000306c] myapp.secevent - (112) Server connection failed secure handshake. User: . Target: . Client: 192.168.1.111:52179.\",\n \"input\": {\n \"type\": \"filestream\"\n },\n \"event\": {\n \"dataset\": \"my.app\"\n }\n}","component":{"binary":"filebeat","dataset":"elastic_agent.filebeat","id":"filestream-default","type":"filestream"},"log":{"source":"filestream-default"},"service.name":"filebeat","log.type":"event","ecs.version":"1.6.0","log.logger":"processors","log.origin":{"file.line":215,"file.name":"processing/processors.go","function":"github.com/elastic/beats/v7/libbeat/publisher/processing.debugPrintProcessor.func1"},"ecs.version":"1.6.0"} ``` However, when i check all the existing log indexes in Elasticsearch, i do not see any mention of `logs-my.app-default`: ![image](https://us1.discourse-cdn.com/elastic/original/3X/6/1/6104e83ec97f329f326308699739b775975d00b2.png) The problem then appears to be with the connection establishment of some sorts: ```auto PS C:\Users\Administrator\elastic-agent-8.17.7-windows-x86_64> .\elastic-agent status Error: failed to communicate with Elastic Agent daemon: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: open \\\\.\\pipe\\B2kC2dS9J8fZ9F6omPD2JN1wibs4Yzc7.sock: The system cannot find the file specified." For help, please see our troubleshooting guide at https://www.elastic.co/guide/en/fleet/8.17/fleet-troubleshooting.html ``` Also: ```auto PS C:\Users\Administrator\elastic-agent-8.17.7-windows-x86_64> .\elastic-agent diagnostics [WARNING]: failed to fetch agent diagnostics: error in DiagnosticAgent RPC call: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: open \\\\.\\pipe\\B2kC2dS9J8fZ9F6omPD2JN1wibs4Yzc7.sock: The system cannot find the file specified."[WARNING]: failed to fetch unit diagnostics: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: open \\\\.\\pipe\\B2kC2dS9J8fZ9F6omPD2JN1wibs4Yzc7.sock: The system cannot find the file specified."[WARNING]: failed to fetch component diagnostics: error in DiagnosticComponents RPC call: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: open \\\\.\\pipe\\B2kC2dS9J8fZ9F6omPD2JN1wibs4Yzc7.sock: The system cannot find the file specified."Error: failed collecting diagnostics: no diags could be fetched ``` I tried to consult the troubleshooting guide which was kindly provided in the error message but was unable to procure any solution to this problem. If it's of any importance, the TCP connection between my Linux and Windows machines is established via port 8220: ```auto PS C:\Users\Administrator\elastic-agent-8.17.7-windows-x86_64> netstat -nat | findstr 8220 TCP 192.168.1.111:58370 192.168.1.113:8220 ESTABLISHED InHost ``` ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 28, 2025, 12:22am UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/6 "2025-06-28T00:22:18Z")
Hi @kiedis Lets Back Up ... exactly how did you install Elastic Agent? Did you follow the guided Installation from the Fleet UI? It should look something like ![Screenshot 2025-06-27 at 5.19.59 PM](https://us1.discourse-cdn.com/elastic/original/3X/e/0/e0bfee7107c37a2e057d3cdfcce61f14ac194a87.png) ```auto $ProgressPreference = 'SilentlyContinue' Invoke-WebRequest -Uri https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.18.3-windows-x86_64.zip -OutFile elastic-agent-8.18.3-windows-x86_64.zip Expand-Archive .\elastic-agent-8.18.3-windows-x86_64.zip -DestinationPath . cd elastic-agent-8.18.3-windows-x86_64 .\elastic-agent.exe install --url=https://sadfasdfasdfb3bd74c467.fleet.us-west1.gcp.cloud.es.io:443 --enrollment-token=asdfasdfasdfsh5NUk6dk9NUkphaUZRU0MwYWotZTIzdTU1Zw== ``` And if so if should install the Agent in ```auto Directory: C:\Program Files\Elastic\Agent Mode LastWriteTime Length Name ---- ------------- ------ ---- d----- 3/12/2025 3:10 AM data d----- 1/13/2025 7:36 PM vault -a---- 1/13/2025 7:36 PM 41 .build_hash.txt -a---- 2/28/2025 5:54 PM 6 .elastic-agent.active.commit -a---- 1/13/2025 7:36 PM 0 .installed -a---- 6/24/2025 12:19 PM 20972015 elastic-agent-20250623-1.ndjson -a---- 6/25/2025 1:07 AM 20971666 elastic-agent-20250624.ndjson -a---- 6/26/2025 2:44 AM 20972114 elastic-agent-20250625-1.ndjson -a---- 6/25/2025 1:56 PM 20973411 elastic-agent-20250625.ndjson -a---- 6/27/2025 4:21 AM 20972958 elastic-agent-20250626-1.ndjson -a---- 6/26/2025 3:32 PM 20972571 elastic-agent-20250626.ndjson -a---- 6/28/2025 12:18 AM 11719442 elastic-agent-20250627-1.ndjson -a---- 6/27/2025 5:09 PM 20973053 elastic-agent-20250627.ndjson -a---l 2/28/2025 5:54 PM 0 elastic-agent.exe -a---- 1/13/2025 7:36 PM 14829 elastic-agent.reference.yml -a---- 1/13/2025 7:36 PM 1947 elastic-agent.yml -a---- 1/13/2025 7:36 PM 12306 elastic-agent.yml.2025-01-13T19-36-59.6439.bak -a---- 6/11/2025 3:20 AM 735 fleet.enc -a---- 1/13/2025 7:36 PM 0 fleet.enc.lock -a---- 1/13/2025 7:36 PM 13675 LICENSE.txt -a---- 1/13/2025 7:36 PM 3388637 NOTICE.txt -a---- 1/13/2025 7:36 PM 643 otel.yml -a---- 1/13/2025 7:36 PM 88 otelcol.ps1 -a---- 1/13/2025 7:36 PM 351 README.md -a---- 2/28/2025 5:54 PM 0 watcher.lock PS C:\Program Files\Elastic\Agent> ``` Also if you dig deep into that error above ```auto "message": "2025-06-27 22:52:48,950 WARN [0x0000306c] myapp.secevent - (112) Server connection failed secure handshake. User: . Target: . Client: 192.168.1.111:52179.", ``` I think the agent is not connecting to elasticsearch Data is sent to Elasticsearch Typically `:9200` not the Fleet Endpoint `:8220` you may have a FW issues OR you default output for the Policy is not set up correct.... How did you set up Fleet? What does this look like on the Policy ![Screenshot 2025-06-27 at 5.37.55 PM](https://us1.discourse-cdn.com/elastic/original/3X/c/c/cc42bba1754e1c1c71b2127c2e5b616ea00a2685.png) ![Screenshot 2025-06-27 at 5.37.19 PM](https://us1.discourse-cdn.com/elastic/original/3X/7/1/718f66479dcc74429e80d8429d251cb5a2002aad.jpeg) ---
**Author:** ![kiedis](https://avatars.discourse-cdn.com/v4/letter/k/ecc23a/32.png) [@kiedis](https://discuss.elastic.co/u/kiedis)\ **Post date:** [June 30, 2025, 4:45pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/7 "2025-06-30T16:45:21Z")
dear Mr @stephenb, Your last 2 screenshots are especially helpful and i think i have narrowed down to the root cause. As i mentioned in my earlier posts, i left most of the settings to their default values in the Elastic Agent enrollment wizard. Among those defaults was Output option which assumes Elasticsearch running on the localhost: ![image](https://us1.discourse-cdn.com/elastic/original/3X/3/2/32940fcc8bdd94d1a77db428806820e0d17c16a3.png) However, my Elastic Agent should send data to the `Remote Elasticsearch` output (Elastic Agent is running on Windows host and Elasticsearch on Linux host). So now i am adding a new output but ran into another problem - i dont know how to generate Service Token. I try to follow the instructions provided in the wizard and used both curl via command line and via Kibana's dev tools but was unable to generate any token: `"error": "no handler found for uri [/localhost%3A5601/%3A/api/fleet/service_tokens?pretty=true] and method [POST]"` To recap: 1. Elasticsearch and Kibana are running on Linux host 192.168.1.113 2. Elastic Agent is running on Windows host 192.168.1.111 ![image](https://us1.discourse-cdn.com/elastic/original/3X/5/7/5777805612536f463c7c7d51b9a6b0bc36b44ad8.png) ---
**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\ **Post date:** [June 30, 2025, 4:59pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/8 "2025-06-30T16:59:21Z")
> [@kiedis](#): > > However, my Elastic Agent should send data to the `Remote Elasticsearch` output (Elastic Agent is running on Windows host and Elasticsearch on Linux host). OK So I know this sounds confusing.... but it is NOT a Remote Elastic Search Cluster... that is for something different .... so that is not the right path. Just go into the Default Output and Edit it and try putting in the correct URL to the elasticsearch host... you should not need to generate a new token etc... ![Screenshot 2025-06-30 at 9.54.22 AM](https://us1.discourse-cdn.com/elastic/original/3X/9/3/9383fba1fd8b09386dc957a17fd68744f81fd0b8.png) Try putting in `https://192.168.1.113:9200` I also notice that is say `http` not `https` Then try to redeploy the Agent.... if you are using self signed certs you will still need to use `--insecure` > [@kiedis](#): > > - Elasticsearch and Kibana are running on Linux host 192.168.1.113 > - Elastic Agent is running on Windows host 192.168.1.111 So when you first installed fleet most likely in the install command you used `localhost` for the Elasticsearch endpoint ... that is where it got it from... Instead you should have use the actual hostname / or IP. ---
**Author:** ![kiedis](https://avatars.discourse-cdn.com/v4/letter/k/ecc23a/32.png) [@kiedis](https://discuss.elastic.co/u/kiedis)\ **Post date:** [July 7, 2025, 8:31pm UTC](https://discuss.elastic.co/t/elastic-agent-8-17-7-logs-are-not-being-harvested-from-log-files-on-windows-host/379447/9 "2025-07-07T20:31:19Z")
thank you very much Mr @stephenb, the `https://192.168.1.113:9200` in the Default output fixed the problem