# Send logs from filebeat to elasticsearch

**URL:** <https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628>\
**Category:** Elasticsearch\
**Created:** [April 24, 2023, 9:56am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628 "2023-04-24T09:56:25Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Neelam\_Zanvar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/neelam_zanvar/32/103116_2.png) [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\
**Post date:** [April 24, 2023, 9:56am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/1 "2023-04-24T09:56:25Z")

</div>

I am trying to send logs from filebeat to elasticsearch. Here is the filbeat.yml

```auto
filebeat.inputs:
- type: filestream
  id: my-filestream-id
  enabled: true
  paths:
    - C:\ProgramData\sample_logs\sample.log
- type: log
  enabled: true
  paths:
    - C:\ProgramData\sample_logs\sample.log

filebeat.config.modules:
  # Glob pattern for configuration loading
  path: ${path.config}/modules.d/*.yml

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["elastic_ip:9200"]

  # Protocol - either `http` (default) or `https`.
  #protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "pwd"

```

Have enabled elasticsearch module.  
The command  
.\filebeat -e -c "C:\Program Files\Filebeat\filebeat.yml" test output  
gives connection ok result.  
.\filebeat -e -c "C:\Program Files\Filebeat\filebeat.yml" -d "publish"  
Displays alot of entries on the console.

on kibana, When i navigate to discover tab i am getting an option of creating data view for index pattern filebeat-8.7.0. So the index has been created. But there are no logs on the dashboard

 ![image](https://us1.discourse-cdn.com/elastic/original/3X/c/a/caa5a58b0f2356b439aa1ff0f1f8dbec3b944d7b.png)

Also there is no entry of filebeat-8.7.0 in index management tab.  
Why are there no entries coming? the path configured has log data.

---

<div class="post-metadata">

**Author:** ![Christian\_Dahlqvist](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/christian_dahlqvist/32/4617_2.png) [@Christian\_Dahlqvist](https://discuss.elastic.co/u/Christian_Dahlqvist)\
**Post date:** [April 24, 2023, 10:28am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/2 "2023-04-24T10:28:38Z")

</div>

> [@Neelam\_Zanvar](#):
>
> ```auto
> # Protocol - either `http` (default) or `https`.
> #protocol: "https"
> 
> ```

If the cluster is secured this should be `https`, so try comment out that line.

Also check the Filebeat logs as it may contain clues and please capture and post what it output in the console.

---

<div class="post-metadata">

**Author:** ![Neelam\_Zanvar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/neelam_zanvar/32/103116_2.png) [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\
**Post date:** [April 24, 2023, 10:35am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/3 "2023-04-24T10:35:21Z")

</div>

I've disabled https for the cluster. There are no traces in the filebeat logs. It's very big and difficult to post in here

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 24, 2023, 1:28pm UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/4 "2023-04-24T13:28:36Z")

</div>

From Kibana Dev Tools run

`GET _cat/indices/?`

And show the output

---

<div class="post-metadata">

**Author:** ![Neelam\_Zanvar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/neelam_zanvar/32/103116_2.png) [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\
**Post date:** [April 25, 2023, 4:13am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/5 "2023-04-25T04:13:45Z")

</div>

```auto
yellow open .ds-filebeat-8.7.0-2023.04.24-000001 f4117_dZSOGMLf_Ivl5CJw 1 1 0 0 225b 225b

```

Here's the output

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [April 25, 2023, 5:19am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/6 "2023-04-25T05:19:14Z")

</div>

Yup no data...

Can you share the filebeat logs.

Also filebeat will only read the file once... So if it read it already it not read it again.

You will need to clean out the data registry.

---

<div class="post-metadata">

**Author:** ![Neelam\_Zanvar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/neelam_zanvar/32/103116_2.png) [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\
**Post date:** [April 25, 2023, 5:34am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/7 "2023-04-25T05:34:19Z")

</div>

This is filebeat.yml

```auto
filebeat.inputs:

- type: filestream
  id: my-filestream-id

  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - C:\ProgramData\sample_logs\sample.log

- type: log

  enabled: true
  paths:
    - C:\ProgramData\sample_logs\sample.log

- type: syslog
  enabled: false
  logging:
  level: info
  to_files: true
  to_syslog: false

output.elasticsearch:
  # Array of hosts to connect to.
  hosts: ["elastic_ip:9200"]

  # Protocol - either `http` (default) or `https`.
  #protocol: "https"

  # Authentication credentials - either API key or username/password.
  #api_key: "id:api_key"
  username: "elastic"
  password: "pwd"

```

This is the log file C:\ProgramData\sample\_logs\sample.log

```auto
192.168.2.20 - - [28/Jul/2006:10:27:10 -0300] "GET /cgi-bin/try/ HTTP/1.0" 200 3395
127.0.0.1 - - [28/Jul/2006:10:22:04 -0300] "GET / HTTP/1.0" 200 2216
192.168.2.20 - - [28/Jul/2006:10:27:10 -0300] "GET /cgi-bin/try/ HTTP/1.0" 200 395
127.0.0.1 - - [28/Jul/2006:10:22:04 -0300] "GET / HTTP/1.0" 200 221

```

If the file has been read once i should be able to see it in the logs right? But there is no data at all

---

<div class="post-metadata">

**Author:** ![Neelam\_Zanvar](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/neelam_zanvar/32/103116_2.png) [@Neelam\_Zanvar](https://discuss.elastic.co/u/Neelam_Zanvar)\
**Post date:** [April 25, 2023, 5:39am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/8 "2023-04-25T05:39:27Z")

</div>

Hey, I received the logs. I was wrong with the file path. The file was saved as sample.log.txt and i was specifying sample.log  
Thank you so much for the quick response

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 23, 2023, 5:40am UTC](https://discuss.elastic.co/t/send-logs-from-filebeat-to-elasticsearch/330628/9 "2023-05-23T05:40:22Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.
