# Cannot start filebeat with configuration file

**URL:** <https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338>\
**Category:** Beats\
**Tags:** filebeat\
**Created:** [May 12, 2023, 9:05pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338 "2023-05-12T21:05:26Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 12, 2023, 9:05pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/1 "2023-05-12T21:05:26Z")

</div>

I'm not able to start filebeat with below config file:

```auto
filebeat.inputs:
  - type: filestream
    id: input1-id
    paths:
      - /home/yasser/data/sample1.log
output.elasticsearch:
  hosts: ["https://localhost:9200"]
  index: ["test-index"]

```

I get error:

```auto
{"log.level":"info","@timestamp":"2023-05-12T17:31:53.579+0300","log.origin":{"file.name":"instance/beat.go","file.line":724},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2023-05-12T17:31:53.580+0300","log.origin":{"file.name":"instance/beat.go","file.line":732},"message":"Beat ID: 56e8404f-da9e-403c-b3cb-225acdd386b8","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-05-12T17:31:53.580+0300","log.origin":{"file.name":"instance/beat.go","file.line":1071},"message":"Exiting: can not convert 'object' into 'string' accessing 'output.elasticsearch.index' (source:'/etc/filebeat/filebeat.yml')","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: **can not convert 'object' into 'string' accessing 'output.elasticsearch.index'** (source:'/etc/filebeat/filebeat.yml')
   ...fail!

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 12, 2023, 9:16pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/2 "2023-05-12T21:16:19Z")

</div>

Hi @Yasser_Alsawy Welcome to the community!

> [@Yasser\_Alsawy](#):
>
> ` index: ["test-index"]`

index is not an array  
` index: "test-index"`

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 13, 2023, 7:21am UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/3 "2023-05-13T07:21:30Z")

</div>

Thanks it starts now but the index is not created:  
curl -XGET localhost:9200/\_cat/indices

Isn't this configuration enough to create the index?

```auto
filebeat.inputs:
  - type: filestream
    id: input1-id
    paths:
      - /home/yasser/data/sample1.log
output.elasticsearch:
  hosts: ["https://localhost:9200"]
  index: "test-index-%{+yyyy.MM.dd}]"
setup.template:
  name: "test-index-*"
  pattern: "test-index"

```

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 13, 2023, 1:37pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/4 "2023-05-13T13:37:43Z")

</div>

Is there data in the log file??

What version?

Filebeat will only try to load the data once so if it already tried it won't do it again

You will need to clean out the data registry.

What did the filebeat log show? Any errors?

> [@Yasser\_Alsawy](#):
>
> ```auto
> output.elasticsearch:
> hosts: ["https://localhost:9200"]
> 
> ```

Is elasticsearch really run you on HTTPS?

I suspect there's connection errors... You will need to supply a username and password.  
If it's a self-signed shirt you're going to have to ignore the verification or provide the CA?

How did you set up elasticsearch?

If you want help, you're going to have to provide more detail.

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 13, 2023, 8:10pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/5 "2023-05-13T20:10:11Z")

</div>

> [@stephenb](#):
>
> You will need to clean out the data registry.

it worked showing the index when I added below line to config file:

```auto
setup.ilm:
  enabled: false

```

now I can see the index at Discover but with no data. and it is still not shown at Index Management. and curl does not get this index as well. Does this mean index is created but not single transaction done so no single instance of this index exists?

the logs when starting show:

```auto
> 
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.161+0300","log.origin":{"file.name":"instance/beat.go","file.line":724},"message":"Home path: [/usr/share/filebeat] Config path: [/etc/filebeat] Data path: [/var/lib/filebeat] Logs path: [/var/log/filebeat]","service.name":"filebeat","ecs.version":"1.6.0"}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.161+0300","log.origin":{"file.name":"instance/beat.go","file.line":732},"message":"Beat ID: 56e8404f-da9e-403c-b3cb-225acdd386b8","service.name":"filebeat","ecs.version":"1.6.0"}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.161+0300","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1096},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/etc/filebeat","data":"/var/lib/filebeat","home":"/usr/share/filebeat","logs":"/var/log/filebeat"},"type":"filebeat","uuid":"56e8404f-da9e-403c-b3cb-225acdd386b8"},"ecs.version":"1.6.0"}}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.161+0300","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1105},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"a8dbc6c06381f4fe33a5dc23906d63c04c9e2444","libbeat":"8.7.0","time":"2023-03-23T00:44:06.000Z","version":"8.7.0"},"ecs.version":"1.6.0"}}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.161+0300","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1108},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"amd64","max_procs":20,"version":"go1.19.7"},"ecs.version":"1.6.0"}}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.162+0300","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1114},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"x86_64","boot_time":"2023-05-13T12:21:34+03:00","containerized":false,"name":"Yasser-Dell","ip":["127.0.0.1","::1","172.25.6.13","fe80::215:5dff:fed7:dc17"],"kernel_version":"5.15.90.1-microsoft-standard-WSL2","mac":["00:15:5d:d7:dc:17"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.6 LTS (Focal Fossa)","major":20,"minor":4,"patch":6,"codename":"focal"},"timezone":"+03","timezone_offset_sec":10800,"id":"3bda63e1e73958fadc5c99fa642befe5"},"ecs.version":"1.6.0"}}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.162+0300","log.logger":"beat","log.origin":{"file.name":"instance/beat.go","file.line":1143},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"effective":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"bounding":["chown","dac_override","dac_read_search","fowner","fsetid","kill","setgid","setuid","setpcap","linux_immutable","net_bind_service","net_broadcast","net_admin","net_raw","ipc_lock","ipc_owner","sys_module","sys_rawio","sys_chroot","sys_ptrace","sys_pacct","sys_admin","sys_boot","sys_nice","sys_resource","sys_time","sys_tty_config","mknod","lease","audit_write","audit_control","setfcap","mac_override","mac_admin","syslog","wake_alarm","block_suspend","audit_read","38","39","40"],"ambient":null},"cwd":"/","exe":"/usr/share/filebeat/bin/filebeat","name":"filebeat","pid":2255,"ppid":2254,"seccomp":{"mode":"disabled","no_new_privs":false},"start_time":"2023-05-13T17:06:05.020+0300"},"ecs.version":"1.6.0"}}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.162+0300","log.origin":{"file.name":"instance/beat.go","file.line":297},"message":"Setup Beat: filebeat; Version: 8.7.0","service.name":"filebeat","ecs.version":"1.6.0"}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.166+0300","log.logger":"esclientleg","log.origin":{"file.name":"eslegclient/connection.go","file.line":108},"message":"elasticsearch url: https://localhost:9200","service.name":"filebeat","ecs.version":"1.6.0"}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.166+0300","log.logger":"publisher","log.origin":{"file.name":"pipeline/module.go","file.line":113},"message":"Beat name: Yasser-Dell","service.name":"filebeat","ecs.version":"1.6.0"}
> {"log.level":"info","@timestamp":"2023-05-13T17:06:05.166+0300","log.logger":"modules","log.origin":{"file.name":"fileset/modules.go","file.line":120},"message":"Enabled modules/filesets: ","service.name":"filebeat","ecs.version":"1.6.0"}
> Config OK

```

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 13, 2023, 8:21pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/6 "2023-05-13T20:21:21Z")

</div>

Elasticsearch runs properly at http localhost:9200  
I neither installed a certificate nor set user/password. I have corrected input path to be http not https but still having same issue of no data uploaded.

the input file has just testing data in format:  
name: value

I tried to load the file many times but data never uploaded, do I still to delete registry?

As you can see from logs, I run Elasticsearch and filebeat at my Windows 11 WSL (ubuntu) for testing. does this include any shortage or challenge?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 13, 2023, 10:17pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/7 "2023-05-13T22:17:15Z")

</div>

> [@Yasser\_Alsawy](#):
>
> I tried to load the file many times but data never uploaded, do I still to delete registry?

Yes probably... Look [here](https://www.elastic.co/guide/en/beats/filebeat/current/directory-layout.html#directory-layout-archive) and clean the data directory

> [@Yasser\_Alsawy](#):
>
> As you can see from logs, I run Elasticsearch and filebeat at my Windows 11 WSL (ubuntu) for testing. does this include any shortage or challenge?

That should be fine...

One thing is I rarely advise starting with a custom index name if you are new as there are many benefits of using the defaults... But that is just a suggestion.

For new users I recommend start with the defaults, get to know the components, configuration and options and _ **then** _ start changing things...

If you had just taken the default filebeat.yml and simply enabled the filestream and put the path to your log file it would all probably already be working.

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 14, 2023, 10:22am UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/8 "2023-05-14T10:22:28Z")

</div>

I have followed your suggestion using default yml file with no index name:

```auto
###################### Filebeat Configuration Example #########################

# ============================== Filebeat inputs ===============================

filebeat.inputs:

# filestream is an input for collecting log messages from files.
- type: filestream

  # Unique ID among all inputs, an ID is required.
  id: my-filestream-id

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/sample1.log

# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

# ================================== Outputs ===================================

# Configure what output to use when sending the data collected by the beat.

# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  hosts: ["localhost:9200"]

# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

```

I ran all successfully before starting filebeat service:  
./filebeat test config  
./filebeat test output  
./filebeat setup

but neither index exists at Kibana index management not kibana index patterns

but when running my customized yml with an index name, index pattern exists but no index at index management. I can find the index at Discover but with no data.

```auto
filebeat.inputs:
  - type: filestream
    id: input1-id
    paths:
      - /home/yasser/data/sample1.log
output.elasticsearch:
  hosts: ["https://localhost:9200"]
  index: "test-index-%{+yyyy.MM.dd}]"
setup.ilm:
  enabled: false
setup.template:
  enabled: false
  name: "test-index"
  pattern: "test-index-*"

setup.dashboards.index: "test-index-*"

```

how could I troubleshoot both cases?

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 14, 2023, 10:32am UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/9 "2023-05-14T10:32:45Z")

</div>

when trying to run filebeat using ./filebeat -e, instead of service filebeat start, as I used before, I got a clear error:

```auto
{"log.level":"info","@timestamp":"2023-05-14T13:26:56.036+0300","log.origin":{"file.name":"instance/beat.go","file.line":483},"message":"filebeat stopped.","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"error","@timestamp":"2023-05-14T13:26:56.036+0300","log.origin":{"file.name":"instance/beat.go","file.line":1071},"message":"Exiting: error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/filebeat/bin/kibana: No directory /usr/share/filebeat/bin/kibana/7","service.name":"filebeat","ecs.version":"1.6.0"}
Exiting: error importing Kibana dashboards: fail to import the dashboards in Kibana: Error importing directory /usr/share/filebeat/bin/kibana: No directory /usr/share/filebeat/bin/kibana/7

```

should I set a dashboard property here as it is trying to read from a missing directory: /usr/share/filebeat/bin/kibana ?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 14, 2023, 1:31pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/10 "2023-05-14T13:31:06Z")

</div>

With the default...

> [@Yasser\_Alsawy](#):
>
> ```auto
> paths:
> - /var/log/sample1.log
> 
> ```

You need to set this to the location of your sample log.

` /home/yasser/data/sample1.log`  
Also did you clean out the data directory as I instructed?

Then try starting as a service.

This will creat a data stream did you read about data stream...

Should be named

`filebeat-8.7.1`

Assuming you are using the latest version

Do not try to change anything with the dashboards, try running as a service again

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 14, 2023, 4:28pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/11 "2023-05-14T16:28:19Z")

</div>

I have moved my input log file into /var/log as the path dexcribed in template config file  
I have deleted both log.json and meta.json from /var/lib/filebeat/registry/filebeat

but the link you provided shows data folder as:  
data: The location for persistent data files. - {extract.path}/data  
is this: /etc/filebeat ? I cannot find data folder there

I started filebeat as service (after restarting both elasticsearch and kibana)

Index management --\> reload indices: nothing starting with filebeat is shown  
Kibana --\> Index patterns: only old index created by specifying the name is shown but nothing started with filebeat is shown  
Discover: no new index or starting with filebeat is shown

```auto
###################### Filebeat Configuration Example #########################

# ============================== Filebeat inputs ===============================

filebeat.inputs:

# filestream is an input for collecting log messages from files.
- type: filestream

  # Unique ID among all inputs, an ID is required.
  id: my-filestream-id

  # Change to true to enable this input configuration.
  enabled: true

  # Paths that should be crawled and fetched. Glob based paths.
  paths:
    - /var/log/sample1.log

# ======================= Elasticsearch template setting =======================

setup.template.settings:
  index.number_of_shards: 1
  #index.codec: best_compression
  #_source.enabled: false

# ================================== Outputs ===================================

# Configure what output to use when sending the data collected by the beat.

# ---------------------------- Elasticsearch Output ----------------------------
output.elasticsearch:
  hosts: ["localhost:9200"]

# ================================= Processors =================================
processors:
  - add_host_metadata:
      when.not.contains.tags: forwarded
  - add_cloud_metadata: ~
  - add_docker_metadata: ~
  - add_kubernetes_metadata: ~

```

I'm running Elasticsearch 7.17.10 as shown from localhost:9200

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 14, 2023, 4:39pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/12 "2023-05-14T16:39:12Z")

</div>

From Kibana -\> Dev Tools run

`GET _cat/indices?v`

Something simple is going on. If nothing else, just delete your entire installation of filebeat and start over. Not sure what went wrong.

And follow the [quick start guide](https://www.elastic.co/guide/en/beats/filebeat/7.17/filebeat-installation-configuration.html)

I'm not an expert on the Ubuntu on Windows. I suppose it could be something to do with that...

But I don't think so because I think we'd get fundamental errors.... You're just having problems with reading data

Exactly. How did you install? Did you use a tar.gz or RPM

This makes a difference of how you run the setup command.

I think you were mixing things up and moving things around perhaps.

So exactly how did you install? What exact steps?

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 14, 2023, 5:23pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/13 "2023-05-14T17:23:42Z")

</div>

> [@stephenb](#):
>
> GET \_cat/indices?v

here is the indices for dev tools:

```auto
#! Elasticsearch built-in security features are not enabled. Without authentication, your cluster could be accessible to anyone. See https://www.elastic.co/guide/en/elasticsearch/reference/7.17/security-minimal-setup.html to enable security.
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
green open .geoip_databases DlDbDHzXRtqxnjRnOtrUgQ 1 0 37 0 35.6mb 35.6mb
yellow open cars OGYpKAnQS0Wo-QSmglws3A 1 1 11 0 42.9kb 42.9kb
green open .apm-custom-link VeQQANccQSaqaakqTum_xg 1 0 0 0 226b 226b
green open .kibana_7.17.9_001 u_H7qUAZQYq20Aw5TD2wNQ 1 0 431 11 2.4mb 2.4mb
green open .apm-agent-configuration sfhKwy0-QoKysQHszY_W4Q 1 0 0 0 226b 226b
green open .kibana_7.17.10_001 c07j9LEcQbejxNkFQyXKoA 1 0 2227 34 3.2mb 3.2mb
green open .kibana_task_manager_7.17.9_001 MzvCefHnRA2B_0n1kqRLSQ 1 0 17 16 19.5kb 19.5kb
green open .async-search hbo_Ww-wQBatJXSR1R5qvQ 1 0 0 0 256b 256b
green open .kibana_task_manager_7.17.10_001 lPWjIWv4R6GIn-cx6Zql0Q 1 0 17 3264 1.8mb 1.8mb
green open .tasks nDfvbnf5TkWgh0xSokV41A 1 0 27 0 53.2kb 53.2kb
yellow open kids V_a-9pl-SFCCrugfd3TTqA 1 1 7 0 17.4kb 17.4kb

```

neither filebeat nor test-index I created using specific index name exists.  
before going in uninstall and install again:

- is any issue with v 7.17? how could I get v 8?
- any issue with error for folder of kibana does not exist when starting using ./filebeat instead of starting as a service?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 14, 2023, 5:25pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/14 "2023-05-14T17:25:55Z")

</div>

Exactly how did you install? ... the most important question to answer... which you have not.

Exactly  
What did you download?  
Exactly How did you download it?  
Exactly What commands did you run?

I would uninstall / delete everything and reinstall, this should not be hard if you followed the exact steps in the quick start guide.

I would just use the tar.gz to start that is easiest to get to work.

THEN if that works you can make it a service.

This is not a 7.x vs 8.x issue, there is something fundamental incorrect.

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 14, 2023, 6:07pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/15 "2023-05-14T18:07:34Z")

</div>

When trying to uninstall elasticseardch I was getting error:  
E: Conflicting values set for option Signed-By regarding source [https://artifacts.elastic.co/packages/7.x/apt/](https://artifacts.elastic.co/packages/7.x/apt/) stable: /usr/share/keyrings/elastic.gpg !=  
E: The list of sources could not be read.

so I did below (even I do not fully understand the keyrings):  
yasser@Yasser-Dell:/usr/share/keyrings$ cd /etc/apt/sources.list.d  
yasser@Yasser-Dell:/etc/apt/sources.list.d$ ls -ltr  
total 12  
-rw-r--r-- 1 root root 112 Apr 4 12:33 docker.list  
-rw-r--r-- 1 root root 274 May 12 10:29 elastic-7.x.list  
-rw-r--r-- 1 root root 112 May 14 20:54 elastic-8.x.list  
yasser@Yasser-Dell:/etc/apt/sources.list.d$ sudo rm elastic-8.x.list  
yasser@Yasser-Dell:/etc/apt/sources.list.d$ sudo rm elastic-7.x.list

it did not fix the problem of filebeat index, but I'm able to uninstall elasticsearch now and I'm able to run now:  
sudo apt update

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 14, 2023, 6:24pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/16 "2023-05-14T18:24:14Z")

</div>

Apologies I was asking about how exactly you installed filebeat.

I am unclear on what the state of filebeat is at this point.

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 14, 2023, 8:21pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/17 "2023-05-14T20:21:48Z")

</div>

I remember I used only:  
sudo apt install filebeat

should I use:  
curl -L -O [https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.6.2-x86\_64.rpm](https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.6.2-x86_64.rpm)

do you recommend uninstalling and installing again? keeping elasticsearch 7.17 as is?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 14, 2023, 8:39pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/18 "2023-05-14T20:39:46Z")

</div>

Hi Yasser

I am trying to be very clear.... As I already recommended above

> [@stephenb](#):
>
> I would uninstall / delete everything and reinstall, this should not be hard if you followed the exact steps in the quick start guide.
> 
> I would just use the tar.gz to start that is easiest to get to work.

So yes uninstall, remove, delete, purge whatever you need to do ... clean up.

And then I would install with the tar.gz from the quick start guide I recommended above.

DO not change anything and Only enable the filestream and the path to your file

run

```auto
./filebeat setup -e
./filebeat -e

```

IF / When you get that working you can save that configuration cleanup and then install as an RPM if you like.

Again I suggest if you want to install with APY YUM .rpm or .deb that you follow the documentation here...

> **[Repositories for APT and YUM | Filebeat Reference \[7.17\] | Elastic](https://www.elastic.co/guide/en/beats/filebeat/7.17/setup-repositories.html)**

Again if you **follow the quick start guide** it tells you EXACTLY what to do .. .or you can just keep trying random commands

I would not recommend going to 8.x since your cluster is 7.x

Good luck I am sure you can get it working.

---

<div class="post-metadata">

**Author:** ![Yasser\_Alsawy](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/yasser_alsawy/32/120976_2.png) [@Yasser\_Alsawy](https://discuss.elastic.co/u/Yasser_Alsawy)\
**Post date:** [May 15, 2023, 1:38pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/19 "2023-05-15T13:38:19Z")

</div>

Thanks a lot, Stephenb for your support and patience!  
I found the problem when I uninstalled every thing before installing again indifferent way:

1. I installed using Debian file (because I have ubuntu) which makes a real mess as some files are in /usr/share, /etc and the running directory as well
2. some parts of installation done with sudo which causes conflict at files access

I have:

- uninstalled using: sudo apt remove
- then removing the files from all folders
- installing using only tar files so installation directory is at one place containing all running file, log and config file
- I have used custom config file (sorry for not starting with the default template):

```auto
filebeat.inputs:
  - type: filestream
    id: input1-id
    paths:
      - /home/yasser/data/sample1.log
output.elasticsearch:
  hosts: ["http://localhost:9200"]
  index: "yasser-%{+yyyy.MM.dd}]"
setup.ilm:
  enabled: false
setup.template:
  enabled: false
  name: "yasser"
  pattern: "yasser-*"

```

but problem now, I find at Kibana index management:  
yasser-2023.05.15] (I do not know why extra closing square bracket!)

while at discover only:  
filebeat-\*  
with error message: No matching indices found: No indices match "filebeat-\*"

and I cannot find kibana --\> indext patterns  
as I used to find at previous kibana

did I miss something here?

---

<div class="post-metadata">

**Author:** ![stephenb](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/stephenb/32/40856_2.png) [@stephenb](https://discuss.elastic.co/u/stephenb)\
**Post date:** [May 15, 2023, 2:23pm UTC](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338/20 "2023-05-15T14:23:31Z")

</div>

> [@Yasser\_Alsawy](#):
>
> `index: "yasser-%{+yyyy.MM.dd}]"` \<\<\<\<

Typos.....it's right there

And since you create your own index you need to create your own index pattern it is not automatically created for you.

And shouldn't this be enabled

> [@Yasser\_Alsawy](#):
>
> ```auto
> setup.template:
> enabled: false
> 
> ```

Also you might want to look at [this](https://discuss.elastic.co/t/filebeat-writing-to-its-own-index/310842/28) post If you are indeed, set to on creating your own indices 🙂

Again, I normally suggest starting with the defaults and learning and then creating interesting things ... 🙂

[Next page](https://discuss.elastic.co/t/cannot-start-filebeat-with-configuration-file/333338.md?page=2)