# I cannot see two indices in Kibana

**URL:** <https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755>\
**Category:** Logstash\
**Created:** [March 9, 2020, 3:40pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755 "2020-03-09T15:40:04Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [March 9, 2020, 3:40pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/1 "2020-03-09T15:40:04Z")

</div>

I have the following output in my logstash output file. When I look at the loaded indices in Kibana I only see the **var\_log\_common-** \*

30-elasticsearch-output.conf

```
output {
  if [source] == "/var/log/nginx/access.log" or [source] == "/var/log/nginx/error.log" {
    elasticsearch {
      hosts => "localhost:9200"
      index => "var_log_nginx-%{+YYYY.MM.dd}"
    }
  } 
  else {
    elasticsearch {
      hosts => "localhost:9200"
      index => "var_log_common-%{+YYYY.MM.dd}"
    }
  }
}

```

Does this look correct to you?

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2020, 3:57pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/2 "2020-03-09T15:57:06Z")

</div>

It looks OK. Are you sure that you are receiving messages that should match? If the condition isn't matching you would see these messages in the `var_log_common-*` index, as that is where they would land if the condition was false.

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [March 9, 2020, 3:57pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/3 "2020-03-09T15:57:17Z")

</div>

Update:

After running this in elasticsearch I see the index is not listed:

```
curl http://localhost:9200/_aliases?pretty=true
{
  ".kibana_1" : {
    "aliases" : {
      ".kibana" : { }
    }
  },
  "var_log_common-2020.03.09" : {
    "aliases" : { }
  },
  ".kibana_task_manager_1" : {
    "aliases" : {
      ".kibana_task_manager" : { }
    }
  },
  "var_log_common-2020.03.08" : {
    "aliases" : { }
  },
  ".apm-agent-configuration" : {
    "aliases" : { }
  }
}
```

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [March 9, 2020, 3:58pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/4 "2020-03-09T15:58:47Z")

</div>

So if I understand the logic correctly my if statement is say that I should receive messages from both sources if they are available or?

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2020, 4:03pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/5 "2020-03-09T16:03:25Z")

</div>

What your statement says is... if the value of the `source` field is `/var/log/nginx/access.log` or the value of the `source` field is `/var/log/nginx/error.log` then output the messages to the index `var_log_nginx-%{+YYYY.MM.dd}`, otherwise output the messages to `var_log_common-%{+YYYY.MM.dd}`.

If the `source` field is not being set as you expect, then the messages won't match and will land in `var_log_common-%{+YYYY.MM.dd}`.

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [March 9, 2020, 4:06pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/6 "2020-03-09T16:06:06Z")

</div>

ok then that is correct. Here are the grok filters for my nginx:

```
 match => { "message" => "%{NGINX_ERROR}" } 
 match => { "message" => "%{NGINX_ACCESS}" }

```

Patterns

```
ERRORDATE %{YEAR}/%{MONTHNUM}/%{MONTHDAY} %{TIME}
NGINX_ERROR %{ERRORDATE:timestamp} \[%{LOGLEVEL:loglevel}\] %{INT:process_id}#%{INT:thread_id}: \*(%{INT:connection_id})? %{NOTSPACE:request} %{NOTSPACE:method} %{GREEDYDATA:Error_Message} %{IP:ClientIP}

METHOD (OPTIONS|GET|HEAD|POST|PUT|DELETE|TRACE|CONNECT)
NGINX_ACCESS %{IPORHOST:visitor_ip} - %{USERNAME:remote_user} \[%{HTTPDATE:time_local}\] "%{DATA:server_name}" "%{METHOD:method} %{URIPATHPARAM:path} HTTP/%{NUMBER:http_version}" %{INT:status} %{INT:body_bytes_sent} "%{URI:referer}" %{QS:user_agent}
```

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2020, 4:07pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/7 "2020-03-09T16:07:57Z")

</div>

That grok stuff isn't setting the `source` field. How are you setting `source`?

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [March 9, 2020, 4:11pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/8 "2020-03-09T16:11:04Z")

</div>

Good question how to identify what you want to see?

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2020, 4:13pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/9 "2020-03-09T16:13:37Z")

</div>

If you don't know, you probably aren't setting it. Show your logstash input section.

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [March 9, 2020, 4:19pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/10 "2020-03-09T16:19:09Z")

</div>

```
input {
  beats {
  port => 2561
  ssl => true
  ssl_certificate => "/etc/ssl/certs/mon-1.crt"
  ssl_key => "/etc/ssl/private/mon-1.p8"
  }
}

```

And here is my filebeats.yml

```
filebeat.inputs:
- type: log
  enabled: true
  paths:
    - /var/log/nginx/error.log
    - /var/log/nginx/access.log
    - /var/log/common/admin.log
    
  exclude_files: ['\.gz$']
  multiline.pattern: ^\[
  multiline.negate: true
  multiline.match: after

  #filebeat.config.modules:
  # Glob pattern for configuration loading
  #path: ${path.config}/modules.d/*.yml
  
  logging.level: debug
  logging.to_files: true
  logging.files:
  path: /var/log/filebeat
  name: filebeat
  keepfiles: 7
  permissions: 0644
  
  # Set to true to enable config reloading
  reload.enabled: false

  # Period on which files under path should be checked for changes
  #reload.period: 10s

setup.template.settings:
  index.number_of_shards: 3
  #index.codec: best_compression
  #_source.enabled: false

name: "app2-filebeat"
tags: ["app2", "app"]

output.logstash:
  hosts: ["mon-1:2561"]
  ssl.certificate_authorities: #####
  ssl.certificate: ######
  ssl.key: #####
  ssl.verification_mode: none

```

ignore the ssl stuff I have commented it out for this post.

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2020, 4:32pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/11 "2020-03-09T16:32:39Z")

</div>

This is coming from Filebeat. In filebeat 6.x the name of the field was `source`, as of 7.x the field is named `log.file.path`. Try changing your `if` statement to use `[log][file][path]` instead of `[source]`.

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [March 9, 2020, 4:36pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/12 "2020-03-09T16:36:26Z")

</div>

that was it. Thanks man

---

<div class="post-metadata">

**Author:** ![rcowart](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/rcowart/32/88091_2.png) [@rcowart](https://discuss.elastic.co/u/rcowart)\
**Post date:** [March 9, 2020, 4:42pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/13 "2020-03-09T16:42:07Z")

</div>

Glad to help. I am going to add this Beats-to-Logstash use-case to my list of future "how-to" videos ([https://www.youtube.com/channel/UCivWvTx1DwrWNcDLV58kmOg](https://www.youtube.com/channel/UCivWvTx1DwrWNcDLV58kmOg)). I use this method a lot, but have found there are a lot of questions about it.

---

<div class="post-metadata">

**Author:** ![calanon](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/calanon/32/64004_2.png) [@calanon](https://discuss.elastic.co/u/calanon)\
**Post date:** [March 9, 2020, 4:55pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/14 "2020-03-09T16:55:27Z")

</div>

great I just subscribed, but now thats working none of my filters are working 😉

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [April 6, 2020, 4:59pm UTC](https://discuss.elastic.co/t/i-cannot-see-two-indices-in-kibana/222755/15 "2020-04-06T16:59:01Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.