Dashboard Navigation [Filebeat Netflow] not working

Elastic Stack Kibana
1

Hello,

I recently upgraded our Elastic Stack from 6.8 to 7.9.1 and I am unable to get several Dashboards to work in Kibana. Specifically, the [Filebeat Netflow] Overview dashboard. Whenever I select it in Kibana I receive the following error, “Could not locate that index-pattern (id: filebeat-), click here to re-create it.*” I confirmed there’s an index pattern in Kibana with that name. Also, I’ve tried the steps in https://www.elastic.co/guide/en/beats/filebeat/current/could-not-locate-index-pattern.html but I am still getting the error. Any suggestions? Thanks!

2

Hello @HelpComputer,

Thanks for trying it out.
As it's a small gap between 6.8 and 7.9.1, have you tried recreating an index pattern?

3

Thanks for the reply. I just now deleted the index pattern and tried to create a new one but when I do my index pattern name doesn't pick up the 7.9 indexes just version 6. The index pattern name I am trying is: "filebeat-*" Any thoughts on why that might be happening? I've already confirmed there are 7.9 indexes under the Index Management page.

My index names

  • filebeat-6.8.1-2020.07.10
  • filebeat-7.9.1-2020.09.17
4

What about ticking the indices in the checkboxes, clicking on "manage indices", and select "refresh indices"?

If you click on title field of the index in index management page, is there any further information given?

5

Refreshed all the filebeat-* indices and tried again with the Kibana Index pattern but I still don't see the filebeat-7* indices in the list that it finds. Should it list every single indices it finds or just a partial list?

Here are the settings for the two indices. Hopefully, they will help.

filebeat-7.9.1-2020.09.17

{
  "settings": {
    "index": {
      "number_of_shards": "1",
      "provided_name": "filebeat-7.9.1-2020.09.17",
      "creation_date": "1600374857006",
      "priority": "100",
      "number_of_replicas": "1",
      "uuid": "ICql4fgiR4WWt06rM8qtmQ",
      "version": {
        "created": "7090199"
      }
    }
  },
  "defaults": {
    "index": {
      "flush_after_merge": "512mb",
      "final_pipeline": "_none",
      "max_inner_result_window": "100",
      "unassigned": {
        "node_left": {
          "delayed_timeout": "1m"
        }
      },
      "max_terms_count": "65536",
      "lifecycle": {
        "name": "",
        "parse_origination_date": "false",
        "indexing_complete": "false",
        "rollover_alias": "",
        "origination_date": "-1"
      },
      "routing_partition_size": "1",
      "force_memory_term_dictionary": "false",
      "max_docvalue_fields_search": "100",
      "merge": {
        "scheduler": {
          "max_thread_count": "2",
          "auto_throttle": "true",
          "max_merge_count": "7"
        },
        "policy": {
          "reclaim_deletes_weight": "2.0",
          "floor_segment": "2mb",
          "max_merge_at_once_explicit": "30",
          "max_merge_at_once": "10",
          "max_merged_segment": "5gb",
          "expunge_deletes_allowed": "10.0",
          "segments_per_tier": "10.0",
          "deletes_pct_allowed": "33.0"
        }
      },
      "max_refresh_listeners": "1000",
      "max_regex_length": "1000",
      "load_fixed_bitset_filters_eagerly": "true",
      "number_of_routing_shards": "1",
      "write": {
        "wait_for_active_shards": "1"
      },
      "verified_before_close": "false",
      "mapping": {
        "coerce": "false",
        "nested_fields": {
          "limit": "50"
        },
        "depth": {
          "limit": "20"
        },
        "field_name_length": {
          "limit": "9223372036854775807"
        },
        "total_fields": {
          "limit": "1000"
        },
        "nested_objects": {
          "limit": "10000"
        },
        "ignore_malformed": "false"
      },
      "source_only": "false",
      "soft_deletes": {
        "enabled": "false",
        "retention": {
          "operations": "0"
        },
        "retention_lease": {
          "period": "12h"
        }
      },
      "max_script_fields": "32",
      "query": {
        "default_field": [
          "*"
        ],
        "parse": {
          "allow_unmapped_fields": "true"
        }
      },
      "format": "0",
      "frozen": "false",
      "sort": {
        "missing": [],
        "mode": [],
        "field": [],
        "order": []
      },
      "codec": "default",
      "max_rescore_window": "10000",
      "max_adjacency_matrix_filters": "100",
      "analyze": {
        "max_token_count": "10000"
      },
      "gc_deletes": "60s",
      "top_metrics_max_size": "10",
      "optimize_auto_generated_id": "true",
      "max_ngram_diff": "1",
      "hidden": "false",
      "translog": {
        "generation_threshold_size": "64mb",
        "flush_threshold_size": "512mb",
        "sync_interval": "5s",
        "retention": {
          "size": "512MB",
          "age": "12h"
        },
        "durability": "REQUEST"
      },
      "auto_expand_replicas": "false",
      "mapper": {
        "dynamic": "true"
      },
      "recovery": {
        "type": ""
      },
      "requests": {
        "cache": {
          "enable": "true"
        }
      },
      "data_path": "",
      "highlight": {
        "max_analyzed_offset": "1000000"
      },
      "routing": {
        "rebalance": {
          "enable": "all"
        },
        "allocation": {
          "enable": "all",
          "total_shards_per_node": "-1"
        }
      },
      "search": {
        "slowlog": {
          "level": "TRACE",
          "threshold": {
            "fetch": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            },
            "query": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            }
          }
        },
        "idle": {
          "after": "30s"
        },
        "throttled": "false"
      },
      "fielddata": {
        "cache": "node"
      },
      "default_pipeline": "_none",
      "max_slices_per_scroll": "1024",
      "shard": {
        "check_on_startup": "false"
      },
      "xpack": {
        "watcher": {
          "template": {
            "version": ""
          }
        },
        "version": "",
        "ccr": {
          "following_index": "false"
        }
      },
      "percolator": {
        "map_unmapped_fields_as_text": "false"
      },
      "allocation": {
        "max_retries": "5",
        "existing_shards_allocator": "gateway_allocator"
      },
      "refresh_interval": "1s",
      "indexing": {
        "slowlog": {
          "reformat": "true",
          "threshold": {
            "index": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            }
          },
          "source": "1000",
          "level": "TRACE"
        }
      },
      "compound_format": "0.1",
      "blocks": {
        "metadata": "false",
        "read": "false",
        "read_only_allow_delete": "false",
        "read_only": "false",
        "write": "false"
      },
      "max_result_window": "10000",
      "store": {
        "stats_refresh_interval": "10s",
        "type": "",
        "fs": {
          "fs_lock": "native"
        },
        "preload": []
      },
      "queries": {
        "cache": {
          "enabled": "true"
        }
      },
      "warmer": {
        "enabled": "true"
      },
      "max_shingle_diff": "3",
      "query_string": {
        "lenient": "false"
      }
    }
  }
}

filebeat-6.8.1-2020.07.10

{
  "settings": {
    "index": {
      "lifecycle": {
        "rollover_alias": "filebeat"
      },
      "number_of_shards": "5",
      "provided_name": "filebeat-6.8.1-2020.07.10",
      "creation_date": "1594339208616",
      "number_of_replicas": "1",
      "uuid": "fIfaQ7kUQYiyP-s6SOI0Lw",
      "version": {
        "created": "6070299",
        "upgraded": "7090199"
      }
    }
  },
  "defaults": {
    "index": {
      "flush_after_merge": "512mb",
      "final_pipeline": "_none",
      "max_inner_result_window": "100",
      "unassigned": {
        "node_left": {
          "delayed_timeout": "1m"
        }
      },
      "max_terms_count": "65536",
      "lifecycle": {
        "name": "",
        "parse_origination_date": "false",
        "indexing_complete": "false",
        "origination_date": "-1"
      },
      "routing_partition_size": "1",
      "force_memory_term_dictionary": "false",
      "max_docvalue_fields_search": "100",
      "merge": {
        "scheduler": {
          "max_thread_count": "2",
          "auto_throttle": "true",
          "max_merge_count": "7"
        },
        "policy": {
          "reclaim_deletes_weight": "2.0",
          "floor_segment": "2mb",
          "max_merge_at_once_explicit": "30",
          "max_merge_at_once": "10",
          "max_merged_segment": "5gb",
          "expunge_deletes_allowed": "10.0",
          "segments_per_tier": "10.0",
          "deletes_pct_allowed": "33.0"
        }
      },
      "max_refresh_listeners": "1000",
      "max_regex_length": "1000",
      "load_fixed_bitset_filters_eagerly": "true",
      "number_of_routing_shards": "1",
      "write": {
        "wait_for_active_shards": "1"
      },
      "verified_before_close": "false",
      "mapping": {
        "coerce": "false",
        "nested_fields": {
          "limit": "50"
        },
        "depth": {
          "limit": "20"
        },
        "field_name_length": {
          "limit": "9223372036854775807"
        },
        "total_fields": {
          "limit": "1000"
        },
        "nested_objects": {
          "limit": "10000"
        },
        "ignore_malformed": "false"
      },
      "source_only": "false",
      "soft_deletes": {
        "enabled": "false",
        "retention": {
          "operations": "0"
        },
        "retention_lease": {
          "period": "12h"
        }
      },
      "max_script_fields": "32",
      "query": {
        "default_field": [
          "*"
        ],
        "parse": {
          "allow_unmapped_fields": "true"
        }
      },
      "format": "0",
      "frozen": "false",
      "sort": {
        "missing": [],
        "mode": [],
        "field": [],
        "order": []
      },
      "priority": "1",
      "codec": "default",
      "max_rescore_window": "10000",
      "max_adjacency_matrix_filters": "100",
      "analyze": {
        "max_token_count": "10000"
      },
      "gc_deletes": "60s",
      "top_metrics_max_size": "10",
      "optimize_auto_generated_id": "true",
      "max_ngram_diff": "1",
      "hidden": "false",
      "translog": {
        "generation_threshold_size": "64mb",
        "flush_threshold_size": "512mb",
        "sync_interval": "5s",
        "retention": {
          "size": "512MB",
          "age": "12h"
        },
        "durability": "REQUEST"
      },
      "auto_expand_replicas": "false",
      "mapper": {
        "dynamic": "true"
      },
      "recovery": {
        "type": ""
      },
      "requests": {
        "cache": {
          "enable": "true"
        }
      },
      "data_path": "",
      "highlight": {
        "max_analyzed_offset": "1000000"
      },
      "routing": {
        "rebalance": {
          "enable": "all"
        },
        "allocation": {
          "enable": "all",
          "total_shards_per_node": "-1"
        }
      },
      "search": {
        "slowlog": {
          "level": "TRACE",
          "threshold": {
            "fetch": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            },
            "query": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            }
          }
        },
        "idle": {
          "after": "30s"
        },
        "throttled": "false"
      },
      "fielddata": {
        "cache": "node"
      },
      "default_pipeline": "_none",
      "max_slices_per_scroll": "1024",
      "shard": {
        "check_on_startup": "false"
      },
      "xpack": {
        "watcher": {
          "template": {
            "version": ""
          }
        },
        "version": "",
        "ccr": {
          "following_index": "false"
        }
      },
      "percolator": {
        "map_unmapped_fields_as_text": "false"
      },
      "allocation": {
        "max_retries": "5",
        "existing_shards_allocator": "gateway_allocator"
      },
      "refresh_interval": "1s",
      "indexing": {
        "slowlog": {
          "reformat": "true",
          "threshold": {
            "index": {
              "warn": "-1",
              "trace": "-1",
              "debug": "-1",
              "info": "-1"
            }
          },
          "source": "1000",
          "level": "TRACE"
        }
      },
      "compound_format": "0.1",
      "blocks": {
        "metadata": "false",
        "read": "false",
        "read_only_allow_delete": "false",
        "read_only": "false",
        "write": "false"
      },
      "max_result_window": "10000",
      "store": {
        "stats_refresh_interval": "10s",
        "type": "",
        "fs": {
          "fs_lock": "native"
        },
        "preload": []
      },
      "queries": {
        "cache": {
          "enabled": "true"
        }
      },
      "warmer": {
        "enabled": "true"
      },
      "max_shingle_diff": "3",
      "query_string": {
        "lenient": "false"
      }
    }
  }
}
6

Hello @HelpComputer,

Could you please check if your data is sending via the latest filebeat agent?
If not, please download it from here: https://www.elastic.co/downloads/beats/filebeat
and see if the index pattern appears after setting it up.

7

Hi @AngelaChuang
Yes, just confirmed it's the latest version (7.9.1-1). I went ahead and created the Kibana Index Pattern using "filebeat-" just to see what would happen. I do see a conflict when reviewing the filebeat- index. Below is the message that is shown. Would this cause the issue?

Mapping conflict

A field is defined as several types (string, integer, etc) across the indices that match this pattern. You may still be able to use these conflict fields in parts of Kibana, but they will be unavailable for functions that require Kibana to know their type. Correcting this issue will require reindexing your data.
8

What about reindexing data as it suggested in the message?

9

I've never reindexed before, is this the simplest method? - https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html ?

10

Sounds like you are on the right path. If you delete the filebeat-* index pattern then run filebeat setup --dashboards -e this will recreate the index pattern using the fields from that version of filebeat.

If you have old data whose types conflict then you'll see this in Kibana when you view the index pattern. If you care to migrate the old data to the new schema then you can reindex. If you filter on the conflict fields then if you click on an individual field it will show you what indices contain the conflict. This can help guide your reindexing efforts.

index-pattern-conflicts
index-pattern-conflicts1568×852 90.2 KB

index-pattern-conflicts-indices
index-pattern-conflicts-indices1106×530 31.9 KB

From 6.x to 7.0 there were breaking changes in the field names. https://www.elastic.co/guide/en/beats/libbeat/7.9/breaking-changes-7.0.html

1 Like
11

Hi @andrewkroh, thanks for the suggestions. I tried re-indexing the indices but it gave errors for most of them. I ended up just deleting the old filebeat-6* indices. Ran "filebeat setup --dashboards" afterward from the latest filebeat agent and the Kibana pattern saw the 7.9.1 indices and gave no warnings/errors about conflicts. I did this all before seeing your post. Should I still run filebeat setup --dashboards -e?

The Filebeat Netflow dashboard is still not loading correctly. It no longer complains about not being able to find the index-pattern. Now it throws warning similar to this, Saved "field" parameter is now invalid. Please select a new field. All of the indices are from filebeat 7.9.1 agents. Any thoughts on what I might try? Thanks!

12

You don't need to run it again with the -e (that's just for adding more logging).

This sounds like a bug with the dashboard. It might be referencing any old field. Which netflow dashboard has the problem (IIRC there are several for netflow)?

13

@andrewkroh Thanks for the heads up about the netflow dashboard having some issues. I did get some of visualizations to show up after updating the filebeat index template using the command below. I forgot to do that after the 7.9 upgrade. However several are still not working. The Geo Location tab from the Netflow Overview shows the world map but the heat map data is not working. I may just need to wait for an update.

filebeat setup --index-management
14

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.