# Elastic 7.0 and Netflow module : no more indices created since upgrade to 7.0 **URL:** \ **Category:** Logstash\ **Created:** [May 9, 2019, 6:08pm UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424 "2019-05-09T18:08:03Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![samia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samia/32/29046_2.png) [@samia](https://discuss.elastic.co/u/samia)\ **Post date:** [May 9, 2019, 6:08pm UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424/1 "2019-05-09T18:08:03Z")
Hi, I had elastic stack 5.6.4 running and netflow module used with logstash. Every thing was working normally. I just upgraded to 7.0 and saw that the Netflow module couldn't work any more : No more new netflow\* indices are created. Here's what I have in the logstash logs. I have a lot of warning of the same type ("Could not index event to Elasticsearch") and sometimes an ERROR ("An unknown error occurred sending a bulk request to Elasticsearch") : (See logs and configuration in the following messages of this thread) ---
**Author:** ![samia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samia/32/29046_2.png) [@samia](https://discuss.elastic.co/u/samia)\ **Post date:** [May 9, 2019, 6:17pm UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424/2 "2019-05-09T18:17:03Z")
Here are the logs of Logstash : //////////////////////////////////// LOGS of Logstash /////////////////// ``` [2019-05-09T19:32:10,759][INFO][logstash.runner] Starting Logstash {"logstash.version"=>"7.0.1"} [2019-05-09T19:32:13,446][INFO][logstash.monitoring.internalpipelinesource] Monitoring License OK [2019-05-09T19:32:13,448][INFO][logstash.monitoring.internalpipelinesource] Validated license for monitoring. Enabling monitoring pipeline. [2019-05-09T19:32:13,532][INFO][logstash.config.modulescommon] Starting the netflow module [2019-05-09T19:32:28,692][INFO][logstash.outputs.elasticsearch] Elasticsearch pool URLs updated {:changes=>{:removed=>[], :added=>[http://localhost:9200/]}} [2019-05-09T19:32:28,715][WARN][logstash.outputs.elasticsearch] Restored connection to ES instance {:url=>"http://localhost:9200/"} [2019-05-09T19:32:28,721][INFO][logstash.outputs.elasticsearch] ES Output version determined {:es_version=>7} [2019-05-09T19:32:30,806][INFO][logstash.javapipeline] Starting pipeline {:pipeline_id=>"module-netflow", "pipeline.workers"=>5, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>625, :thread=>"#"} [2019-05-09T19:32:30,871][INFO][logstash.javapipeline] Pipeline started {"pipeline.id"=>"module-netflow"} [2019-05-09T19:32:31,178][INFO][logstash.inputs.udp] Starting UDP listener {:address=>"0.0.0.0:2055"} [2019-05-09T19:32:31,246][INFO][logstash.agent] Pipelines running {:count=>1, :running_pipelines=>[:"module-netflow"], :non_running_pipelines=>[]} [2019-05-09T19:32:31,383][INFO][logstash.inputs.udp] UDP listener started {:address=>"0.0.0.0:2055", :receive_buffer_bytes=>"212992", :queue_size=>"2000"} [2019-05-09T19:32:31,415][INFO][logstash.config.modulescommon] Starting the netflow module ////// A Lot of warnings of this type : [2019-05-09T19:33:05,342][WARN][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"netflow-2019.05.09", :_type=>"_doc", :routing=>nil}, #], :response=>{"index"=>{"_index"=>"netflow-2019.05.09", "_type"=>"_doc", "_id"=>nil, "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"The [default] mapping cannot be updated on index [netflow-2019.05.09]: defaults mappings are not useful anymore now that indices can have at most one type."}}}} ////// And sometimes an error of this type : [2019-05-09T19:34:58,580][ERROR][logstash.outputs.elasticsearch] An unknown error occurred sending a bulk request to Elasticsearch. We will retry indefinitely {:error_message=>"bignum too big to convert into `long'", :error_class=>"LogStash::Json::GeneratorError", :backtrace=>["/usr/share/logstash/logstash-core/lib/logstash/json.rb:27:in `jruby_dump'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:119:in `block in bulk'", "org/jruby/RubyArray.java:2577:in `map'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:119:in `block in bulk'", "org/jruby/RubyArray.java:1792:in `each'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.2-java/lib/logstash/outputs/elasticsearch/http_client.rb:117:in `bulk'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:286:in `safe_bulk'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:191:in `submit'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:159:in `retrying_submit'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-output-elasticsearch-10.0.2-java/lib/logstash/outputs/elasticsearch/common.rb:38:in `multi_receive'", "org/logstash/config/ir/compiler/OutputStrategyExt.java:118:in `multi_receive'", "org/logstash/config/ir/compiler/AbstractOutputDelegatorExt.java:101:in `multi_receive'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:235:in `block in start_workers'"]} //////// END Excerpt of logs ``` ---
**Author:** ![samia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samia/32/29046_2.png) [@samia](https://discuss.elastic.co/u/samia)\ **Post date:** [May 9, 2019, 6:20pm UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424/3 "2019-05-09T18:20:31Z")
Here's my logstash config file : (Only the uncommented lines) ////////////////// Config file /////// ``` node.name: xxx.xxxx.xxxx path.data: /usr/share/logstash/data_modules/ pipeline.id: modules pipeline.workers: 5 modules: - name: netflow path.logs: /var/log/logstash_modules/ xpack.monitoring.enabled: true xpack.monitoring.elasticsearch.hosts: ["http://localhost:9200"] xpack.monitoring.collection.pipeline.details.enabled: true ``` ///////// ---
**Author:** ![samia](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/samia/32/29046_2.png) [@samia](https://discuss.elastic.co/u/samia)\ **Post date:** [May 9, 2019, 6:33pm UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424/4 "2019-05-09T18:33:37Z")
I have an old template for this kind of indices (netflow\*) called "netflow\_template" and apparently version 7.0 created another one "netflow" : These two templates are used in elasticsearch. ``` ///////// Old netflow template ///////// { "netflow_template" : { "order" : 10, "version" : 60003, "index_patterns" : [ "netflow-*" ], "settings" : { "index" : { "number_of_shards" : "1", "number_of_replicas" : "0" } }, "mappings" : { }, "aliases" : { } } } ////// End first template ``` Second template : ``` { "netflow" : { "order" : 0, "index_patterns" : [ "netflow-*" ], "settings" : { }, "mappings" : { }, "aliases" : { } } } ``` ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [June 6, 2019, 6:33pm UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424/5 "2019-06-06T18:33:54Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed. ---
**Author:** ![joshdover](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joshdover/32/42020_2.png) [@joshdover](https://discuss.elastic.co/u/joshdover)\ **Post date:** [June 25, 2019, 5:10pm UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424/6 "2019-06-25T17:10:53Z")
---
**Author:** ![joshdover](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/joshdover/32/42020_2.png) [@joshdover](https://discuss.elastic.co/u/joshdover)\ **Post date:** [June 25, 2019, 5:13pm UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424/7 "2019-06-25T17:13:36Z")
@samia: Can you check to see if you have any index templates in Elasticsearch for netflow? You can do this by hitting the \_cat API: ```auto GET /_cat/templates ``` If you see any related to netflow, you may want to see if they include any default mappings. You can use the [template API](https://www.elastic.co/guide/en/elasticsearch/reference/7.1/indices-templates.html) to get these: ```auto GET _template/netflow ``` If there are any default mappings, you'll want to remove those by updating the template. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [November 4, 2022, 7:07am UTC](https://discuss.elastic.co/t/elastic-7-0-and-netflow-module-no-more-indices-created-since-upgrade-to-7-0/180424/8 "2022-11-04T07:07:16Z")