# Metricbeat - Limit of total fields \[1000\] has been exceeded **URL:** \ **Category:** Beats\ **Tags:** metricbeat\ **Created:** [October 28, 2022, 5:43am UTC](https://discuss.elastic.co/t/metricbeat-limit-of-total-fields-1000-has-been-exceeded/317660 "2022-10-28T05:43:53Z")\ **Posts on this page:** 3\ **Page:** 1
**Author:** ![parthmaniar](https://avatars.discourse-cdn.com/v4/letter/p/71e660/32.png) [@parthmaniar](https://discuss.elastic.co/u/parthmaniar)\ **Post date:** [October 28, 2022, 5:43am UTC](https://discuss.elastic.co/t/metricbeat-limit-of-total-fields-1000-has-been-exceeded/317660/1 "2022-10-28T05:43:53Z")
Hello, I hope my question finds you and your loved ones safe and healthy. I am having errors trying to ingest **system metrics** (system module) from a single Ubuntu system running 22.04.1 using **Metricbeat (version 7.17.7 and 8.4.3)** via **Logstash (7.17.7)**. The logs are ingested via a common beats pipeline running having the following configuration: ```auto input { beats { port => 5074 #id => "beats_ingest" } } output { elasticsearch { hosts => ["https://192.168.0.235:9200","https://192.168.0.236:9200"] ssl => true user => 'redacted' password => 'redacted' cacert => '/etc/logstash/elasticsearch-ca.pem' ssl_certificate_verification => false index => "%{[@metadata][beat]}-%{+YYYY.MM}" ilm_enabled => auto ilm_rollover_alias => "%{[@metadata][beat]}" } } ``` The error I get is: `Limit of total fields [1000] has been exceeded while adding new fields` I have applied the following setting from: 1. [What does "Limit of total fields [1000] in index has been exceeded" means in Elasticsearch - Stack Overflow](https://stackoverflow.com/questions/55372330/what-does-limit-of-total-fields-1000-in-index-has-been-exceeded-means-in) 2. [Approaches to deal with "Limit of total fields [1000] in index has been exceeded"](https://discuss.elastic.co/t/approaches-to-deal-with-limit-of-total-fields-1000-in-index-has-been-exceeded/241039) I've applied the following: ```auto PUT metricbeat/_settings { "index.mapping.total_fields.limit": 2000 } ``` and also ```auto PUT metricbeat*/_settings { "index.mapping.total_fields.limit": 2000 } ``` However the error is persistent. **I am confused as to why Metricbeat running default configuration to capture system metrics is giving this error** Error log on logstash and it is odd that I see fields such as ios while the system is `[2022-10-27T23:20:39,929][WARN][logstash.outputs.elasticsearch][beats_ingest][71731faf1263f2b407ad56129215210a4b6a97a861ce28f6c83821a199a44315] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"metricbeat", :routing=>nil}, {"host"=>{"hostname"=>"inmum-x-pslp01", "mac"=>["00:50:56:ae:6c:e9"], "os"=>{"codename"=>"jammy", "version"=>"22.04.1 LTS (Jammy Jellyfish)", "type"=>"linux", "name"=>"Ubuntu", "kernel"=>"5.15.0-52-generic", "family"=>"debian", "platform"=>"ubuntu"}, "id"=>"9b066925c16c49b99a4aa4e287d27c85", "architecture"=>"x86_64", "ip"=>["192.168.0.50", "fe80::250:56ff:feae:6ce9"], "containerized"=>false, "name"=>"inmum-x-pslp01"}, "user"=>{"name"=>"root"}, "ecs"=>{"version"=>"8.0.0"}, "@timestamp"=>2022-10-27T23:20:38.803Z, "system"=>{"process"=>{"cgroup"=>{"path"=>"/system.slice/ElasticEndpoint.service", "io"=>{"stats"=>{"dm-1"=>{"discarded"=>{"bytes"=>0, "ios"=>0}, "read"=>{"bytes"=>78123008, "ios"=>1305}, "write"=>{"bytes"=>2068480, "ios"=>114}}, "loop7"=>{"discarded"=>{"bytes"=>0, "ios"=>0}, "read"=>{"bytes"=>7168, "ios"=>2}, "write"=>{"bytes"=>0, "ios"=>0}}, "loop14"=>{"discarded"=>{"bytes"=>0, "ios"=>0}, "read"=>{"bytes"=>4319232, "ios"=>44}, "write"=>{"bytes"=>0, "ios"=>0}}, "sda"=>{"discarded"=>{"bytes"=>0, "ios"=>0}, "read"=>{"bytes"=>78645760, "ios"=>2326}, "write"=>{"bytes"=>0, "ios"=>0}}, "dm-0"=>{"discarded"=>{"bytes"=>0, "ios"=>0}, "read"=>{"bytes"=>78123008, "ios"=>1305}, "write"=>{"bytes"=>2068480, "ios"=>114}}}, "path"=>"/system.slice/ElasticEndpoint.service", "pressure"=>{"some"=>{"total"=>255628, "300"=>{"pct"=>0}, "60"=>{"pct"=>0}, "10"=>{"pct"=>0}}, "full"=>{"total"=>252043, "300"=>{"pct"=>0}, "60"=>{"pct"=>0}, "10"=>{"pct"=>0}}}, "id"=>"ElasticEndpoint.service"}, "cpu"=>{"pressure"=>{"some"=>{"total"=>9621535, "300"=>{"pct"=>0.25}, "60"=>{"pct"=>0.03}, "10"=>{"pct"=>0.02}}, "full"=>{"total"=>9030204, "300"=>{"pct"=>0.21}, "60"=>{"pct"=>0.01}, "10"=>{"pct"=>0}}}, "path"=>"/system.slice/ElasticEndpoint.service", "stats"=>{"periods"=>0, "user"=>{"pct"=>0, "norm"=>{"pct"=>0}, "ns"=>10303865}, "system"=>{"pct"=>0, "norm"=>{"pct"=>0}, "ns"=>6652095}, "throttled"=>{"periods"=>0, "us"=>0}, "usage"=>{"pct"=>0, "norm"=>{"pct"=>0}, "ns"=>16955960}}, "id"=>"ElasticEndpoint.service"}, "id"=>"ElasticEndpoint.service", "memory"=>{"stats"=>{"page_faults"=>32150, "page_steal"=>0, "file_thp"=>{"bytes"=>0}, "inactive_anon"=>{"bytes"=>52658176}, "workingset_activate_anon"=>0, "swap_cached"=>{"bytes"=>0}, "page_refill"=>0, "slab_reclaimable"=>{"bytes"=>3350560}, "kernel_stack"=>{"bytes"=>376832}, "page_lazy_freed"=>0, "workingset_restore_anon"=>0, "file"=>{"bytes"=>43384832}, "slab"=>{"bytes"=>3579568}, "major_page_faults"=>72, "inactive_file"=>{"bytes"=>36892672}, "workingset_node_reclaim"=>0, "active_file"=>{"bytes"=>6492160}, "file_writeback"=>{"bytes"=>0}, "anon"=>{"bytes"=>52662272}, "workingset_restore_file"=>0, "page_lazy_free"=>0, "unevictable"=>{"bytes"=>0}, "workingset_refault_file"=>0, "file_mapped"=>{"bytes"=>8216576}, "active_anon"=>{"bytes"=>4096}, "workingset_refault_anon"=>0, "page_activate"=>1794, "anon_thp"=>{"bytes"=>0}, "page_scan"=>0, "page_deactivate"=>0, "shmem_thp"=>{"bytes"=>0}, "per_cpu"=>{"bytes"=>0}, "workingset_activate_file"=>0, "thp_fault_alloc"=>0, "shmem"=>{"bytes"=>0}, "file_dirty"=>{"bytes"=>16384}, "sock"=>{"bytes"=>0}, "page_tables"=>{"bytes"=>335872}, "htp_collapse_alloc"=>0, "slab_unreclaimable"=>{"bytes"=>229008}}, "mem"=>{"low"=>{"bytes"=>0}, "events"=>{"high"=>0, "max"=>0, "low"=>0, "oom"=>0, "oom_kill"=>0}, "usage"=>{"bytes"=>100360192}}, "path"=>"/system.slice/ElasticEndpoint.service", "memsw"=>{"events"=>{"high"=>0, "max"=>0, "fail"=>0}, "low"=>{"bytes"=>0}, "usage"=>{"bytes"=>0}}, "id"=>"ElasticEndpoint.service"}, "cgroups_version"=>2}, "cpu"=>{"total"=>{"pct"=>0.006, "norm"=>{"pct"=>0.006}, "value"=>16950}, "start_time"=>"2022-10-27T23:06:45.000Z"}, "memory"=>{"size"=>764297216, "share"=>12304384, "rss"=>{"bytes"=>63598592, "pct"=>0.0155}}, "state"=>"sleeping", "fd"=>{"open"=>121, "limit"=>{"soft"=>8192, "hard"=>8192}}, "cmdline"=>"/opt/Elastic/Endpoint/elastic-endpoint run"}}, "service"=>{"type"=>"system"}, "tags"=>["beats_input_raw_event"], "process"=>{"pid"=>932, "memory"=>{"pct"=>0.0155}, "executable"=>"/opt/Elastic/Endpoint/elastic-endpoint", "name"=>"elastic-endpoin", "state"=>"sleeping", "command_line"=>"/opt/Elastic/Endpoint/elastic-endpoint run", "pgid"=>932, "args"=>["/opt/Elastic/Endpoint/elastic-endpoint", "run"], "cpu"=>{"start_time"=>"2022-10-27T23:06:45.000Z", "pct"=>0.006}, "working_directory"=>"/", "parent"=>{"pid"=>1}}, "agent"=>{"version"=>"8.4.3", "type"=>"metricbeat", "name"=>"inmum-x-pslp01", "ephemeral_id"=>"1d78db85-6e26-4c81-9e37-de822c322216", "id"=>"81759243-2e2f-4bd8-bc61-83ff4b43b543"}, "@version"=>"1", "metricset"=>{"name"=>"process", "period"=>10000}, "event"=>{"dataset"=>"system.process", "duration"=>115778553, "module"=>"system"}}], :response=>{"index"=>{"_index"=>"metricbeat", "_id"=>"hHy_G4QBBPNtLXwjabQS", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [1000] has been exceeded while adding new fields [2]"}}}}}` ---
**Author:** ![Mario\_Castro](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/mario_castro/32/35107_2.png) [@Mario\_Castro](https://discuss.elastic.co/u/Mario_Castro)\ **Post date:** [November 8, 2022, 2:44pm UTC](https://discuss.elastic.co/t/metricbeat-limit-of-total-fields-1000-has-been-exceeded/317660/2 "2022-11-08T14:44:30Z")
This is (possibly) because you are ingesting into an index which has already a lot of indexed fields. After applying the new index limit config, did you restart elasticsearch? Be aware, Metricbeat creates a new index every day in a form similar to `metricbeat-{year}-{month}-{date}` System modules is configured, by default, so that it's not indexing more tan a hunded or a couple of hundred fields. Can you share your Metricbeat configuration too? Is it possible that you have much more data in that index? Is it possible to share one of the system events you are generating using logstash? Overall, I'd focus in why you are generating so many fields because it seems like an unexpected output from logstash, generating many more fields. ---
**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\ **Post date:** [December 6, 2022, 2:45pm UTC](https://discuss.elastic.co/t/metricbeat-limit-of-total-fields-1000-has-been-exceeded/317660/3 "2022-12-06T14:45:18Z")
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.