# Pulling data from a statsd server

**URL:** <https://discuss.elastic.co/t/pulling-data-from-a-statsd-server/284625>\
**Category:** Beats\
**Tags:** metricbeat\
**Created:** [September 20, 2021, 1:30pm UTC](https://discuss.elastic.co/t/pulling-data-from-a-statsd-server/284625 "2021-09-20T13:30:39Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![humartinez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/humartinez/32/46395_2.png) [@humartinez](https://discuss.elastic.co/u/humartinez)\
**Post date:** [September 20, 2021, 1:30pm UTC](https://discuss.elastic.co/t/pulling-data-from-a-statsd-server/284625/1 "2021-09-20T13:30:39Z")

</div>

Hi,  
I need to pull data from a statsd server,whats the best approach to do this? Im trying to import the statsd metrics from a spark jobs using metricbeats but metricbeat is getting the following error

> ...  
> spark-5a9e0fdbde2e44b38a075a6bdbb67ecb\_d  
> river\_LiveListenerBus\_queue\_shared\_numDroppedEvents":{"count":0},"spark-5a9e0fdbde2e44b38a075a6bdbb67ecb\_driver\_LiveListenerBus\_queue\_shared\_size":{"value":0.000000}}}, Private:interface {}(nil), TimeSeries:true}, Flags:0x0, Cache:publisher.EventCache{m:common.MapStr(ni  
> l)}} (status=400): {"type":"mapper\_parsing\_exception","reason":"failed to parse","caused\_by":{"type":"illegal\_argument\_exception","reason":"Limit of total fields [10000] has been exceeded while adding new fields [3621]"}}

Is there a workarround to this? Is there a better way to ingests these metrics?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![warkolm](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/warkolm/32/39224_2.png) [@warkolm](https://discuss.elastic.co/u/warkolm)\
**Post date:** [September 21, 2021, 12:52am UTC](https://discuss.elastic.co/t/pulling-data-from-a-statsd-server/284625/2 "2021-09-21T00:52:44Z")

</div>

TLDR this is managed by [Mapping limit settings | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-settings-limit.html), and it looks like you have already increased it from the default of 1000. Increasing it more isn't a great idea as it causes mapping explosions and inefficiencies.

> [@humartinez](#):
>
> Limit of total fields [10000] has been exceeded while adding new fields

What other data are you pulling in from Metricbeat?  
Can you put data from different sources into different indices?

---

<div class="post-metadata">

**Author:** ![humartinez](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/humartinez/32/46395_2.png) [@humartinez](https://discuss.elastic.co/u/humartinez)\
**Post date:** [September 23, 2021, 7:39am UTC](https://discuss.elastic.co/t/pulling-data-from-a-statsd-server/284625/3 "2021-09-23T07:39:45Z")

</div>

Im trying to pull data from some sparks jobs via statsd. I think that the whole issue is about metrics mappings, without it the ELK stack is trying to index the metrics with the sparkIds as a field thats why the number of records exceds the limits.

Regarding to mappings how do i translate a graphite map like this

```
- match: '*.*.LiveListenerBus.listenerProcessingTime.org.apache.spark.sql.execution.ui.*.*'
  name: Live_Listener_Bus_ExecutionUI
  labels:
    application: $1
    executor_id: $2
    type: $3
    qty: $4

```

Currently Im starting to use a graphite exporter to get de metrics from spark, and translate them to prometheus metrics labels. But I would really like to try Elastic as a metrics collector

Thanks

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [October 21, 2021, 9:40am UTC](https://discuss.elastic.co/t/pulling-data-from-a-statsd-server/284625/4 "2021-10-21T09:40:18Z")

</div>

Closed on October 21, 2021, 9:40am UTC

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.