Issue with the fileds.env value

Elastic Stack Beats
1

Hi

I have been installing beats by categorizing then with a field value.

I used integer value on all my beats. Now when I am trying to use a string value I see errors.
fields:
env: 10.6

Can I not use multiple data types for fields.env value.
fields:
env: Stage

|2019-04-03T06:43:14.423-0700|WARN|elasticsearch/client.go:539|Cannot index event publisher.Event{Content:beat.Event{Timestamp:time.Time{wall:0xbf214bfb96b57b90, ext:869286001, loc:(*time.Location)(0x2b2ed00)}, Meta:common.MapStr(nil), Fields:common.MapStr{"fields":common.MapStr{"env":"systems"}, "beat":common.MapStr{"name":"88888", "hostname":"8888888", "version":"6.7.0"}, "host":common.MapStr{"name":"888888", "architecture":"x86_64", "os":common.MapStr{"build":"9600.18684", "platform":"windows", "version":"6.3", "family":"windows", "name":"Windows Server 2012 R2 Standard"}, "id":"ce032784-436a-4022-aeff-5feb9ac6015d"}, "event":common.MapStr{"dataset":"system.memory"}, "system":common.MapStr{"memory":common.MapStr{"swap":common.MapStr{"total":0x22bf8d000, "used":common.MapStr{"bytes":0xf37ab000, "pct":0.4379}, "free":0x1387e2000}, "total":0x1dff8d000, "used":common.MapStr{"bytes":0xf4067000, "pct":0.5084}, "free":0xebf26000, "actual":common.MapStr{"free":0xebf26000, "used":common.MapStr{"pct":0.5084, "bytes":0xf4067000}}}}, "metricset":common.MapStr{"name":"memory", "module":"system"}}, Private:interface {}(nil)}, Flags:0x0} (status=400): {"type":"mapper_parsing_exception","reason":"failed to parse field [fields.env] of type [float]","caused_by":{"type":"number_format_exception","reason":"For input string: \"systems\""}}|

2

Hi,

I don't think it is possible. In Elasticsearch all fields are mapped to a type(like text, long, float,...) and the type is validated when inserting new data.

What you could do: separe the fields in the beats config:

fields:
  env: stage
  version: 10.6

This creates two fields in elasticsearch (fields.env, fields.version) which can have different types.

Best regards
Wolfram

3

@Wolfram_Haussig

Is there a way I can add a number to field. env like 10.6 and define it as a string and not an integer. Or by default beats considers this as an integer?

4

@Wolfram_Haussig
Is there a way I can add a number to field. env like 10.6 and define it as a string and not an integer. Or by default beats considers this as an integer?

5

Hi,

By default Elasticsearch recognises the type of a field on the first occurence in a log entry which would be float I guess(see supported types here: Data Types | Elasticsearch Guide [8.11] | Elastic).
If you want 10.6 as a string the following should do:

fields:
  env: stage
  version: "10.6"

Another possibility(More powerful but bit more complex) would be to create a custom index template: Index templates | Elasticsearch Guide [8.11] | Elastic

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.