Different documents types in same index


(Alberto Gonzalez) #1

in 6.0 _type is discontinued, but we can use custom field like "type" to index different types of objects in same index and then filter by that.

Is there any downside to this approach and store different documents types with differents fields in same index? or is better to use a different index per document type?


(Mark Walkom) #2

Yes.

Yes, if they aren't the same shape/sort of data then you will run into problems with sparsity and the efficiency of your mappings.

If they are totally different structures, yes.


(Alberto Gonzalez) #3

but metricbeat stores different structures for process utilization, flesystem ,network, psql etc in same metricbeat- index, doesnt it? why is not a problem there?


(Mark Walkom) #4

Because the values are different but the structure is the same.


(Alberto Gonzalez) #5

in metricbeat structure of a filesystem document is not the same of a process document or network document


(Alberto Gonzalez) #6

as i see metricbeat use the same root fields but then nested fields are totally different. Do you mean that if root fields are the same but not nested fields is fine to store in same index like these 2 docs from vmware module:

doc for virtual machines

{
"@timestamp": "2017-10-12T08:05:34.853Z",
"beat": {
"hostname": "host.example.com",
"name": "host.example.com"
},
"metricset": {
"host": "http://127.0.0.1:35887/sdk",
"module": "vsphere",
"name": "virtualmachine",
"rtt": 115
},
"vsphere": {
"virtualmachine": {
"cpu": {
"used": {
"mhz": 0
}
},
"host": "ha-host",
"memory": {
"free": {
"guest": {
"bytes": 33554432
}
},
"total": {
"guest": {
"bytes": 33554432
}
},
"used": {
"guest": {
"bytes": 0
},
"host": {
"bytes": 0
}
}
},
"name": "ha-host_VM1"
}
}
}

doc for datastores:

{
"@timestamp": "2017-10-12T08:05:34.853Z",
"beat": {
"hostname": "host.example.com",
"name": "host.example.com"
},
"metricset": {
"host": "http://127.0.0.1:36937/sdk",
"module": "vsphere",
"name": "datastore",
"rtt": 115
},
"vsphere": {
"datastore": {
"capacity": {
"free": {
"bytes": 50021150720
},
"total": {
"bytes": 63918878720
},
"used": {
"bytes": 13897728000,
"pct": 21
}
},
"fstype": "local",
"name": "LocalDS_0"
}
}
}


(system) #7

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.