Different documents types in same index

[agonzalez (Alberto)]

in 6.0 _type is discontinued, but we can use custom field like "type" to index different types of objects in same index and then filter by that.

Is there any downside to this approach and store different documents types with differents fields in same index? or is better to use a different index per document type?

[warkolm (Mark Walkom)]

@agonzalez

in 6.0 _type is discontinued, but we can use custom field like "type" to index different types of objects in same index and then filter by that.

Yes.

@agonzalez

Is there any downside to this approach and store different documents types with differents fields in same index?

Yes, if they aren't the same shape/sort of data then you will run into problems with sparsity and the efficiency of your mappings.

@agonzalez

or is better to use a different index per document type?

If they are totally different structures, yes.

[agonzalez (Alberto)]

but metricbeat stores different structures for process utilization, flesystem ,network, psql etc in same metricbeat- index, doesnt it? why is not a problem there?

[warkolm (Mark Walkom)]

Because the values are different but the structure is the same.

[agonzalez (Alberto)]

in metricbeat structure of a filesystem document is not the same of a process document or network document

[agonzalez (Alberto)]

as i see metricbeat use the same root fields but then nested fields are totally different. Do you mean that if root fields are the same but not nested fields is fine to store in same index like these 2 docs from vmware module:

doc for virtual machines

{
"@timestamp": "2017-10-12T08:05:34.853Z",
"beat": {
"hostname": "host.example.com",
"name": "host.example.com"
},
"metricset": {
"host": "http://127.0.0.1:35887/sdk",
"module": "vsphere",
"name": "virtualmachine",
"rtt": 115
},
"vsphere": {
"virtualmachine": {
"cpu": {
"used": {
"mhz": 0
}
},
"host": "ha-host",
"memory": {
"free": {
"guest": {
"bytes": 33554432
}
},
"total": {
"guest": {
"bytes": 33554432
}
},
"used": {
"guest": {
"bytes": 0
},
"host": {
"bytes": 0
}
}
},
"name": "ha-host_VM1"
}
}
}

doc for datastores:

{
"@timestamp": "2017-10-12T08:05:34.853Z",
"beat": {
"hostname": "host.example.com",
"name": "host.example.com"
},
"metricset": {
"host": "http://127.0.0.1:36937/sdk",
"module": "vsphere",
"name": "datastore",
"rtt": 115
},
"vsphere": {
"datastore": {
"capacity": {
"free": {
"bytes": 50021150720
},
"total": {
"bytes": 63918878720
},
"used": {
"bytes": 13897728000,
"pct": 21
}
},
"fstype": "local",
"name": "LocalDS_0"
}
}
}