I am analyzing syslogs with Elastic. They all arrive on a single port and this can't be changed. However, the kind of syslog messages are very diverse, lets call it 4 categories: dns, dhcp, switch, httpd
Each of these categories has several specific search fields, lets say 30 specific search fields.
My question is:
- does it make sense to create 4x 30 = 120 search fields in a single index, while each message only uses 1/4 of these fields (it is only 1 of the 4 categories) ? In other words, does an "empty" field take up space in an index ?
- wouldn't it be better to create 4 seperate indexes, each with only the relevant fields ?
a) would i still be possible to link the indexes together or do queries across the indexes ?
for example, link a dns.ip field in dns index with a httpd.ip field in httpd index ?
b) is this even possible in logstash for example ?
take a single input stream and split it into 4 different output streams to 4 different indexes with different fields ? Or do i need 4 input streams also in this case ?