Winlogbeat to same index as logstash

[lschoonaert (Luk Schoonaert)]

Hi there,

I'm trying to get all my logs into 1 index - I have DNS logs currently going into logstash (lumberjack), now when I want to sen windows eventlogs, I'd like them to be index in the logstash-* index. I this possible, as far as I see using winlogbeat you need to create a new index called winlogbeat-*.

The use case is being able to interrogate all data (network/machine) created by sysmon, eventlogs, dns logs, proxy logs). Having a separate index for the wineventlogs makes the correlation impossible.

Any help would be greatly appreciated!
Luk

[theuntergeek (Aaron Mildenstein)]

@lschoonaert

I’m trying to get all my logs into 1 index

This is perhaps not in your best interest. Starting in version 6, Elasticsearch will only allow one document type per index. Separate indices are a better way to go for future compatibility.

[lschoonaert (Luk Schoonaert)]

Hi AAron,

thanks for the heads up - but what's the points of log centralization if you then can'ts search centralized? or will there be mutliple-index search support?

And what exactly do you mean by "one document type"?

Thanks,
Luk

[warkolm (Mark Walkom)]

@lschoonaert

thanks for the heads up - but what’s the points of log centralization if you then can’ts search centralized? or will there be mutliple-index search support?

Elasticsearch, and Kibana, can search across multiple indices out of the box.

@lschoonaert

And what exactly do you mean by “one document type”?

Have a look at Index vs. Type | Elastic Blog

[lschoonaert (Luk Schoonaert)]

when I perform a search I need to select the index I perform the search on, so I don't see how I can query multiple indexes simultaneously.

I.e. I have 2 indexes one for syslog and one for event logs - I search for IP 10.0.0.1 and I want results from both indexes - how do I do that then?

[warkolm (Mark Walkom)]

How are you searching?

[lschoonaert (Luk Schoonaert)]

Hi Mark,
I go to Discover, select the index and then search in the field above it. Maybe I'm missing something really obvious?
L.

[warkolm (Mark Walkom)]

You need to have a common prefix in front of the index, eg commonword- so each index becomes commonword-winlogbeat- etc, and then you can setup an index pattern called commonword-* that will read them all.

That is a bit hard to backfill so it might be easier to setup an alias across things - https://www.elastic.co/guide/en/elasticsearch/reference/5.5/indices-aliases.html#alias-adding

[lschoonaert (Luk Schoonaert)]

Ahhh thank you! I'll try that!
Many thanks for your help
L.