Logstash configuration file to reference variables found in another file

[CDR (Colton)]

Hello,

I am working on a project that will have logstash ingest data from many different servers. All of the servers will have a unique IP address and will be present on each log message. All of the servers have special tasks and I want to have metadata for each log message stating what server the log message does and its role.
Example:

Server: 127.0.0.1
Name: Master
Purpose: Production
Other: some other data

Server: 127.0.0.2
Name: Server Uno
Purpose: Backup
Other: some other data

Server: 127.0.0.3
Name: Another one
Purpose: Redundant
Other: some other data

...maybe more...

Basically, I have the IP address and can write if statements to determine what to add to the new fields. My problem is that I do not want people to have to go in and manually change the logstash configuration file as they could mess up other settings. Ideally it would be nice for them to go to another dedicated file, maybe an Excel spreadsheet, and fill in the fields necessary. Logstash would then read this file in first and 'know' what the IP addresses meta data fields were.

Is this possible? Is there a better way?
If this is possible, how would I do it?
If this is possible, can we make the # of servers dynamic?

Let me know if this is not clear.

Thanks!

[magnusbaeck (Magnus Bäck)]

Have a look at the translate and jdbc_streaming filters.

[CDR (Colton)]

Thanks magnus. The translate filter seems to be more of what I am looking for and using the csv input. From the documentation it looks like it does a replacement based on if the string/integer is exactly the same as the one given in the first column. What I want to do is match the IP address and then add fields, more than one, based on if the IP address was this or that. Am I misunderstanding the functionality of the translate filter?

[magnusbaeck (Magnus Bäck)]

No, I think you've got it. The translate filter doesn't directly support adding multiple fields, but you achieve that anyway by letting the added field value be a JSON string that you feed to a json filter.