Want to extract data from a file path, i have the path in mysql ? possible in logstash?

Elastic Stack Logstash
1

Hi,
I am very new to logstah. my requirement is read the log file path from mysql field and extract file for elasticseacrh.

something like this:
Mysql database:testDB->fileTable->fileLocation :"/path/to/the/file.log"

I don't know if logstash will be able to do this>?
input { read the mysql fileLocation content using the input-jdbc-plugin}
file{ path => pass/the/fileLocation } //this is another input but the path is coming from the above input.
output to { Elasticsearch }

Thanks in advance
-neo

2

If the file whose path you get from the database is small and you're able to read the whole file in one swoop (rather than tail the file over a long time like the file input is meant to be used) you might be able to get away with a ruby filter or a small custom plugin, but it sounds like you're interested in a database driven Logstash configuration. That's not supported.

3

Thanks.
I have a small file like 200kb - 1000kb and its a base64 encoded file, so we have a file
Ex. test.file which contains base64 .

path from database will /document/datasets_file/test.file

Please let me know what rugby filter to be used here..

4

I can only point you in the right direction; I'm not going to write the Ruby code for you.