Sending logs from ibm i to elsatic

Elastic Stack Elasticsearch
1

hello guys,
currently im sending logs of jobs from the ibm i system (as400) to elastic by converting the logs to txt files and using curl to send them,
curl is part of pase environment witch is unix commands.

as i have large logs, i want to compress them to zip files and then to base64 and then send it via curl.

have you guys any suggestion how to use curl with base64 file, the syntax.
and maybe a suggestion to compress the text file to base64 zip in one step?
i can use jar file to do so or even python. but the system isnt connected to the internet for online import.

i can download jar/java script and such from a pc before hand.

thank you in advance

2

Hi @itayber,

Welcome! How are you sending your logs at the moment? Are you using Filebeat or Logstash for example?

Having a quick look at GitHub, I see that for Filebeat there is a feature request for processing zipped files, but as of this time it's not implemented.

3

sorry for the late responses,

as of today i'm sending it vie CURL command. as400 have a UNIX portion so i can send text files as the message body vie CURL.
i dont have any elastic client installed on the system that is used.
i dont know the overall config of the elastic in our organization.
if need to i can consult the team that are in charge of this.