We have a team that looks after the elastic stack, all I am doing is running the yml via a deployment.
Ok understood... I am not sure I will be able to help much if you can not get to the server.
What kind of deployment on K8s or something?
Does the deployment just send a new filebeat.yml and start and stop the service?
Do you know how to look at the logs of the deployment?
So here is a key... filebeat only loads / ingest a file Once... it keeps track.. so if the file(s) that you trying to ingest "- /var/log/myindex-app/*.log" have already been loaded, filebeat will not load / ingest them again... you can send new filebeat.yml all day long and unless there is new data nothing will happen...
Is there new data being written to those app logs?
Are the logs not visible through kibana in some way?
I have no idea where elastic is installed, I have some very silly long url that connects.
I cannot say what the install does, but when it is run it seems to change things.
I delete the index, when one exists, and the index gets recreated, just when I try and name the index in config it always fails.
I am happy to switch over to option2 as long as all of that can be done via the kibana ui.
I have spent 3 weeks trying to get this too work and can do the change manually if that's all that can be got to work.
Apologies when I said the server I mean the server where filebeat / application logs are sent... but assume you can not log into that either.
I am running the exact example above that I showed in Method 1 (without the multiline) and it works fine..
Not sure I can help if you have no access and very little experience... I am sorry!
My suggestion use the default index names and query / separate on the Kibana / Query side.
IF you have no control of over any of the environments .. then perhaps you should just use all the defaults... and just take the default index names etc... and just add your multi-line...
I have tried with the defaults, but when I try to get the filebeat index to use the ILP I get errors.
I have some access to servers, but am not aware of what I am looking for - Do you know maybe what path the logs are written to be default?
I have to log of for the evening at its 18:30. I'll pick up any update in the morning and try to continue then.
What is ILP. Index Life Cycle Policy.. if you use the defaults... you can fix that all from the Kibana Side... Basically you would just edit the default policy to what ever you want... Right through the GUI...
If you use all the defaults... you can fix almost everything from the Kibana side...
Thanks for trying.
Now going to try and get something working through the GUI, not hopefully based on experience, but gotta try something.
I really didn't expect to spend 3 weeks on this!!
Yes good luck the fact you have no access to your target machine is causing many of your issues.
I would start with complete defaults (except perhaps your multiline) and work from that.
This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.