Is there a way I can send only certain parts of a log to elasticsearch. I don't want to remove some logs, I want to remove some parts of a log. For eg. I want to remove [INFO] from my log, and only send the rest of it to elasticsearch. Is there a way to do it?
I moved your question to Logstash.
Can you also help me with the solution?
Normally one would use a grok filter to extract fields from a flat log message, and then you can simply skip extracting fields from the parts of the message that you don't care about.
If that doesn't answer your question you have to be more specific.
I understand that I can index certain parts of the log message, But actually the entire log message gets stores in the _source field. What is I don't want the unidexed parts to get staore in the _source field? Do I have to then disable _source and then enable store for each individual field? Or is there a better alternative?
You can indeed disable _source and save some space, or you can remove specific fields completely. What I don't think you can do is index the contents of field A without including A in the document saved in _source.