I currently have setup Eland to pull data from Elasticsearch and I am trying to rename some of the columns. However, I realised that to use the .rename() function, I would have to convert the data to Pandas Dataframe as in the Eland documentation there is no mention of being able to use the .rename() function. So I tried converting using to_pandas(), however I received this error after it runs for awhile - NotFoundError: NotFoundError(404, 'search_phase_execution_exception', 'No search context found for id [5388191]')
I have over 700 million lines of logs, is that the reason this caused a problem? Due to timeout issues? If it is, what are the ways I could resolve it?
Thanks.