Reindex From json File only specific log file path docs

Elastic Stack Elasticsearch
1

Good Morning I hope you're Alright.

I'm a newbie and i would like to ask something. I've made an export process with elasticdump and it was a success. I would like to ask . My end goal is to keep only valuable documents which derive from specific log file. The rest is junk.

The question is the following one. Is it possible to reindex from a dumped json file keeping only valuable files from a specific log file path to a new destination index?

Thanks in advance your help is more than valuable.

2

Hi!

Also a newbie here. Did you take a look at pipelines? They are capable of removing or dissecting fieldname. It is also usable to drop an document.

drop pipeline
Dissect pipeline
Remove pipeline

3

Good Mornign Lou!

Imagine it this way. I've Got an index which i dumped . In order to Preserve space i dump indexes & delete them from my elastic. The dumps are json file which is made from elasticdump.
Now i've got json dumps. I would like from all those dumps which are some TBs to be reindexed back from the json dump file only the valuable content which in case are the files from a specific log.file.path.

So now the question remains how will i be in posistion to reindex back only the valuable documents saving valuable TB of storage + perfomance.

Moreover reindex based on a specific field for example log.file.path can be achieved directly from elasticseaarch while indexes are open..but in my case in order to avoid floods i run back to dumps.

Hope that i gave you a more detailed approach.
Im here to discuss anything you might have questions for..

Bill!

4

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.