End-User Customize Field Names

[rockso]

I am building a project that allows the user to export data to Elasticsearch. The users have asked for the ability to change several of the field names that my project uses so that my project's data matches their other existing data. So, it seems I would need the ability for the user to transform both the mapping and the JSON data as it's being exported to Elastic. We currently use the Elastic bulk insert API to export data.

It seems like Logstash will allow us to do what we need. I'm aware of the Logstash HTTP input but I'm concerned about performance since the documentation says that Logstash will block when ingesting large amounts of data. Also, I'm worried that I would need write and maintain two separate methods in my project, one to export to Elastic directly and the other to export to Logstash.

Maybe there's another approach I'm not aware of. Ideally, the REST API or other export mechanism is the identical when exporting to Elastic and Logstash, that way my code is agnostic.

[gabriel_tessier]

Hi @rockso,

Another approach maybe more simple is using ingest, you can allow their data to be imported to elastic and convert them to your format. So you'll have only one entry to maintain.
https://www.elastic.co/guide/en/elasticsearch/reference/current/ingest.html

If I understand correctly your problem you have let's say 2 customers:
CustomerA which will import data like this
{"name": "foo",
"bar": 123
}
And CustomerB:
{"fist_name": "bar",
"bar_bar":213}

So you can use ingest to transform the data of CustomerB where CustomerA will be saved without changes as they respect your format.

[rockso]

Ingest seems like exactly what I need in this situation. Thank you!