What are my options with dots in field names moving from 1.7 to 2.3.5?

Elastic Stack Elasticsearch
1

Hi all,

We have been using elasticsearch 1.7.2 for a year or so, writing events into elasticsearch from a java client, not through logstash.

We are trying to upgrade to 2.3.5 and copied a snapshot from the 1.7.2 repository to the 2.3.5 repository.

When trying to 'restore' the repository (really loading into 2.3.5 for the first time), we get the error "Field name cannot contain '.' "

Reading around a bit, it seems this is a restriction that has been introduced in 2.x. It's very unfortunate for us, as our whole event naming convention is based off using names like 'frontend.page.click' and things like that.

We have about 300Gig of data that we want to load into 2.3.5, and don't really want to rewrite all our code to change the event names in our reporting etc, but we'll do that if we have to of course.

I'm a little confused about our options.

Many thanks!

2

Hmm. Actually don't worry about it. I have an idea of what I'm going to do. We have files containing the raw events. I will write a simple field-converter in our java api that changes dots in field names to dashes, and reload all the raw events, rather than trying to load a 1.7 snapshot. I'll put these converters at all the input to elasticsearch/output from elasticsearch points in our code, so that to clients, it looks as if elasticsearch is using dots.

I get that dots was a dangerous naming convention in the first place - inherited it from someone in marketing. Children should be taught in school not to put spaces in file names etc.

3

The Elasticsearch reversed their decision regarding dot-names under certain
strict conditions. It should be available in 5.0, but it might make it in
the 2.4 branch as well, so it might be worth to wait a bit.

4

If you are going to delete everything in the thread, please report it instead and we can remove it.

1 Like