We are using elastic search index a bunch of JSON documents which all have *
roughly* the same structure, but there are some variations which is causing
headaches with dynamic mapping. Specifically fields in the document that,
when first observed by elastic search, were numeric types (and therefore
mapped as numeric by ES), but in later documents these same fields have
string values. This results in a NumberFormaException in ES and the
document cannot be indexed.
A couple of caveats:
We don't control the documents. They come from an external source and
are "supposed" to conform to a standard, but often don't
We can't plausibly provide a mapping for the entire document. They
are huge (in structure) and not always consistent.
We do provide a custom mapping for the fields we really care about
filtering on, but the others we we want indexed but care a little less
about how we would filter on these values (if at all).
Is there a way to tell elastic search to treat any new field it sees as a
string? I.e., don't infer the type for a new field, just treat it as a
string?
We are using Elasticsearch index a bunch of JSON documents which all have roughly the same structure, but there are some variations which is
causing headaches with dynamic mapping. Specifically fields in the
document that, when first observed by Elasticsearch, were numeric types
(and therefore mapped as numeric by ES), but in later documents these same
fields have string values. This results in a NumberFormaException in ES
and the document cannot be indexed.
A couple of caveats:
We don't control the documents. They come from an external source
and are "supposed" to conform to a standard, but often don't
We can't plausibly provide a mapping for the entire document. They
are huge (in structure) and not always consistent.
We do provide a custom mapping for the fields we really care about
filtering on, but the others we we want indexed but care a little less
about how we would filter on these values (if at all).
Is there a way to tell Elasticsearch to treat any new field it sees as a
string? I.e., don't infer the type for a new field, just treat it as a
string?
We are using Elasticsearch index a bunch of JSON documents which all
have roughly the same structure, but there are some variations which
is causing headaches with dynamic mapping. Specifically fields in the
document that, when first observed by Elasticsearch, were numeric types
(and therefore mapped as numeric by ES), but in later documents these same
fields have string values. This results in a NumberFormaException in ES
and the document cannot be indexed.
A couple of caveats:
We don't control the documents. They come from an external source
and are "supposed" to conform to a standard, but often don't
We can't plausibly provide a mapping for the entire document. They
are huge (in structure) and not always consistent.
We do provide a custom mapping for the fields we really care about
filtering on, but the others we we want indexed but care a little less
about how we would filter on these values (if at all).
Is there a way to tell Elasticsearch to treat any new field it sees as a
string? I.e., don't infer the type for a new field, just treat it as a
string?
Apache, Apache Lucene, Apache Hadoop, Hadoop, HDFS and the yellow elephant
logo are trademarks of the
Apache Software Foundation
in the United States and/or other countries.