_meta: Root mapping definition has unsupported parameters

Elastic Stack Elasticsearch
1

I have ES7. Currently the mapping looks like:

{
'mappings':
    {
     'properties': 
        {
        'aIndex': {'type': 'integer'},
        'AC': {'type': 'integer'},
        'AF': {'type': 'double'},
        ...
        },
    '_meta': {'gencodeVersion': '25', 'sourceFilePath': 's3://grch38_subset.vcf', 'genomeVersion': '38', 'sampleType': 'WGS', 'hail_version': '0.2.57'}
}

And it gives 400 Request error because it does not like keys inside _meta:

elasticsearch.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'Root mapping definition has unsupported parameters: [gencodeVersion : 25] [hail_version : 0.2.57] [genomeVersion : 38] [sampleType : WGS] [sourceFilePath : s3://grch38_subset.vcf]')

Below is the full mapping inside mappings key:

{'properties': {'aIndex': {'type': 'integer'}, 'AC': {'type': 'integer'}, 'AF': {'type': 'double'}, 'alt': {'type': 'keyword'}, 'AN': {'type': 'integer'}, 'bgi_AC': {'type': 'integer'}, 'bgi_AF': {'type': 'double'}, 'bgi_AN': {'type': 'integer'}, 'cadd_PHRED': {'type': 'float'}, 'cidr_AC': {'type': 'integer'}, 'cidr_AF': {'type': 'double'}, 'cidr_AN': {'type': 'integer'}, 'clinvar_allele_id': {'type': 'integer'}, 'clinvar_clinical_significance': {'type': 'keyword'}, 'clinvar_gold_stars': {'type': 'integer'}, 'codingGeneIds': {'type': 'keyword'}, 'contig': {'type': 'keyword'}, 'dbnsfp_SIFT_pred': {'type': 'keyword'}, 'dbnsfp_Polyphen2_HVAR_pred': {'type': 'keyword'}, 'dbnsfp_MutationTaster_pred': {'type': 'keyword'}, 'dbnsfp_FATHMM_pred': {'type': 'keyword'}, 'dbnsfp_MetaSVM_pred': {'type': 'keyword'}, 'dbnsfp_REVEL_score': {'type': 'keyword'}, 'dbnsfp_GERP_RS': {'type': 'keyword'}, 'dbnsfp_phastCons100way_vertebrate': {'type': 'keyword'}, 'docId': {'type': 'keyword'}, 'domains': {'type': 'keyword'}, 'eigen_Eigen_phred': {'type': 'double'}, 'end': {'type': 'integer'}, 'exac_AF_POPMAX': {'type': 'double'}, 'exac_AF': {'type': 'double'}, 'exac_AC_Adj': {'type': 'integer'}, 'exac_AC_Het': {'type': 'integer'}, 'exac_AC_Hom': {'type': 'integer'}, 'exac_AC_Hemi': {'type': 'integer'}, 'exac_AN_Adj': {'type': 'integer'}, 'filters': {'type': 'keyword'}, 'g1k_AC': {'type': 'integer'}, 'g1k_AF': {'type': 'double'}, 'g1k_AN': {'type': 'integer'}, 'g1k_POPMAX_AF': {'type': 'double'}, 'geneIds': {'type': 'keyword'}, 'geno2mp_HPO_Count': {'type': 'integer'}, 'genotypes': {'properties': {'num_alt': {'type': 'integer'}, 'gq': {'type': 'integer'}, 'ab': {'type': 'double'}, 'dp': {'type': 'double'}, 'ad': {'type': 'keyword'}, 'pl': {'type': 'keyword'}, 'sample_id': {'type': 'keyword'}}, 'type': 'nested'}, 'gnomad_exome_coverage': {'type': 'double'}, 'gnomad_exomes_AF': {'type': 'double'}, 'gnomad_exomes_AN': {'type': 'integer'}, 'gnomad_exomes_AC': {'type': 'integer'}, 'gnomad_exomes_Hom': {'type': 'integer'}, 'gnomad_exomes_AF_POPMAX_OR_GLOBAL': {'type': 'double'}, 'gnomad_exomes_FAF_AF': {'type': 'double'}, 'gnomad_exomes_Hemi': {'type': 'integer'}, 'gnomad_genome_coverage': {'type': 'float'}, 'gnomad_genomes_AF': {'type': 'double'}, 'gnomad_genomes_AN': {'type': 'integer'}, 'gnomad_genomes_AC': {'type': 'integer'}, 'gnomad_genomes_Hom': {'type': 'integer'}, 'gnomad_genomes_AF_POPMAX_OR_GLOBAL': {'type': 'double'}, 'gnomad_genomes_FAF_AF': {'type': 'double'}, 'gnomad_genomes_Hemi': {'type': 'integer'}, 'hgmd_accession': {'type': 'keyword'}, 'hgmd_class': {'type': 'keyword'}, 'hgmd_like': {'type': 'keyword'}, 'hgsc_wes_AC': {'type': 'integer'}, 'hgsc_wes_AF': {'type': 'double'}, 'hgsc_wes_AN': {'type': 'integer'}, 'hgsc_wgs_AC': {'type': 'integer'}, 'hgsc_wgs_AF': {'type': 'double'}, 'hgsc_wgs_AN': {'type': 'integer'}, 'mainTranscript_biotype': {'type': 'keyword'}, 'mainTranscript_canonical': {'type': 'integer'}, 'mainTranscript_category': {'type': 'keyword'}, 'mainTranscript_cdna_start': {'type': 'integer'}, 'mainTranscript_cdna_end': {'type': 'integer'}, 'mainTranscript_codons': {'type': 'keyword'}, 'mainTranscript_gene_id': {'type': 'keyword'}, 'mainTranscript_gene_symbol': {'type': 'keyword'}, 'mainTranscript_hgvs': {'type': 'keyword'}, 'mainTranscript_hgvsc': {'type': 'keyword'}, 'mainTranscript_major_consequence': {'type': 'keyword'}, 'mainTranscript_major_consequence_rank': {'type': 'integer'}, 'mainTranscript_transcript_id': {'type': 'keyword'}, 'mainTranscript_amino_acids': {'type': 'keyword'}, 'mainTranscript_domains': {'type': 'keyword'}, 'mainTranscript_hgvsp': {'type': 'keyword'}, 'mainTranscript_lof': {'type': 'keyword'}, 'mainTranscript_lof_flags': {'type': 'keyword'}, 'mainTranscript_lof_filter': {'type': 'keyword'}, 'mainTranscript_lof_info': {'type': 'keyword'}, 'mainTranscript_polyphen_prediction': {'type': 'keyword'}, 'mainTranscript_protein_id': {'type': 'keyword'}, 'mainTranscript_sift_prediction': {'type': 'keyword'}, 'mpc_MPC': {'type': 'keyword'}, 'nisc_AC': {'type': 'integer'}, 'nisc_AF': {'type': 'double'}, 'nisc_AN': {'type': 'integer'}, 'originalAltAlleles': {'type': 'keyword'}, 'pos': {'type': 'integer'}, 'primate_ai_score': {'type': 'double'}, 'ref': {'type': 'keyword'}, 'rg37_locus': {'type': 'object', 'properties': {'contig': {'type': 'keyword'}, 'position': {'type': 'integer'}}}, 'rsid': {'type': 'keyword'}, 'samples_ab_0_to_5': {'type': 'keyword'}, 'samples_ab_5_to_10': {'type': 'keyword'}, 'samples_ab_10_to_15': {'type': 'keyword'}, 'samples_ab_15_to_20': {'type': 'keyword'}, 'samples_ab_20_to_25': {'type': 'keyword'}, 'samples_ab_25_to_30': {'type': 'keyword'}, 'samples_ab_30_to_35': {'type': 'keyword'}, 'samples_ab_35_to_40': {'type': 'keyword'}, 'samples_ab_40_to_45': {'type': 'keyword'}, 'samples_gq_0_to_5': {'type': 'keyword'}, 'samples_gq_5_to_10': {'type': 'keyword'}, 'samples_gq_10_to_15': {'type': 'keyword'}, 'samples_gq_15_to_20': {'type': 'keyword'}, 'samples_gq_20_to_25': {'type': 'keyword'}, 'samples_gq_25_to_30': {'type': 'keyword'}, 'samples_gq_30_to_35': {'type': 'keyword'}, 'samples_gq_35_to_40': {'type': 'keyword'}, 'samples_gq_40_to_45': {'type': 'keyword'}, 'samples_gq_45_to_50': {'type': 'keyword'}, 'samples_gq_50_to_55': {'type': 'keyword'}, 'samples_gq_55_to_60': {'type': 'keyword'}, 'samples_gq_60_to_65': {'type': 'keyword'}, 'samples_gq_65_to_70': {'type': 'keyword'}, 'samples_gq_70_to_75': {'type': 'keyword'}, 'samples_gq_75_to_80': {'type': 'keyword'}, 'samples_gq_80_to_85': {'type': 'keyword'}, 'samples_gq_85_to_90': {'type': 'keyword'}, 'samples_gq_90_to_95': {'type': 'keyword'}, 'samples_no_call': {'type': 'keyword'}, 'samples_num_alt_1': {'type': 'keyword'}, 'samples_num_alt_2': {'type': 'keyword'}, 'sortedTranscriptConsequences': {'properties': {'biotype': {'type': 'keyword'}, 'canonical': {'type': 'integer'}, 'cdna_start': {'type': 'integer'}, 'cdna_end': {'type': 'integer'}, 'codons': {'type': 'keyword'}, 'gene_id': {'type': 'keyword'}, 'gene_symbol': {'type': 'keyword'}, 'hgvsc': {'type': 'keyword'}, 'hgvsp': {'type': 'keyword'}, 'transcript_id': {'type': 'keyword'}, 'amino_acids': {'type': 'keyword'}, 'lof': {'type': 'keyword'}, 'lof_filter': {'type': 'keyword'}, 'lof_flags': {'type': 'keyword'}, 'lof_info': {'type': 'keyword'}, 'polyphen_prediction': {'type': 'keyword'}, 'protein_id': {'type': 'keyword'}, 'protein_start': {'type': 'integer'}, 'sift_prediction': {'type': 'keyword'}, 'consequence_terms': {'type': 'keyword'}, 'domains': {'type': 'keyword'}, 'major_consequence': {'type': 'keyword'}, 'category': {'type': 'keyword'}, 'hgvs': {'type': 'keyword'}, 'major_consequence_rank': {'type': 'integer'}, 'transcript_rank': {'type': 'integer'}}, 'type': 'nested'}, 'splice_ai_delta_score': {'type': 'float'}, 'start': {'type': 'integer'}, 'topmed_AC': {'type': 'integer'}, 'topmed_AF': {'type': 'double'}, 'topmed_AN': {'type': 'integer'}, 'topmed_Hom': {'type': 'integer'}, 'topmed_Het': {'type': 'integer'}, 'transcriptConsequenceTerms': {'type': 'keyword'}, 'transcriptIds': {'type': 'keyword'}, 'utrVariantAnnotation': {'properties': {'existing_inframe_oorfs': {'type': 'integer'}, 'existing_outofframe_oorfs': {'type': 'integer'}, 'existing_uorfs': {'type': 'integer'}, 'five_prime_utr_variant_consequence': {'type': 'keyword'}, 'five_prime_utr_variant_annotation': {'type': 'keyword'}}, 'type': 'nested'}, 'variantId': {'type': 'keyword'}, 'xpos': {'type': 'long'}, 'xstart': {'type': 'long'}, 'xstop': {'type': 'long'}}, '_meta': {'gencodeVersion': '25', 'sourceFilePath': 's3://grch38_subset.vcf', 'genomeVersion': '38', 'sampleType': 'WGS', 'hail_version': '0.2.57'}}

Could you suggest on how I should change the mappings of the _meta for it to work?

2

Hi @NLSVTN Just a bit of a syntax error you have the "_meta" at the wrong level it goes inside the "mappings" you have i outside.

PUT discuss
{
  "mappings": {
    "properties": {
      "aIndex": {
        "type": "integer"
      },
      "AC": {
        "type": "integer"
      },
      "AF": {
        "type": "double"
      }
    },
    "_meta": {
      "gencodeVersion": "25"
    }
  }
}

# Result
{
  "acknowledged": true,
  "shards_acknowledged": true,
  "index": "discuss"
}

GET discuss

#Result

{
  "discuss": {
    "aliases": {},
    "mappings": {
      "_meta": {
        "gencodeVersion": "25"
      },
      "properties": {
        "AC": {
          "type": "integer"
        },
        "AF": {
          "type": "double"
        },
        "aIndex": {
          "type": "integer"
        }
      }
    },
    "settings": {
      "index": {
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_content"
            }
          }
        },
        "number_of_shards": "1",
        "provided_name": "discuss",
        "creation_date": "1659155996920",
        "number_of_replicas": "1",
        "uuid": "79mz2pieRVel4u3Q9GSYYQ",
        "version": {
          "created": "8030299"
        }
      }
    }
  }
}
5

It turned out that somehow an older version of ES was used even though we installed ES7, so adding additional level between mappings and _meta solved the issue:

'mappings': {
     'variant': {
          '_meta': {...}
      },
 },
6

Glad it works.. interesting that is different than the doc show here

Unless you are going all the way back to 6.X here and that is a bit different as well here

1 Like
7

Yeah, it looks like we can put all mapping under an arbitrary named _doc, not only _doc.

8

It is because version 6.X still support types and the type can be anything, but just one type per index, in your case it seems that the type name seems to be variant.

9

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.