Limit of total fields Error, Correct structure of .Json-File?

Elastic Stack Elasticsearch
1

Hello,

im new to Elasticsearch.

I got .json -Files from scraping a finances page.

They can look like this:

{
    "1. Information": "Daily Prices (open, high, low, close) and Volumes", 
    "2. Symbol": "MMM", 
    "3. Last Refreshed": "2018-01-10", 
    "4. Output Size": "Full size", 
    "5. Time Zone": "US/Eastern"
}{
    "2000-01-03": {
        "1. open": "96.0600", 
        "2. high": "96.4400", 
        "3. low": "94.0600", 
        "4. close": "94.3800", 
        "5. volume": "1086700"
    }, 
    "2000-01-04": {
        "1. open": "92.8800", 
        "2. high": "94.8100", 
        "3. low": "90.6300", 
        "4. close": "90.6300", 
        "5. volume": "1356900"
    }, 
    "2000-01-05": {
        "1. open": "91.1300", 
        "2. high": "95.3100", 
        "3. low": "91.1300", 
        "4. close": "95.2500", 
        "5. volume": "1849700"
    }, ....

or like this:

{
    "2000-01-03": {
        "1. open": "96.0600", 
        "2. high": "96.4400", 
        "3. low": "94.0600", 
        "4. close": "94.3800", 
        "5. volume": "1086700"
    }, 
    "2000-01-04": {
        "1. open": "92.8800", 
        "2. high": "94.8100", 
        "3. low": "90.6300", 
        "4. close": "90.6300", 
        "5. volume": "1356900"
    }, 
    "2000-01-05": {
        "1. open": "91.1300", 
        "2. high": "95.3100", 
        "3. low": "91.1300", 
        "4. close": "95.2500", 
        "5. volume": "1849700"
    },...

I used a mapping like this:

{
 "mappings": {
  "doc": {
   "properties": {
      "open": {"type": "double"},
      "high": {"type": "double"},
      "low": {"type": "double"},
      "close": {"type": "double"},
      "volume": {"type": "double"}
     }
    }
   }
  }

and then tried to upload the .json -File with this command:

curl -XPOST 'http://localhost:9200/index/doc' -H 'Content-Type: application/json' -d @test.json

but i got an Error:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Limit of total fields [1000] in index [index] has been exceeded"}], "type":"illegal_argument_exception","reason":"Limit of total fields [1000] in index [index] has been exceeded"},"status":400}

My aim is to upload every share of a Company and to reproduce the graph with Kibana.

How can I increase the "Limit of total fields"?
And the 2nd thing is im not sure about the structure in the .json-Files. Is it even possible to do this with these .json-Files?

Im thankful for every help. :slight_smile:

2

Change your model and index each share quotation instead (with its timestamp).

3

Could u show me a short example how the json-File should look Like?

4

Probably something like:

{
  "date": "2000-01-03",
  "Symbol": "MMM",
  "open": 96.0600, 
  "high": 96.4400, 
  "low": 94.0600, 
  "close": 94.3800, 
  "volume": 1086700
}
5

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.