# Use API of elasticsearch ? Change settings

**URL:** <https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971>\
**Category:** Elasticsearch\
**Created:** [April 3, 2017, 9:18am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971 "2017-04-03T09:18:42Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 3, 2017, 9:18am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/1 "2017-04-03T09:18:42Z")

</div>

Hello,

I want increase performance of indexing, usage disk of elasticsearch. I don't know elk API

Some questions stay without response (after reading the official doc)

1. How Use bulk requests (I have for the moment, 2 index per days)

2. Disable refresh and replicas.  
/!\ : I am on test step, so i delete, create index to find the better configuration.

So, **i want know how configure this options**? (It's impossible in elasticsearch.yml since version 5+ of elk). I believe we must use api ? But how.

3.Indexing buffer size : By default is take 10% of jvm (so it's not necessary to modify it)

4.Use index template

I don't know how to create index, where create index... I believe we can change refresh interval and replicas by default before creation of any index.

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 3, 2017, 9:39am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/2 "2017-04-03T09:39:15Z")

</div>

1. [Bulk API | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html)
2. [Update index settings API | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-update-settings.html#bulk)
3. Don't touch it yet. Unless you really know what you are doing. Defaults are generally good.
4. [Index templates | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html)

> [@Beuhlet\_Reseau](#):
>
> I don't know how to create index, where create index...

```auto
PUT index

```

> [@Beuhlet\_Reseau](#):
>
> I believe we can change refresh interval and replicas by default before creation of any index.

No you can change it dynamically. See [Index modules | Elasticsearch Guide [8.11] | Elastic](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html#dynamic-index-settings)

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 3, 2017, 11:54am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/3 "2017-04-03T11:54:52Z")

</div>

[https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html)

Can you explain me with whats is bulk ?? What does it really and how use it ?

About analyzers, it increase performance ? Here is an example of line :

```
149332,AS1,0-diamproxy;1086841;88958;58da02,REQUEST,367,30987,37887,201,350657887623,LEGANT,,0,,,,,,,,0,POS__,,,,,,,,,,0,4238626,ore,,MAX_TO,858992,0
```

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 3, 2017, 4:45pm UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/4 "2017-04-03T16:45:54Z")

</div>

Bulk is explained here: [Cheaper in Bulk | Elasticsearch: The Definitive Guide [2.x] | Elastic](https://www.elastic.co/guide/en/elasticsearch/guide/2.x/bulk.html)

> About analyzers, it increase performance ?

If you mean indexing performance, I'd say no. It's the opposite. No analysis will mean less work to do.

> `149332,AS1,0-diamproxy;1086841;88958;58da02,REQUEST,367,30987,37887,201,350657887623,LEGANT,,0,,,,,,,,0,POS__,,,,,,,,,,0,4238626,ore,,MAX_TO,858992,0`

Well. It looks like a CSV file. You should do a structured document out of it IMO if you want to be able to query for anything.

I wrote a CSV import with LS tutorial a while ago: [https://david.pilato.fr/blog/2015-04-28-exploring-capitaine-train-dataset/](http://david.pilato.fr/blog/2015/04/28/exploring-capitaine-train-dataset/)

Might help.

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 4, 2017, 8:32am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/5 "2017-04-04T08:32:15Z")

</div>

Yes I use already a csv filter. IT's perfect for this line type.

Just you talking about the definition of mapping ? How to create one for my data (example of line Above) ?

I must go in kibana, devtools ? or in command line on my linux server ? Can you tell me an example of mapping for my line ?

Thank you a lots

PS : To remove empty fields in filter, i must use remove\_field ? or in the "template", or with expression as [if field = ""] drop {} ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 4, 2017, 9:33am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/6 "2017-04-04T09:33:57Z")

</div>

What does a document look like after it has been parsed by logstash?

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 4, 2017, 9:42am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/7 "2017-04-04T09:42:11Z")

</div>

{  
"ca-d-2017.04": {  
"aliases": {},  
"mappings": {  
"cra": {  
"properties": {  
"@timestamp": {  
"type": "date"  
},  
"@version": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"beat": {  
"properties": {  
"hostname": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"name": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"version": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
}  
}  
},  
"ca\_PN": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},  
"cra\_AVol": {  
"type": "long"  
},  
"cra\_II": {  
"type": "long"  
},  
"cra\_ISI": {  
"type": "long"  
},  
"cra\_MDN": {  
"type": "long"  
},  
"cra\_PType": {  
"type": "text",  
"fields": {  
"keyword": {  
"type": "keyword",  
"ignore\_above": 256  
}  
}  
},

[...]

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 4, 2017, 9:44am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/8 "2017-04-04T09:44:50Z")

</div>

I use dev tools of kibana. It's simply of use.

Can ia create template from this tools ?

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 4, 2017, 1:39pm UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/9 "2017-04-04T13:39:18Z")

</div>

Here how i see things to create template :

 ![](https://us1.discourse-cdn.com/elastic/original/3X/3/5/35955e628c168b40cbf3756ccaf7ba280086fd82.png)

What do you think about that ?

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 4, 2017, 1:42pm UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/10 "2017-04-04T13:42:09Z")

</div>

Looks correct.

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 5, 2017, 8:16am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/11 "2017-04-05T08:16:37Z")

</div>

Hello,

I am questio about template i have created. I don't want my text fields are cut (analyzed grrr)

Instead of specifying it on each field, I would make a global rule for all my template, this possible? If yes, how?

For now, here my template. What do you think about that ? I create it on kibana devtools :

PUT \_template/templatecra  
{  
"template" : "cra\*",  
"order" : 0,  
"settings" : {  
"number\_of\_shards" : 1 , "refresh\_interval": "-1" , "number\_of\_replicas": 0  
},

```
"mappings" : {
    "cra" : {
      "properties": {
        "source" : { "enabled" : false },
        "beat": { "enabled" : false },
        "@version": { "enabled" : false },
        "name": { "enabled" : false },
        "version": { "enabled" : false },
        "host": { "enabled" : false },
        "input_type": { "enabled" : false },
        "tags": { "enabled" : false },
        "type": { "enabled" : false },
        
        "cra_AN": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
          }
         }
        },
        "cra_ountVol": {
          "type": "long"
        },
        "cra_EI": {
          "type": "long"
        },
        "cra_SI": {
          "type": "long"
        },
        "cra_ISDN": {
          "type": "long"
        },
        "cra_PType": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_Nid": {
          "type": "integer"
        },
        "cra_Sere2": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_Tade": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_biPlan": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_dataVol": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_dataVolO": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_leglDist": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_mobrage": {
          "type": "text"
        },
        "cra_ratType": {
          "type": "short"
        },
        "cra_rType": {
          "type": "byte"
        },
        "cra_servID": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_startTime": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_terv": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
        "cra_zoD": {
          "type": "text",
          "fields": {
            "keyword": {
              "type": "keyword",
              "ignore_above": 256
            }
          }
        },
      
        }
      }
  }

```

}

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 5, 2017, 8:40am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/12 "2017-04-05T08:40:38Z")

</div>

oh i have error ... invisible

"type": "illegal\_argument\_exception",  
"reason": "failed to parse template source",  
"caused\_by": {  
"type": "json\_parse\_exception",  
"reason": "Unexpected character ('}' (code 125)): was expecting double-quote to start field name\n at [Source: org.elasticsearch.transport.netty4.ByteBufStreamInput@3bb8d48c; line: 155, column: 14]"  
}  
},  
"status": 400  
}

---

<div class="post-metadata">

**Author:** ![dadoonet](https://sea2.discourse-cdn.com/elastic/user_avatar/discuss.elastic.co/dadoonet/32/137187_2.png) [@dadoonet](https://discuss.elastic.co/u/dadoonet)\
**Post date:** [April 5, 2017, 9:39am UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/13 "2017-04-05T09:39:15Z")

</div>

Please format your code using `</>` icon as explained in [this guide](https://discuss.elastic.co/t/about-the-elasticsearch-category/21). It will make your post more readable.

Or use markdown style like:

````
```
CODE
```

````

If the JSON is incorrect I believe that Kibana console is telling you that with a red cross.

---

<div class="post-metadata">

**Author:** ![Beuhlet\_Reseau](https://avatars.discourse-cdn.com/v4/letter/b/e95f7d/32.png) [@Beuhlet\_Reseau](https://discuss.elastic.co/u/Beuhlet_Reseau)\
**Post date:** [April 5, 2017, 1:08pm UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/14 "2017-04-05T13:08:07Z")

</div>

Ok here with code \</\>.

Where i can put not analyze string type ? ( **text & keyword from ES 5.x** ).

So, maybe I don't need metafield as source, score, beats info ... where i can delete few of them ?

```
    {
    "order": 0,
    "version": 50001,
    "template": "ta-test-edr",
    "settings": {
      "index": {
        "number_of_replicas": 0,
        "number_of_shards" : 1,
        "refresh_interval": "-1"
      }
    },
    "mappings": {
      "_default_": {
        "dynamic_templates": [
          {
            "string_fields": {
              "mapping": {
                "norms": false,
                "type": "text",
                "fields": {
                  "keyword": {
                    "type": "keyword"
                  }
                }
              },
              "match_mapping_type": "string",
              "match": "*"
            }
          }
        ],
        "_all": {
          "norms": false,
          "enabled": true
        },
        "properties": {
          "@timestamp": {
            "include_in_all": false,
            "type": "date"
          },
          "geoip": {
            "dynamic": true,
            "properties": {
              "ip": {
                "type": "ip"
              },
              "latitude": {
                "type": "half_float"
              },
              "location": {
                "type": "geo_point"
              },
              "longitude": {
                "type": "half_float"
              }
            }
          },
          "@version": {
            "include_in_all": false,
            "type": "keyword"
          }
        }
      }
    },
    "aliases": {}
  }
```

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/elastic/original/3X/1/a/1ac57faf039f6b580b3f104ef42a2a89e41014de.png) [@system](https://discuss.elastic.co/u/system)\
**Post date:** [May 3, 2017, 1:08pm UTC](https://discuss.elastic.co/t/use-api-of-elasticsearch-change-settings/80971/15 "2017-05-03T13:08:36Z")

</div>

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.