ElasticSearch.Net Bulk - JsonParseException[Unexpected character (',' (code 44)):

Elastic Stack Elasticsearch
1

I'm trying to do bulk inserts using ElasticSearch.net but am getting this issue:

{"error":"JsonParseException[Unexpected character (',' (code 44)): expected a valid value (number, String, array, object, 'true', 'false' or 'null')\n at [Source: org.elasticsearch.transport.netty.ChannelBufferStreamInput@2c347b7; line: 1, column: 2]]","status":500}

Code:

public void BulkIndex(object mydata, string indexType)
{
ElasticsearchResponse searchResponse;
string output = JsonConvert.SerializeObject(myData);
if (output != "[]")
{
ConnectionConfiguration config = new ConnectionConfiguration(new Uri(this.uri));
ConnectionConfiguration exposed = config.ExposeRawResponse(true);
ElasticsearchClient client = new ElasticsearchClient(config, null, null, null);
output = output.Replace("}", "}\r\n");
searchResponse = client.Bulk("jdbc", indexType, output);

                if (searchResponse != null)
                  Console.WriteLine(searchResponse.Response);
                
            }

}

object is the results of a linq query.

The JSON is nice and i'm adding the carriage return line feeds:

I believe it is crashing on this part:

[{"_id":"DW1Nh31izEyyKlBWu/","MyFieldName":null,"

The null?

I have lots of nulls and not sure what to do since I'm not mapping it.

Is that the problem? It is trying to Map the field and can't because null isn't valid.

If I ignore Null columns will it keep adding to the mapping on it's own?

Any one else using Bulk Inserts with Linq to a dbml as the source and successful?

I will be passing many types to the same code and the types are variant and are not predefined.

Garrett

2

If you do this does it work:

  1. Create a file called test.txt with the following:
{ "index" : { "_index" : "testing1", "_type" : "type1", "_id" : "1" } }
{ "field1" : "foo bar" }
{ "index" : { "_index" : "testing1", "_type" : "type1", "_id" : "2" } }
{ "field1" : null }
  1. Run:

curl -s -XPOST localhost:9200/_bulk --data-binary @test.txt

  1. If you do GET /testing1/_search it should return 2 documents. One with field1 = "foo bar" and one as null.

Do you have strict set to for your mapping?

https://www.elastic.co/guide/en/elasticsearch/guide/current/dynamic-mapping.html

3

My ES farm is working with JDBC rivers now.

What I'm trying to do is create my own Windows Feeder Service that translates Linq output to JSON and creates an index.

I don't have a mapping. I was hoping to get away with creating the index and the mapping would be discovered. But because a lot of my data is null ES doesn't like it.

Or should I just not send null data?

4

Yes the simple example works with nulls.

5

Ok I think I know what i'm doing wrong.

I'm trying to do this:

{ "index" : { "_index" : "jdbc", "_type" : "testing1"} }\r\n"+
{ "_id" : "1" , "field1":null}\r\n
{ "index" : { "_index" : "jdbc", "_type" : "testing1",} }\r\n
{ "_id" : "2" , "field1":"foobar" }\r\n\r\n

Because my original json dataset has the ID's in it. I don't have a data set that has just the id's.

I have no idea how to easily stuff the ID's from the object output into a separate string.

Is there a way to tell it that the ID is in the doc?