Java client put logstash pipeline

Elastic Stack Elasticsearch
1

Hi,
I'm using the new Java elasticsearch client to PUT a logstash pipeline.

client.logstash().putPipeline(b -> b
                        .id(id)
                        .pipeline(p -> p
                                .description(id)
                                .lastModified(DateTime.ofEpochMilli(System.currentTimeMillis()))
                                .pipeline("input {}\n filter { grok {} }\n output {}")
                                .username("elastic")
                                .pipelineMetadata(m -> m
                                        .type("logstash_pipeline")
                                        .version("1")
                                )
                                .pipelineSettings(s -> s
                                        .pipelineBatchDelay(50)
                                        .pipelineBatchSize(125)
                                        .pipelineWorkers(1)
                                        .queueCheckpointWrites(1024)
                                        .queueMaxBytesNumber(1)
                                        .queueMaxBytesUnits("gb")
                                        .queueType("memory")
                                )
                        )
                )

but I get back an error

DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "PUT /_logstash/pipeline/org_dataset HTTP/1.1[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "User-Agent: elastic-java/8.3.2 (Java/17.0.3)[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "X-Elastic-Client-Meta: es=8.3.2,jv=17,hl=2,t=8.3.2,hc=4.1.4[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "Accept: application/vnd.elasticsearch+json; compatible-with=8[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "Content-Length: 0[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "Host: xxxxxxxxxx:9243[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "Connection: Keep-Alive[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "Authorization: Basic xxxxxxxxxxxxx=[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.wire: http-outgoing-0 >> "[\r][\n]"
DEBUG [2022-07-11 16:09:26,004] org.apache.http.impl.nio.client.InternalIODispatch: http-outgoing-0 [ACTIVE] Request ready
DEBUG [2022-07-11 16:09:26,004] org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl: http-outgoing-0 192.168.0.14:59600<->xxxxx:9243[ACTIVE][r:w][ACTIVE][r][NOT_HANDSHAKING][0][0][0]: Event cleared [w]
DEBUG [2022-07-11 16:09:26,030] org.apache.http.impl.nio.conn.ManagedNHttpClientConnectionImpl: http-outgoing-0 192.168.0.14:59600<->xxxxxx:9243[ACTIVE][r:r][ACTIVE][r][NOT_HANDSHAKING][0][0][0]: 481 bytes read
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "HTTP/1.1 400 Bad Request[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "Content-Length: 163[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "Content-Type: application/json;charset=utf-8[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "X-Cloud-Request-Id: OYCV37idR6S5_nmgbwsOvg[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "X-Elastic-Product: Elasticsearch[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "X-Found-Handling-Cluster: xxxxxxx[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "X-Found-Handling-Instance: instance-0000000000[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "Date: Mon, 11 Jul 2022 16:09:26 GMT[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "[\r][\n]"
DEBUG [2022-07-11 16:09:26,030] org.apache.http.wire: http-outgoing-0 << "{"error":{"root_cause":[{"type":"parse_exception","reason":"request body is required"}],"type":"parse_exception","reason":"request body is required"},"status":400}"

What am I doing wrong?
If I PUT directly via HTTP, it works, although losgstash 8.3.2 complains about:

[2022-07-11T16:19:05,941][WARN ][logstash.configmanagement.elasticsearchsource] Ignoring unsupported or unknown pipeline settings 'queue.max_bytes.number'
[2022-07-11T16:19:05,941][WARN ][logstash.configmanagement.elasticsearchsource] Ignoring unsupported or unknown pipeline settings 'queue.max_bytes.units'
2

This is actually a bug. I just fixed it in Boolean response endpoints can have a request body by swallez · Pull Request #341 · elastic/elasticsearch-java · GitHub

Thanks for having found and reported this issue.

1 Like
3

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.