How to read a file from JMETER in node.js and index into elasticsearch?

Elastic Stack Elasticsearch
1

I am using node.js to index a file into elasticsearch.
i am trying to give input as a file from jmeter? i.e.., as req.file
but it giving output as undefined.
can anyone specify the better solution?

2

Hi,

sounds like you want to benchmark your cluster. We have built Rally for exactly that purpose. In your case you'd need to create your own track based on your input file (in line with the general racing vocabulary we call the benchmark description a "track" in Rally).

Just check whether this fits your use case and feel free to ask questions about Rally in this forum.

Daniel

1 Like
3

thanks for your reply daniel.
i need to index document first through node.js later on benchmark

4

Can you show a bit more of what you are trying to accomplish? Do you have some node.js code that you are using?

We generally use jmeter in combination with logstash, or if you do not mind using alpha software / use ingest. A bit of information about ingest can be found in a blogpost a colleague of mine wrote: https://amsterdam.luminis.eu/2016/04/01/just-tailing-a-file-exploring-new-elasticsearchs-ingest/

1 Like
5

node.js code which i am using in the project

var http = require('http');
var express = require('express');
var app = express();
var CircularJSON = require('circular-json');
var elasticsearch = require('elasticsearch');
var client = new elasticsearch.Client({
host: 'IP:port',
log: 'trace'
});
http.createServer(app).listen(1000);

var bodyParser = require('body-parser');
app.use(bodyParser.urlencoded({
extended: true
}));
app.use(bodyParser.json());
app.post('/post/:index/:type/:id/', function(req, res){
var fs = require("fs");
fs.writeFile('inpu.json',req.file);
client.index({
index: req.params.index,
type: req.params.type,
id: req.params.id,
body:JSON.parse(fs.readFileSync(__dirname +'/inpu.json'))
},function (error, response) {
console.log(response);
res.send(response);
});
});

6

Hi,

I don't know much about Javascript but this code seems just to act as some kind of proxy and as far as I understand it, it also has a race condition. It will fail if two concurrent requests come in because it relies that it owns the file inpu.json (typo?) exclusively. Why do you need this at all?

Also, it is not clear to me what the file contains. Just one document?

If you want to index a lot of documents at once, you should use the bulk API otherwise performance will suffer badly.

W.r.t. to your original question: I don't use JMeter these days but the JMeter docs indicate to me that file upload should be doable with the File Path parameter.

Daniel

7

Pasted image1680×1050 175 KB

i am giving input in file upload space to node.js code from jmeter

8

Hi,

I guess you don't really upload C:/video1.file as indicated by your screenshot, right? :slight_smile:

How does the document look like and what is the problem? You wrote initially that you get "undefined" (where?). Sounds like a problem in your node.js code? Did you try indexing directly into Elasticsearch?

Daniel

9

ya i tried uploading directly to elasticsearch
it worked fine.

file contains video data in base64 format.

the problem is in my code. i need a solution for that.

getting undefined in inpu.json while reading from jmeter using req.file

10

Hi,

I am sorry but solving a Javascript problem is quite offtopic here, no?. :wink: I suggest you better try some Javascript related forum. There will be people that are much more knowledgeable and I am sure they are able to answer your Javascript question. :slight_smile:

Daniel

1 Like
11

ok thanks for your time