Export Transactions

Elastic Stack Elasticsearch
1

hello

my original file;

#!/bin/bash

export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin

if [ -z "$1" ]; then
        ADAYAGO=`date --date="1 day ago" +%Y-%m-%d`
else
        ADAYAGO="$1"
fi

echo "Starting $ADAYAGO"
QUERY_STR='{"query":{"range":{"timestamp":{"gte":"'"$ADAYAGO "'00:00:00.000","lte":"'"$ADAYAGO"' 23:59:59.999"}}}}'
EXPORT_PATH=/home/sgeuser/elasticsearch
EXPORT_FILE_NAME=esbackup.$ADAYAGO.json
es-export-bulk --url http://localhost:9200 --file $EXPORT_PATH/$EXPORT_FILE_NAME --body "$QUERY_STR"
ls -lh $EXPORT_PATH/$EXPORT_FILE_NAME
gzip -9 $EXPORT_PATH/$EXPORT_FILE_NAME
md5sum $EXPORT_PATH/$EXPORT_FILE_NAME.gz > $EXPORT_PATH/$EXPORT_FILE_NAME.gz.hash
sha1sum $EXPORT_PATH/$EXPORT_FILE_NAME.gz >> $EXPORT_PATH/$EXPORT_FILE_NAME.gz.hash
sha256sum $EXPORT_PATH/$EXPORT_FILE_NAME.gz >> $EXPORT_PATH/$EXPORT_FILE_NAME.gz.hash

when i run my command line it gives air?

es-export-bulk --url http://localhost:9200 --file $EXPORT_PATH/$EXPORT_FILE_NAME --body "$QUERY_STR"
fs.js:646
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);

Error: EISDIR: illegal operation on a directory, open '/'
    at Object.fs.openSync (fs.js:646:18)
    at Object.fs.writeFileSync (fs.js:1299:33)
    at Object.<anonymous> (/usr/local/lib/node_modules/elasticsearch-tools/lib/es-export-bulk.js:279:6)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
    at startup (bootstrap_node.js:188:16)

I'm waiting for your help

2

Welcome to our community! :smiley:

There looks to be a problem with your script, not with Elasticsearch. You might be able to diagnose it by adding set -x to the top of your script and running it again, to see what it is doing.

1 Like
3 

++ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
++ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
++ '[' -z '' ']'
+++ date '--date=1 day ago' +%Y-%m-%d
++ ADAYAGO=2022-09-11
++ echo 'Starting 2022-09-11'
Starting 2022-09-11
++ QUERY_STR='{"query":{"range":{"timestamp":{"gte":"2022-09-11 00:00:00.000","lte":"2022-09-11 23:59:59.999"}}}}'
++ EXPORT_PATH=/home/sgeuser/elasticsearch
++ EXPORT_FILE_NAME=esbackup.2022-09-11.json
++ es-export-bulk --url http://localhost:9200 --file /home/sgeuser/elasticsearch/esbackup.2022-09-11.json --body '{"query":{"range":{"timestamp":{"gte":"2022-09-11 00:00:00.000","lte":"2022-09-11 23:59:59.999"}}}}'
/usr/local/lib/node_modules/elasticsearch-tools/node_modules/progress/lib/node-progress.js:55
    if ('number' != typeof options.total) throw new Error('total required');
                                          ^

Error: total required
    at new ProgressBar (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/progress/lib/node-progress.js:55:49)
    at processResults (/usr/local/lib/node_modules/elasticsearch-tools/lib/es-export-bulk.js:191:11)
    at respond (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/elasticsearch/src/lib/transport.js:329:9)
    at checkRespForFailure (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/elasticsearch/src/lib/transport.js:267:7)
    at HttpConnector.<anonymous> (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)
    at IncomingMessage.wrapper (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/lodash/lodash.js:4991:19)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
++ ls -lh /home/sgeuser/elasticsearch/esbackup.2022-09-11.json
-rw-r--r-- 1 root root 0 Sep 12 09:29 /home/sgeuser/elasticsearch/esbackup.2022-09-11.json
++ gzip -9 /home/sgeuser/elasticsearch/esbackup.2022-09-11.json
++ md5sum /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz
++ sha1sum /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz
++ sha256sum /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz
++ sleep 5
++ ls -lh /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz.hash
-rw-r--r-- 1 root root  45 Sep 12 09:29 /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz
-rw-r--r-- 1 root root 310 Sep 12 09:29 /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz.hash
4

It seems you are relying on some third party utility that I have never heard of. You may need to reach out to the creator for help with this.

5

Wants to retrieve log records from the previous day via Graylog, this is the purpose?

6

This topic was automatically closed 28 days after the last reply. New replies are no longer allowed.