Export Transactions

hello

my original file;

#!/bin/bash

export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin

if [ -z "$1" ]; then
        ADAYAGO=`date --date="1 day ago" +%Y-%m-%d`
else
        ADAYAGO="$1"
fi

echo "Starting $ADAYAGO"
QUERY_STR='{"query":{"range":{"timestamp":{"gte":"'"$ADAYAGO "'00:00:00.000","lte":"'"$ADAYAGO"' 23:59:59.999"}}}}'
EXPORT_PATH=/home/sgeuser/elasticsearch
EXPORT_FILE_NAME=esbackup.$ADAYAGO.json
es-export-bulk --url http://localhost:9200 --file $EXPORT_PATH/$EXPORT_FILE_NAME --body "$QUERY_STR"
ls -lh $EXPORT_PATH/$EXPORT_FILE_NAME
gzip -9 $EXPORT_PATH/$EXPORT_FILE_NAME
md5sum $EXPORT_PATH/$EXPORT_FILE_NAME.gz > $EXPORT_PATH/$EXPORT_FILE_NAME.gz.hash
sha1sum $EXPORT_PATH/$EXPORT_FILE_NAME.gz >> $EXPORT_PATH/$EXPORT_FILE_NAME.gz.hash
sha256sum $EXPORT_PATH/$EXPORT_FILE_NAME.gz >> $EXPORT_PATH/$EXPORT_FILE_NAME.gz.hash

when i run my command line it gives air?

es-export-bulk --url http://localhost:9200 --file $EXPORT_PATH/$EXPORT_FILE_NAME --body "$QUERY_STR"
fs.js:646
  return binding.open(pathModule._makeLong(path), stringToFlags(flags), mode);

Error: EISDIR: illegal operation on a directory, open '/'
    at Object.fs.openSync (fs.js:646:18)
    at Object.fs.writeFileSync (fs.js:1299:33)
    at Object.<anonymous> (/usr/local/lib/node_modules/elasticsearch-tools/lib/es-export-bulk.js:279:6)
    at Module._compile (module.js:652:30)
    at Object.Module._extensions..js (module.js:663:10)
    at Module.load (module.js:565:32)
    at tryModuleLoad (module.js:505:12)
    at Function.Module._load (module.js:497:3)
    at Function.Module.runMain (module.js:693:10)
    at startup (bootstrap_node.js:188:16)

I'm waiting for your help

Welcome to our community! :smiley:

There looks to be a problem with your script, not with Elasticsearch. You might be able to diagnose it by adding set -x to the top of your script and running it again, to see what it is doing.

1 Like

++ export PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
++ PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin
++ '[' -z '' ']'
+++ date '--date=1 day ago' +%Y-%m-%d
++ ADAYAGO=2022-09-11
++ echo 'Starting 2022-09-11'
Starting 2022-09-11
++ QUERY_STR='{"query":{"range":{"timestamp":{"gte":"2022-09-11 00:00:00.000","lte":"2022-09-11 23:59:59.999"}}}}'
++ EXPORT_PATH=/home/sgeuser/elasticsearch
++ EXPORT_FILE_NAME=esbackup.2022-09-11.json
++ es-export-bulk --url http://localhost:9200 --file /home/sgeuser/elasticsearch/esbackup.2022-09-11.json --body '{"query":{"range":{"timestamp":{"gte":"2022-09-11 00:00:00.000","lte":"2022-09-11 23:59:59.999"}}}}'
/usr/local/lib/node_modules/elasticsearch-tools/node_modules/progress/lib/node-progress.js:55
    if ('number' != typeof options.total) throw new Error('total required');
                                          ^

Error: total required
    at new ProgressBar (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/progress/lib/node-progress.js:55:49)
    at processResults (/usr/local/lib/node_modules/elasticsearch-tools/lib/es-export-bulk.js:191:11)
    at respond (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/elasticsearch/src/lib/transport.js:329:9)
    at checkRespForFailure (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/elasticsearch/src/lib/transport.js:267:7)
    at HttpConnector.<anonymous> (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/elasticsearch/src/lib/connectors/http.js:166:7)
    at IncomingMessage.wrapper (/usr/local/lib/node_modules/elasticsearch-tools/node_modules/lodash/lodash.js:4991:19)
    at emitNone (events.js:111:20)
    at IncomingMessage.emit (events.js:208:7)
    at endReadableNT (_stream_readable.js:1064:12)
    at _combinedTickCallback (internal/process/next_tick.js:138:11)
++ ls -lh /home/sgeuser/elasticsearch/esbackup.2022-09-11.json
-rw-r--r-- 1 root root 0 Sep 12 09:29 /home/sgeuser/elasticsearch/esbackup.2022-09-11.json
++ gzip -9 /home/sgeuser/elasticsearch/esbackup.2022-09-11.json
++ md5sum /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz
++ sha1sum /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz
++ sha256sum /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz
++ sleep 5
++ ls -lh /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz.hash
-rw-r--r-- 1 root root  45 Sep 12 09:29 /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz
-rw-r--r-- 1 root root 310 Sep 12 09:29 /home/sgeuser/elasticsearch/esbackup.2022-09-11.json.gz.hash

It seems you are relying on some third party utility that I have never heard of. You may need to reach out to the creator for help with this.

Wants to retrieve log records from the previous day via Graylog, this is the purpose?