Yrurtj

I have a list.txt file including a list of log file.

For example

    server_log1

    server_log2

    ........

    server_log50

I have another shell script used to download these logs. It worked like this

./script.sh serverlog1

I want to make it automatically that means it can automatically pass in each log file's name in list.txt to be executed.

Is that possible?
I tried

#!/bin/bash

for i in `cat /home/ec2-user/list.txt` ; do

sh ./workaround.sh $i

done

But it didn't work

The easiest method for reading arguments can be described as follows;

Each argument is referenced and parsed by the $IFS or currently defined internal file separator. The default character is a space.

For example, take the following; # ./script.sh arg1 arg2

The argument list in that example is arg1 = $1 and arg2 = $2 which can be rewritten as arg1 arg2 = $@.

Another note is the use of a list of logs, how often does that change? My assumption is daily. Why not use the directory output as the array of your iterative loop? For example;

for i in $(ls /path/to/logs); do

  ./workaround.sh $i;

done

Or better yet, move on to use of functions in bash to eliminate clutter.

function process_file()

{

  # transfer file code/command

}



function iterate_dir()

{

  local -a dir=($(ls $1))

  for file in ${dir[@]}; do

    process_file $file

  done

}



iterate_dir /path/to/log/for

While these are merely suggestions to improve your shell scripting knowledge I must know if there is an error you are getting and would also need to know the details of each scripts code and or functionality. Making the use of the -x argument helps debug scripting as well.

If you are simply transferring logs you may wish to do away with the scripts all together and make use of rsync, rsyslog or syslog as they all are much more suited for the task in question.

inside the script : you need a read loop like while read ; do ......... ; done < filename to treat lines as $REPLY variable...

for example

while read 

do

mv $REPLY $REPLY.old

done < liste.txt

will rename any filename from inside liste.txt to filename.old

you have the structure now you can adapt to your needs depending on what you mean by " in each log file's name in list.txt to be executed." :)

xargs -n 1 ./script.sh <list.txt

In this example, xargs will execute ./script.sh multiple times with a single argument read from its standard input.

The standard input comes from the file list.txt via a simple shell input redirection. This assumes that the list.txt file contains one argument to use with the script per line.

The ./script.sh script will be executed once for each line in the list.txt file.

About the -n flag to xargs

-n number

Invoke utility using as many standard input arguments as
possible, up to number (a positive decimal integer) arguments
maximum.

Another solution would be to allow the script to read directly from list.txt, or to take multiple command line argument (the entries from list.txt), and to download the files in one or several batches. But since we don't know what the mechanics of the script is, I can't make any detailed explanation of how to go about doing this.

(Note: I personally think Kusalananda's approach is the best in this specific scenario, so I'll just add some explanatory information and recommendations)

Don't read lines with for

Using that approach you:

• Rely on word splitting to make things work.


• Are subject of unintended side effects like globbing.


• Slurp the entire file into memory all at once (may be a problem with big files).

The while + read approach is preferable but it is not a bullet-proof solution:

while IFS= read -r line; do

  # Your code here

done < file

Quote your variables

Write "$i" instead of $i. Unquoted variables are probably the main source of bugs and security holes in shell scripts.

Related questions:

• Why does my shell script choke on whitespace or other special characters?


• Security implications of forgetting to quote a variable in bash/POSIX shells

Use $(...) instead of `...`

$(...) is POSIX-compliant and can be nested easier.

You could also modify your script to support multiple arguments and even a --batch option.

If you don't know where to start, I have a sample/template script which supports those features (it uses Bash-specific syntax, though).

Example of usage:

$ cat list.txt

list_item_1

list_item_2

list_item_3

$ script --batch list.txt item_1 item_2

Operands:

  1: [list_item_1]

  2: [list_item_2]

  3: [list_item_3]

  4: [item_1]

  5: [item_2]