xargs with stdin/stdout redirection
I would like to run:
./a.out < x.dat > x.ans
for each *.dat file in the directory A.
Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):
ls A/*.dat | xargs -I file -a file ./a.out
But -a in xargs doesn't understand replace-str 'file'.
Thank you for help.
io-redirection xargs
add a comment |
I would like to run:
./a.out < x.dat > x.ans
for each *.dat file in the directory A.
Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):
ls A/*.dat | xargs -I file -a file ./a.out
But -a in xargs doesn't understand replace-str 'file'.
Thank you for help.
io-redirection xargs
In addition to the other good answers here, you can use the -a option of GNU xargs.
– James Youngman
Dec 9 '13 at 23:22
add a comment |
I would like to run:
./a.out < x.dat > x.ans
for each *.dat file in the directory A.
Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):
ls A/*.dat | xargs -I file -a file ./a.out
But -a in xargs doesn't understand replace-str 'file'.
Thank you for help.
io-redirection xargs
I would like to run:
./a.out < x.dat > x.ans
for each *.dat file in the directory A.
Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):
ls A/*.dat | xargs -I file -a file ./a.out
But -a in xargs doesn't understand replace-str 'file'.
Thank you for help.
io-redirection xargs
io-redirection xargs
asked Oct 7 '11 at 8:21
Nikolay VyahhiNikolay Vyahhi
203126
203126
In addition to the other good answers here, you can use the -a option of GNU xargs.
– James Youngman
Dec 9 '13 at 23:22
add a comment |
In addition to the other good answers here, you can use the -a option of GNU xargs.
– James Youngman
Dec 9 '13 at 23:22
In addition to the other good answers here, you can use the -a option of GNU xargs.
– James Youngman
Dec 9 '13 at 23:22
In addition to the other good answers here, you can use the -a option of GNU xargs.
– James Youngman
Dec 9 '13 at 23:22
add a comment |
6 Answers
6
active
oldest
votes
First of all, do not use ls
output as a file list. Use shell expansion or find
. See below for potential consequences of ls+xargs misuse and an example of proper xargs
usage.
1. Simple way: for loop
If you want to process just the files under A/
, then a simple for
loop should be enough:
for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done
2.pre1 Why not ls | xargs
?
Here's an example of how bad things may turn if you use ls
with xargs
for the job. Consider a following scenario:
first, let's create some empty files:
$ touch A/mypreciousfile.dat with junk at the end.dat
$ touch A/mypreciousfile.dat
$ touch A/mypreciousfile.dat.ans
see the files and that they contain nothing:
$ ls -1 A/
mypreciousfile.dat
mypreciousfile.dat with junk at the end.dat
mypreciousfile.dat.ans
$ cat A/*
run a magic command using
xargs
:
$ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"
the result:
$ cat A/mypreciousfile.dat
TRICKED with junk at the end.dat.ans
$ cat A/mypreciousfile.dat.ans
TRICKED
So you've just managed to overwrite both mypreciousfile.dat
and mypreciousfile.dat.ans
. If there were any content in those files, it'd have been erased.
2. Using xargs
: the proper way with find
If you'd like to insist on using xargs
, use -0
(null-terminated names) :
find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'
Notice two things:
- this way you'll create files with
.dat.ans
ending; - this will break if some file name contains a quote sign (
"
).
Both issues can be solved by different way of shell invocation:
find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'
3. All done within find ... -exec
find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;
This, again, produces .dat.ans
files and will break if file names contain "
. To go about that, use bash
and change the way it is invoked:
find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;
+1 for mentionning not to parse the output of ls.
– rahmu
Oct 7 '11 at 9:31
2
Option 2 breaks when the filenames contains ".
– thiton
Oct 7 '11 at 11:30
2
Very good point, thanks! I'll update accordingly.
– rozcietrzewiacz
Oct 7 '11 at 11:39
I want just mention, that ifzsh
is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivialfor file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done
works in all cases.
– jofel
Mar 20 '12 at 19:56
add a comment |
Try doing something like this (syntax may vary a bit depending on the shell you use):
$ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done
That would not work. It would try to operate on stuff likesomefile.dat.dat
and redirect all output to a single file.
– rozcietrzewiacz
Oct 7 '11 at 8:54
You're right. I edited the solution to correct it.
– rahmu
Oct 7 '11 at 8:57
OK - Almost good :) Justsomefile.dat.ans
output stuff would look not so nice.
– rozcietrzewiacz
Oct 7 '11 at 9:07
1
Edited! I did not know about '%'. It works like a charm, thanks for the tip.
– rahmu
Oct 7 '11 at 9:23
1
Adding a-type file
would be nice (can't< directory
), and this makes the unusual-filename-fairy sad.
– Mat
Oct 7 '11 at 9:57
|
show 1 more comment
For simple patterns, the for loop is appropriate:
for file in A/*.dat; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:
find A -name '*.dat' -print | while IFS= read -r file; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0
option to read so in those you can handle all characters, but unfortunately it's not POSIX.
add a comment |
Use GNU Parallel:
parallel ./a.out "<{} >{.}.ans" ::: A/*.dat
Added bonus: You get the processing done in parallel.
Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ
add a comment |
I think you need at least a shell invocation in the xargs:
ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"
Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.
Have a look at why it is a bad idea to parsels
output.
– rozcietrzewiacz
Oct 7 '11 at 9:13
@rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.
– thiton
Oct 7 '11 at 9:19
1
You are wrong about the shell expansion.ls
outputs things like spaces without escaping and that is the problem.
– rozcietrzewiacz
Oct 7 '11 at 9:25
@rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.
– thiton
Oct 7 '11 at 10:19
Yes. You see now, parsingls
is no good. Butxargs
can be safely used with find - see the suggestion No 2 in my answer.
– rozcietrzewiacz
Oct 7 '11 at 10:53
add a comment |
There's no need to complicate it. You could do it with a for
loop:
for file in A/*.dat; do
./a.out < "$file" >"${file%.dat}.ans"
done
the ${file%.dat}.ans
bit will remove the .dat
filename suffix from the filename in $file
and instead add .ans
to the end.
add a comment |
Your Answer
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "106"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: false,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: null,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f22229%2fxargs-with-stdin-stdout-redirection%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
6 Answers
6
active
oldest
votes
6 Answers
6
active
oldest
votes
active
oldest
votes
active
oldest
votes
First of all, do not use ls
output as a file list. Use shell expansion or find
. See below for potential consequences of ls+xargs misuse and an example of proper xargs
usage.
1. Simple way: for loop
If you want to process just the files under A/
, then a simple for
loop should be enough:
for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done
2.pre1 Why not ls | xargs
?
Here's an example of how bad things may turn if you use ls
with xargs
for the job. Consider a following scenario:
first, let's create some empty files:
$ touch A/mypreciousfile.dat with junk at the end.dat
$ touch A/mypreciousfile.dat
$ touch A/mypreciousfile.dat.ans
see the files and that they contain nothing:
$ ls -1 A/
mypreciousfile.dat
mypreciousfile.dat with junk at the end.dat
mypreciousfile.dat.ans
$ cat A/*
run a magic command using
xargs
:
$ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"
the result:
$ cat A/mypreciousfile.dat
TRICKED with junk at the end.dat.ans
$ cat A/mypreciousfile.dat.ans
TRICKED
So you've just managed to overwrite both mypreciousfile.dat
and mypreciousfile.dat.ans
. If there were any content in those files, it'd have been erased.
2. Using xargs
: the proper way with find
If you'd like to insist on using xargs
, use -0
(null-terminated names) :
find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'
Notice two things:
- this way you'll create files with
.dat.ans
ending; - this will break if some file name contains a quote sign (
"
).
Both issues can be solved by different way of shell invocation:
find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'
3. All done within find ... -exec
find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;
This, again, produces .dat.ans
files and will break if file names contain "
. To go about that, use bash
and change the way it is invoked:
find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;
+1 for mentionning not to parse the output of ls.
– rahmu
Oct 7 '11 at 9:31
2
Option 2 breaks when the filenames contains ".
– thiton
Oct 7 '11 at 11:30
2
Very good point, thanks! I'll update accordingly.
– rozcietrzewiacz
Oct 7 '11 at 11:39
I want just mention, that ifzsh
is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivialfor file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done
works in all cases.
– jofel
Mar 20 '12 at 19:56
add a comment |
First of all, do not use ls
output as a file list. Use shell expansion or find
. See below for potential consequences of ls+xargs misuse and an example of proper xargs
usage.
1. Simple way: for loop
If you want to process just the files under A/
, then a simple for
loop should be enough:
for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done
2.pre1 Why not ls | xargs
?
Here's an example of how bad things may turn if you use ls
with xargs
for the job. Consider a following scenario:
first, let's create some empty files:
$ touch A/mypreciousfile.dat with junk at the end.dat
$ touch A/mypreciousfile.dat
$ touch A/mypreciousfile.dat.ans
see the files and that they contain nothing:
$ ls -1 A/
mypreciousfile.dat
mypreciousfile.dat with junk at the end.dat
mypreciousfile.dat.ans
$ cat A/*
run a magic command using
xargs
:
$ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"
the result:
$ cat A/mypreciousfile.dat
TRICKED with junk at the end.dat.ans
$ cat A/mypreciousfile.dat.ans
TRICKED
So you've just managed to overwrite both mypreciousfile.dat
and mypreciousfile.dat.ans
. If there were any content in those files, it'd have been erased.
2. Using xargs
: the proper way with find
If you'd like to insist on using xargs
, use -0
(null-terminated names) :
find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'
Notice two things:
- this way you'll create files with
.dat.ans
ending; - this will break if some file name contains a quote sign (
"
).
Both issues can be solved by different way of shell invocation:
find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'
3. All done within find ... -exec
find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;
This, again, produces .dat.ans
files and will break if file names contain "
. To go about that, use bash
and change the way it is invoked:
find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;
+1 for mentionning not to parse the output of ls.
– rahmu
Oct 7 '11 at 9:31
2
Option 2 breaks when the filenames contains ".
– thiton
Oct 7 '11 at 11:30
2
Very good point, thanks! I'll update accordingly.
– rozcietrzewiacz
Oct 7 '11 at 11:39
I want just mention, that ifzsh
is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivialfor file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done
works in all cases.
– jofel
Mar 20 '12 at 19:56
add a comment |
First of all, do not use ls
output as a file list. Use shell expansion or find
. See below for potential consequences of ls+xargs misuse and an example of proper xargs
usage.
1. Simple way: for loop
If you want to process just the files under A/
, then a simple for
loop should be enough:
for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done
2.pre1 Why not ls | xargs
?
Here's an example of how bad things may turn if you use ls
with xargs
for the job. Consider a following scenario:
first, let's create some empty files:
$ touch A/mypreciousfile.dat with junk at the end.dat
$ touch A/mypreciousfile.dat
$ touch A/mypreciousfile.dat.ans
see the files and that they contain nothing:
$ ls -1 A/
mypreciousfile.dat
mypreciousfile.dat with junk at the end.dat
mypreciousfile.dat.ans
$ cat A/*
run a magic command using
xargs
:
$ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"
the result:
$ cat A/mypreciousfile.dat
TRICKED with junk at the end.dat.ans
$ cat A/mypreciousfile.dat.ans
TRICKED
So you've just managed to overwrite both mypreciousfile.dat
and mypreciousfile.dat.ans
. If there were any content in those files, it'd have been erased.
2. Using xargs
: the proper way with find
If you'd like to insist on using xargs
, use -0
(null-terminated names) :
find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'
Notice two things:
- this way you'll create files with
.dat.ans
ending; - this will break if some file name contains a quote sign (
"
).
Both issues can be solved by different way of shell invocation:
find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'
3. All done within find ... -exec
find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;
This, again, produces .dat.ans
files and will break if file names contain "
. To go about that, use bash
and change the way it is invoked:
find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;
First of all, do not use ls
output as a file list. Use shell expansion or find
. See below for potential consequences of ls+xargs misuse and an example of proper xargs
usage.
1. Simple way: for loop
If you want to process just the files under A/
, then a simple for
loop should be enough:
for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done
2.pre1 Why not ls | xargs
?
Here's an example of how bad things may turn if you use ls
with xargs
for the job. Consider a following scenario:
first, let's create some empty files:
$ touch A/mypreciousfile.dat with junk at the end.dat
$ touch A/mypreciousfile.dat
$ touch A/mypreciousfile.dat.ans
see the files and that they contain nothing:
$ ls -1 A/
mypreciousfile.dat
mypreciousfile.dat with junk at the end.dat
mypreciousfile.dat.ans
$ cat A/*
run a magic command using
xargs
:
$ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"
the result:
$ cat A/mypreciousfile.dat
TRICKED with junk at the end.dat.ans
$ cat A/mypreciousfile.dat.ans
TRICKED
So you've just managed to overwrite both mypreciousfile.dat
and mypreciousfile.dat.ans
. If there were any content in those files, it'd have been erased.
2. Using xargs
: the proper way with find
If you'd like to insist on using xargs
, use -0
(null-terminated names) :
find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'
Notice two things:
- this way you'll create files with
.dat.ans
ending; - this will break if some file name contains a quote sign (
"
).
Both issues can be solved by different way of shell invocation:
find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'
3. All done within find ... -exec
find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;
This, again, produces .dat.ans
files and will break if file names contain "
. To go about that, use bash
and change the way it is invoked:
find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;
edited 1 hour ago
iTakeshi
31
31
answered Oct 7 '11 at 8:50
rozcietrzewiaczrozcietrzewiacz
29.5k47392
29.5k47392
+1 for mentionning not to parse the output of ls.
– rahmu
Oct 7 '11 at 9:31
2
Option 2 breaks when the filenames contains ".
– thiton
Oct 7 '11 at 11:30
2
Very good point, thanks! I'll update accordingly.
– rozcietrzewiacz
Oct 7 '11 at 11:39
I want just mention, that ifzsh
is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivialfor file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done
works in all cases.
– jofel
Mar 20 '12 at 19:56
add a comment |
+1 for mentionning not to parse the output of ls.
– rahmu
Oct 7 '11 at 9:31
2
Option 2 breaks when the filenames contains ".
– thiton
Oct 7 '11 at 11:30
2
Very good point, thanks! I'll update accordingly.
– rozcietrzewiacz
Oct 7 '11 at 11:39
I want just mention, that ifzsh
is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivialfor file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done
works in all cases.
– jofel
Mar 20 '12 at 19:56
+1 for mentionning not to parse the output of ls.
– rahmu
Oct 7 '11 at 9:31
+1 for mentionning not to parse the output of ls.
– rahmu
Oct 7 '11 at 9:31
2
2
Option 2 breaks when the filenames contains ".
– thiton
Oct 7 '11 at 11:30
Option 2 breaks when the filenames contains ".
– thiton
Oct 7 '11 at 11:30
2
2
Very good point, thanks! I'll update accordingly.
– rozcietrzewiacz
Oct 7 '11 at 11:39
Very good point, thanks! I'll update accordingly.
– rozcietrzewiacz
Oct 7 '11 at 11:39
I want just mention, that if
zsh
is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done
works in all cases.– jofel
Mar 20 '12 at 19:56
I want just mention, that if
zsh
is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done
works in all cases.– jofel
Mar 20 '12 at 19:56
add a comment |
Try doing something like this (syntax may vary a bit depending on the shell you use):
$ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done
That would not work. It would try to operate on stuff likesomefile.dat.dat
and redirect all output to a single file.
– rozcietrzewiacz
Oct 7 '11 at 8:54
You're right. I edited the solution to correct it.
– rahmu
Oct 7 '11 at 8:57
OK - Almost good :) Justsomefile.dat.ans
output stuff would look not so nice.
– rozcietrzewiacz
Oct 7 '11 at 9:07
1
Edited! I did not know about '%'. It works like a charm, thanks for the tip.
– rahmu
Oct 7 '11 at 9:23
1
Adding a-type file
would be nice (can't< directory
), and this makes the unusual-filename-fairy sad.
– Mat
Oct 7 '11 at 9:57
|
show 1 more comment
Try doing something like this (syntax may vary a bit depending on the shell you use):
$ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done
That would not work. It would try to operate on stuff likesomefile.dat.dat
and redirect all output to a single file.
– rozcietrzewiacz
Oct 7 '11 at 8:54
You're right. I edited the solution to correct it.
– rahmu
Oct 7 '11 at 8:57
OK - Almost good :) Justsomefile.dat.ans
output stuff would look not so nice.
– rozcietrzewiacz
Oct 7 '11 at 9:07
1
Edited! I did not know about '%'. It works like a charm, thanks for the tip.
– rahmu
Oct 7 '11 at 9:23
1
Adding a-type file
would be nice (can't< directory
), and this makes the unusual-filename-fairy sad.
– Mat
Oct 7 '11 at 9:57
|
show 1 more comment
Try doing something like this (syntax may vary a bit depending on the shell you use):
$ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done
Try doing something like this (syntax may vary a bit depending on the shell you use):
$ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done
edited Oct 7 '11 at 9:22
answered Oct 7 '11 at 8:47
rahmurahmu
10.5k2070112
10.5k2070112
That would not work. It would try to operate on stuff likesomefile.dat.dat
and redirect all output to a single file.
– rozcietrzewiacz
Oct 7 '11 at 8:54
You're right. I edited the solution to correct it.
– rahmu
Oct 7 '11 at 8:57
OK - Almost good :) Justsomefile.dat.ans
output stuff would look not so nice.
– rozcietrzewiacz
Oct 7 '11 at 9:07
1
Edited! I did not know about '%'. It works like a charm, thanks for the tip.
– rahmu
Oct 7 '11 at 9:23
1
Adding a-type file
would be nice (can't< directory
), and this makes the unusual-filename-fairy sad.
– Mat
Oct 7 '11 at 9:57
|
show 1 more comment
That would not work. It would try to operate on stuff likesomefile.dat.dat
and redirect all output to a single file.
– rozcietrzewiacz
Oct 7 '11 at 8:54
You're right. I edited the solution to correct it.
– rahmu
Oct 7 '11 at 8:57
OK - Almost good :) Justsomefile.dat.ans
output stuff would look not so nice.
– rozcietrzewiacz
Oct 7 '11 at 9:07
1
Edited! I did not know about '%'. It works like a charm, thanks for the tip.
– rahmu
Oct 7 '11 at 9:23
1
Adding a-type file
would be nice (can't< directory
), and this makes the unusual-filename-fairy sad.
– Mat
Oct 7 '11 at 9:57
That would not work. It would try to operate on stuff like
somefile.dat.dat
and redirect all output to a single file.– rozcietrzewiacz
Oct 7 '11 at 8:54
That would not work. It would try to operate on stuff like
somefile.dat.dat
and redirect all output to a single file.– rozcietrzewiacz
Oct 7 '11 at 8:54
You're right. I edited the solution to correct it.
– rahmu
Oct 7 '11 at 8:57
You're right. I edited the solution to correct it.
– rahmu
Oct 7 '11 at 8:57
OK - Almost good :) Just
somefile.dat.ans
output stuff would look not so nice.– rozcietrzewiacz
Oct 7 '11 at 9:07
OK - Almost good :) Just
somefile.dat.ans
output stuff would look not so nice.– rozcietrzewiacz
Oct 7 '11 at 9:07
1
1
Edited! I did not know about '%'. It works like a charm, thanks for the tip.
– rahmu
Oct 7 '11 at 9:23
Edited! I did not know about '%'. It works like a charm, thanks for the tip.
– rahmu
Oct 7 '11 at 9:23
1
1
Adding a
-type file
would be nice (can't < directory
), and this makes the unusual-filename-fairy sad.– Mat
Oct 7 '11 at 9:57
Adding a
-type file
would be nice (can't < directory
), and this makes the unusual-filename-fairy sad.– Mat
Oct 7 '11 at 9:57
|
show 1 more comment
For simple patterns, the for loop is appropriate:
for file in A/*.dat; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:
find A -name '*.dat' -print | while IFS= read -r file; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0
option to read so in those you can handle all characters, but unfortunately it's not POSIX.
add a comment |
For simple patterns, the for loop is appropriate:
for file in A/*.dat; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:
find A -name '*.dat' -print | while IFS= read -r file; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0
option to read so in those you can handle all characters, but unfortunately it's not POSIX.
add a comment |
For simple patterns, the for loop is appropriate:
for file in A/*.dat; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:
find A -name '*.dat' -print | while IFS= read -r file; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0
option to read so in those you can handle all characters, but unfortunately it's not POSIX.
For simple patterns, the for loop is appropriate:
for file in A/*.dat; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:
find A -name '*.dat' -print | while IFS= read -r file; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done
This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0
option to read so in those you can handle all characters, but unfortunately it's not POSIX.
edited Oct 7 '11 at 23:36
Gilles
544k12811021619
544k12811021619
answered Oct 7 '11 at 12:20
Jan HudecJan Hudec
27519
27519
add a comment |
add a comment |
Use GNU Parallel:
parallel ./a.out "<{} >{.}.ans" ::: A/*.dat
Added bonus: You get the processing done in parallel.
Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ
add a comment |
Use GNU Parallel:
parallel ./a.out "<{} >{.}.ans" ::: A/*.dat
Added bonus: You get the processing done in parallel.
Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ
add a comment |
Use GNU Parallel:
parallel ./a.out "<{} >{.}.ans" ::: A/*.dat
Added bonus: You get the processing done in parallel.
Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ
Use GNU Parallel:
parallel ./a.out "<{} >{.}.ans" ::: A/*.dat
Added bonus: You get the processing done in parallel.
Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ
answered Oct 10 '11 at 13:34
Ole TangeOle Tange
12.8k1456107
12.8k1456107
add a comment |
add a comment |
I think you need at least a shell invocation in the xargs:
ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"
Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.
Have a look at why it is a bad idea to parsels
output.
– rozcietrzewiacz
Oct 7 '11 at 9:13
@rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.
– thiton
Oct 7 '11 at 9:19
1
You are wrong about the shell expansion.ls
outputs things like spaces without escaping and that is the problem.
– rozcietrzewiacz
Oct 7 '11 at 9:25
@rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.
– thiton
Oct 7 '11 at 10:19
Yes. You see now, parsingls
is no good. Butxargs
can be safely used with find - see the suggestion No 2 in my answer.
– rozcietrzewiacz
Oct 7 '11 at 10:53
add a comment |
I think you need at least a shell invocation in the xargs:
ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"
Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.
Have a look at why it is a bad idea to parsels
output.
– rozcietrzewiacz
Oct 7 '11 at 9:13
@rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.
– thiton
Oct 7 '11 at 9:19
1
You are wrong about the shell expansion.ls
outputs things like spaces without escaping and that is the problem.
– rozcietrzewiacz
Oct 7 '11 at 9:25
@rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.
– thiton
Oct 7 '11 at 10:19
Yes. You see now, parsingls
is no good. Butxargs
can be safely used with find - see the suggestion No 2 in my answer.
– rozcietrzewiacz
Oct 7 '11 at 10:53
add a comment |
I think you need at least a shell invocation in the xargs:
ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"
Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.
I think you need at least a shell invocation in the xargs:
ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"
Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.
edited Oct 7 '11 at 10:23
answered Oct 7 '11 at 8:58
thitonthiton
1,44899
1,44899
Have a look at why it is a bad idea to parsels
output.
– rozcietrzewiacz
Oct 7 '11 at 9:13
@rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.
– thiton
Oct 7 '11 at 9:19
1
You are wrong about the shell expansion.ls
outputs things like spaces without escaping and that is the problem.
– rozcietrzewiacz
Oct 7 '11 at 9:25
@rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.
– thiton
Oct 7 '11 at 10:19
Yes. You see now, parsingls
is no good. Butxargs
can be safely used with find - see the suggestion No 2 in my answer.
– rozcietrzewiacz
Oct 7 '11 at 10:53
add a comment |
Have a look at why it is a bad idea to parsels
output.
– rozcietrzewiacz
Oct 7 '11 at 9:13
@rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.
– thiton
Oct 7 '11 at 9:19
1
You are wrong about the shell expansion.ls
outputs things like spaces without escaping and that is the problem.
– rozcietrzewiacz
Oct 7 '11 at 9:25
@rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.
– thiton
Oct 7 '11 at 10:19
Yes. You see now, parsingls
is no good. Butxargs
can be safely used with find - see the suggestion No 2 in my answer.
– rozcietrzewiacz
Oct 7 '11 at 10:53
Have a look at why it is a bad idea to parse
ls
output.– rozcietrzewiacz
Oct 7 '11 at 9:13
Have a look at why it is a bad idea to parse
ls
output.– rozcietrzewiacz
Oct 7 '11 at 9:13
@rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.
– thiton
Oct 7 '11 at 9:19
@rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.
– thiton
Oct 7 '11 at 9:19
1
1
You are wrong about the shell expansion.
ls
outputs things like spaces without escaping and that is the problem.– rozcietrzewiacz
Oct 7 '11 at 9:25
You are wrong about the shell expansion.
ls
outputs things like spaces without escaping and that is the problem.– rozcietrzewiacz
Oct 7 '11 at 9:25
@rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.
– thiton
Oct 7 '11 at 10:19
@rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.
– thiton
Oct 7 '11 at 10:19
Yes. You see now, parsing
ls
is no good. But xargs
can be safely used with find - see the suggestion No 2 in my answer.– rozcietrzewiacz
Oct 7 '11 at 10:53
Yes. You see now, parsing
ls
is no good. But xargs
can be safely used with find - see the suggestion No 2 in my answer.– rozcietrzewiacz
Oct 7 '11 at 10:53
add a comment |
There's no need to complicate it. You could do it with a for
loop:
for file in A/*.dat; do
./a.out < "$file" >"${file%.dat}.ans"
done
the ${file%.dat}.ans
bit will remove the .dat
filename suffix from the filename in $file
and instead add .ans
to the end.
add a comment |
There's no need to complicate it. You could do it with a for
loop:
for file in A/*.dat; do
./a.out < "$file" >"${file%.dat}.ans"
done
the ${file%.dat}.ans
bit will remove the .dat
filename suffix from the filename in $file
and instead add .ans
to the end.
add a comment |
There's no need to complicate it. You could do it with a for
loop:
for file in A/*.dat; do
./a.out < "$file" >"${file%.dat}.ans"
done
the ${file%.dat}.ans
bit will remove the .dat
filename suffix from the filename in $file
and instead add .ans
to the end.
There's no need to complicate it. You could do it with a for
loop:
for file in A/*.dat; do
./a.out < "$file" >"${file%.dat}.ans"
done
the ${file%.dat}.ans
bit will remove the .dat
filename suffix from the filename in $file
and instead add .ans
to the end.
edited Apr 12 '18 at 20:43
answered Oct 7 '11 at 8:52
KusalanandaKusalananda
137k17258426
137k17258426
add a comment |
add a comment |
Thanks for contributing an answer to Unix & Linux Stack Exchange!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f22229%2fxargs-with-stdin-stdout-redirection%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
In addition to the other good answers here, you can use the -a option of GNU xargs.
– James Youngman
Dec 9 '13 at 23:22