xargs with stdin/stdout redirection












20















I would like to run:



./a.out < x.dat > x.ans


for each *.dat file in the directory A.



Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):



ls A/*.dat | xargs -I file -a file ./a.out


But -a in xargs doesn't understand replace-str 'file'.



Thank you for help.










share|improve this question























  • In addition to the other good answers here, you can use the -a option of GNU xargs.

    – James Youngman
    Dec 9 '13 at 23:22
















20















I would like to run:



./a.out < x.dat > x.ans


for each *.dat file in the directory A.



Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):



ls A/*.dat | xargs -I file -a file ./a.out


But -a in xargs doesn't understand replace-str 'file'.



Thank you for help.










share|improve this question























  • In addition to the other good answers here, you can use the -a option of GNU xargs.

    – James Youngman
    Dec 9 '13 at 23:22














20












20








20


8






I would like to run:



./a.out < x.dat > x.ans


for each *.dat file in the directory A.



Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):



ls A/*.dat | xargs -I file -a file ./a.out


But -a in xargs doesn't understand replace-str 'file'.



Thank you for help.










share|improve this question














I would like to run:



./a.out < x.dat > x.ans


for each *.dat file in the directory A.



Sure, it could be done by bash/python/whatsoever script, but I like to write sexy one-liner. All I could reach is (still without any stdout):



ls A/*.dat | xargs -I file -a file ./a.out


But -a in xargs doesn't understand replace-str 'file'.



Thank you for help.







io-redirection xargs






share|improve this question













share|improve this question











share|improve this question




share|improve this question










asked Oct 7 '11 at 8:21









Nikolay VyahhiNikolay Vyahhi

203126




203126













  • In addition to the other good answers here, you can use the -a option of GNU xargs.

    – James Youngman
    Dec 9 '13 at 23:22



















  • In addition to the other good answers here, you can use the -a option of GNU xargs.

    – James Youngman
    Dec 9 '13 at 23:22

















In addition to the other good answers here, you can use the -a option of GNU xargs.

– James Youngman
Dec 9 '13 at 23:22





In addition to the other good answers here, you can use the -a option of GNU xargs.

– James Youngman
Dec 9 '13 at 23:22










6 Answers
6






active

oldest

votes


















28














First of all, do not use ls output as a file list. Use shell expansion or find. See below for potential consequences of ls+xargs misuse and an example of proper xargs usage.



1. Simple way: for loop



If you want to process just the files under A/, then a simple for loop should be enough:



for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done




2.pre1 Why not   ls | xargs ?



Here's an example of how bad things may turn if you use ls with xargs for the job. Consider a following scenario:





  • first, let's create some empty files:



    $ touch A/mypreciousfile.dat with junk at the end.dat
    $ touch A/mypreciousfile.dat
    $ touch A/mypreciousfile.dat.ans



  • see the files and that they contain nothing:



    $ ls -1 A/
    mypreciousfile.dat
    mypreciousfile.dat with junk at the end.dat
    mypreciousfile.dat.ans

    $ cat A/*



  • run a magic command using xargs:



    $ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"



  • the result:



    $ cat A/mypreciousfile.dat
    TRICKED with junk at the end.dat.ans

    $ cat A/mypreciousfile.dat.ans
    TRICKED



So you've just managed to overwrite both mypreciousfile.dat and mypreciousfile.dat.ans. If there were any content in those files, it'd have been erased.





2. Using  xargs : the proper way with  find 



If you'd like to insist on using xargs, use -0 (null-terminated names) :



find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'


Notice two things:




  1. this way you'll create files with .dat.ans ending;

  2. this will break if some file name contains a quote sign (").


Both issues can be solved by different way of shell invocation:



find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'




3. All done within find ... -exec



 find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;


This, again, produces .dat.ans files and will break if file names contain ". To go about that, use bash and change the way it is invoked:



 find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;





share|improve this answer


























  • +1 for mentionning not to parse the output of ls.

    – rahmu
    Oct 7 '11 at 9:31






  • 2





    Option 2 breaks when the filenames contains ".

    – thiton
    Oct 7 '11 at 11:30






  • 2





    Very good point, thanks! I'll update accordingly.

    – rozcietrzewiacz
    Oct 7 '11 at 11:39











  • I want just mention, that if zsh is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done works in all cases.

    – jofel
    Mar 20 '12 at 19:56





















2














Try doing something like this (syntax may vary a bit depending on the shell you use):



$ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done






share|improve this answer


























  • That would not work. It would try to operate on stuff like somefile.dat.dat and redirect all output to a single file.

    – rozcietrzewiacz
    Oct 7 '11 at 8:54











  • You're right. I edited the solution to correct it.

    – rahmu
    Oct 7 '11 at 8:57











  • OK - Almost good :) Just somefile.dat.ans output stuff would look not so nice.

    – rozcietrzewiacz
    Oct 7 '11 at 9:07






  • 1





    Edited! I did not know about '%'. It works like a charm, thanks for the tip.

    – rahmu
    Oct 7 '11 at 9:23






  • 1





    Adding a -type file would be nice (can't < directory), and this makes the unusual-filename-fairy sad.

    – Mat
    Oct 7 '11 at 9:57



















2














For simple patterns, the for loop is appropriate:



for file in A/*.dat; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done


For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:



find A -name '*.dat' -print | while IFS= read -r file; do
./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
done


This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0 option to read so in those you can handle all characters, but unfortunately it's not POSIX.






share|improve this answer

































    2














    Use GNU Parallel:



    parallel ./a.out "<{} >{.}.ans" ::: A/*.dat


    Added bonus: You get the processing done in parallel.



    Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ






    share|improve this answer































      1














      I think you need at least a shell invocation in the xargs:



      ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"


      Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.






      share|improve this answer


























      • Have a look at why it is a bad idea to parse ls output.

        – rozcietrzewiacz
        Oct 7 '11 at 9:13











      • @rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.

        – thiton
        Oct 7 '11 at 9:19






      • 1





        You are wrong about the shell expansion. ls outputs things like spaces without escaping and that is the problem.

        – rozcietrzewiacz
        Oct 7 '11 at 9:25











      • @rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.

        – thiton
        Oct 7 '11 at 10:19











      • Yes. You see now, parsing ls is no good. But xargs can be safely used with find - see the suggestion No 2 in my answer.

        – rozcietrzewiacz
        Oct 7 '11 at 10:53





















      1














      There's no need to complicate it. You could do it with a for loop:



      for file in A/*.dat; do
      ./a.out < "$file" >"${file%.dat}.ans"
      done


      the ${file%.dat}.ans bit will remove the .dat filename suffix from the filename in $file and instead add .ans to the end.






      share|improve this answer

























        Your Answer








        StackExchange.ready(function() {
        var channelOptions = {
        tags: "".split(" "),
        id: "106"
        };
        initTagRenderer("".split(" "), "".split(" "), channelOptions);

        StackExchange.using("externalEditor", function() {
        // Have to fire editor after snippets, if snippets enabled
        if (StackExchange.settings.snippets.snippetsEnabled) {
        StackExchange.using("snippets", function() {
        createEditor();
        });
        }
        else {
        createEditor();
        }
        });

        function createEditor() {
        StackExchange.prepareEditor({
        heartbeatType: 'answer',
        autoActivateHeartbeat: false,
        convertImagesToLinks: false,
        noModals: true,
        showLowRepImageUploadWarning: true,
        reputationToPostImages: null,
        bindNavPrevention: true,
        postfix: "",
        imageUploader: {
        brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
        contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
        allowUrls: true
        },
        onDemand: true,
        discardSelector: ".discard-answer"
        ,immediatelyShowMarkdownHelp:true
        });


        }
        });














        draft saved

        draft discarded


















        StackExchange.ready(
        function () {
        StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f22229%2fxargs-with-stdin-stdout-redirection%23new-answer', 'question_page');
        }
        );

        Post as a guest















        Required, but never shown

























        6 Answers
        6






        active

        oldest

        votes








        6 Answers
        6






        active

        oldest

        votes









        active

        oldest

        votes






        active

        oldest

        votes









        28














        First of all, do not use ls output as a file list. Use shell expansion or find. See below for potential consequences of ls+xargs misuse and an example of proper xargs usage.



        1. Simple way: for loop



        If you want to process just the files under A/, then a simple for loop should be enough:



        for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done




        2.pre1 Why not   ls | xargs ?



        Here's an example of how bad things may turn if you use ls with xargs for the job. Consider a following scenario:





        • first, let's create some empty files:



          $ touch A/mypreciousfile.dat with junk at the end.dat
          $ touch A/mypreciousfile.dat
          $ touch A/mypreciousfile.dat.ans



        • see the files and that they contain nothing:



          $ ls -1 A/
          mypreciousfile.dat
          mypreciousfile.dat with junk at the end.dat
          mypreciousfile.dat.ans

          $ cat A/*



        • run a magic command using xargs:



          $ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"



        • the result:



          $ cat A/mypreciousfile.dat
          TRICKED with junk at the end.dat.ans

          $ cat A/mypreciousfile.dat.ans
          TRICKED



        So you've just managed to overwrite both mypreciousfile.dat and mypreciousfile.dat.ans. If there were any content in those files, it'd have been erased.





        2. Using  xargs : the proper way with  find 



        If you'd like to insist on using xargs, use -0 (null-terminated names) :



        find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'


        Notice two things:




        1. this way you'll create files with .dat.ans ending;

        2. this will break if some file name contains a quote sign (").


        Both issues can be solved by different way of shell invocation:



        find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'




        3. All done within find ... -exec



         find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;


        This, again, produces .dat.ans files and will break if file names contain ". To go about that, use bash and change the way it is invoked:



         find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;





        share|improve this answer


























        • +1 for mentionning not to parse the output of ls.

          – rahmu
          Oct 7 '11 at 9:31






        • 2





          Option 2 breaks when the filenames contains ".

          – thiton
          Oct 7 '11 at 11:30






        • 2





          Very good point, thanks! I'll update accordingly.

          – rozcietrzewiacz
          Oct 7 '11 at 11:39











        • I want just mention, that if zsh is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done works in all cases.

          – jofel
          Mar 20 '12 at 19:56


















        28














        First of all, do not use ls output as a file list. Use shell expansion or find. See below for potential consequences of ls+xargs misuse and an example of proper xargs usage.



        1. Simple way: for loop



        If you want to process just the files under A/, then a simple for loop should be enough:



        for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done




        2.pre1 Why not   ls | xargs ?



        Here's an example of how bad things may turn if you use ls with xargs for the job. Consider a following scenario:





        • first, let's create some empty files:



          $ touch A/mypreciousfile.dat with junk at the end.dat
          $ touch A/mypreciousfile.dat
          $ touch A/mypreciousfile.dat.ans



        • see the files and that they contain nothing:



          $ ls -1 A/
          mypreciousfile.dat
          mypreciousfile.dat with junk at the end.dat
          mypreciousfile.dat.ans

          $ cat A/*



        • run a magic command using xargs:



          $ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"



        • the result:



          $ cat A/mypreciousfile.dat
          TRICKED with junk at the end.dat.ans

          $ cat A/mypreciousfile.dat.ans
          TRICKED



        So you've just managed to overwrite both mypreciousfile.dat and mypreciousfile.dat.ans. If there were any content in those files, it'd have been erased.





        2. Using  xargs : the proper way with  find 



        If you'd like to insist on using xargs, use -0 (null-terminated names) :



        find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'


        Notice two things:




        1. this way you'll create files with .dat.ans ending;

        2. this will break if some file name contains a quote sign (").


        Both issues can be solved by different way of shell invocation:



        find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'




        3. All done within find ... -exec



         find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;


        This, again, produces .dat.ans files and will break if file names contain ". To go about that, use bash and change the way it is invoked:



         find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;





        share|improve this answer


























        • +1 for mentionning not to parse the output of ls.

          – rahmu
          Oct 7 '11 at 9:31






        • 2





          Option 2 breaks when the filenames contains ".

          – thiton
          Oct 7 '11 at 11:30






        • 2





          Very good point, thanks! I'll update accordingly.

          – rozcietrzewiacz
          Oct 7 '11 at 11:39











        • I want just mention, that if zsh is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done works in all cases.

          – jofel
          Mar 20 '12 at 19:56
















        28












        28








        28







        First of all, do not use ls output as a file list. Use shell expansion or find. See below for potential consequences of ls+xargs misuse and an example of proper xargs usage.



        1. Simple way: for loop



        If you want to process just the files under A/, then a simple for loop should be enough:



        for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done




        2.pre1 Why not   ls | xargs ?



        Here's an example of how bad things may turn if you use ls with xargs for the job. Consider a following scenario:





        • first, let's create some empty files:



          $ touch A/mypreciousfile.dat with junk at the end.dat
          $ touch A/mypreciousfile.dat
          $ touch A/mypreciousfile.dat.ans



        • see the files and that they contain nothing:



          $ ls -1 A/
          mypreciousfile.dat
          mypreciousfile.dat with junk at the end.dat
          mypreciousfile.dat.ans

          $ cat A/*



        • run a magic command using xargs:



          $ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"



        • the result:



          $ cat A/mypreciousfile.dat
          TRICKED with junk at the end.dat.ans

          $ cat A/mypreciousfile.dat.ans
          TRICKED



        So you've just managed to overwrite both mypreciousfile.dat and mypreciousfile.dat.ans. If there were any content in those files, it'd have been erased.





        2. Using  xargs : the proper way with  find 



        If you'd like to insist on using xargs, use -0 (null-terminated names) :



        find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'


        Notice two things:




        1. this way you'll create files with .dat.ans ending;

        2. this will break if some file name contains a quote sign (").


        Both issues can be solved by different way of shell invocation:



        find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'




        3. All done within find ... -exec



         find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;


        This, again, produces .dat.ans files and will break if file names contain ". To go about that, use bash and change the way it is invoked:



         find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;





        share|improve this answer















        First of all, do not use ls output as a file list. Use shell expansion or find. See below for potential consequences of ls+xargs misuse and an example of proper xargs usage.



        1. Simple way: for loop



        If you want to process just the files under A/, then a simple for loop should be enough:



        for file in A/*.dat; do ./a.out < "$file" > "${file%.dat}.ans"; done




        2.pre1 Why not   ls | xargs ?



        Here's an example of how bad things may turn if you use ls with xargs for the job. Consider a following scenario:





        • first, let's create some empty files:



          $ touch A/mypreciousfile.dat with junk at the end.dat
          $ touch A/mypreciousfile.dat
          $ touch A/mypreciousfile.dat.ans



        • see the files and that they contain nothing:



          $ ls -1 A/
          mypreciousfile.dat
          mypreciousfile.dat with junk at the end.dat
          mypreciousfile.dat.ans

          $ cat A/*



        • run a magic command using xargs:



          $ ls A/*.dat | xargs -I file sh -c "echo TRICKED > file.ans"



        • the result:



          $ cat A/mypreciousfile.dat
          TRICKED with junk at the end.dat.ans

          $ cat A/mypreciousfile.dat.ans
          TRICKED



        So you've just managed to overwrite both mypreciousfile.dat and mypreciousfile.dat.ans. If there were any content in those files, it'd have been erased.





        2. Using  xargs : the proper way with  find 



        If you'd like to insist on using xargs, use -0 (null-terminated names) :



        find A/ -name "*.dat" -type f -print0 | xargs -0 -I file sh -c './a.out < "file" > "file.ans"'


        Notice two things:




        1. this way you'll create files with .dat.ans ending;

        2. this will break if some file name contains a quote sign (").


        Both issues can be solved by different way of shell invocation:



        find A/ -name "*.dat" -type f -print0 | xargs -0 -L 1 bash -c './a.out < "$0" > "${0%dat}ans"'




        3. All done within find ... -exec



         find A/ -name "*.dat" -type f -exec sh -c './a.out < "{}" > "{}.ans"' ;


        This, again, produces .dat.ans files and will break if file names contain ". To go about that, use bash and change the way it is invoked:



         find A/ -name "*.dat" -type f -exec bash -c './a.out < "$0" > "${0%dat}ans"' {} ;






        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited 1 hour ago









        iTakeshi

        31




        31










        answered Oct 7 '11 at 8:50









        rozcietrzewiaczrozcietrzewiacz

        29.5k47392




        29.5k47392













        • +1 for mentionning not to parse the output of ls.

          – rahmu
          Oct 7 '11 at 9:31






        • 2





          Option 2 breaks when the filenames contains ".

          – thiton
          Oct 7 '11 at 11:30






        • 2





          Very good point, thanks! I'll update accordingly.

          – rozcietrzewiacz
          Oct 7 '11 at 11:39











        • I want just mention, that if zsh is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done works in all cases.

          – jofel
          Mar 20 '12 at 19:56





















        • +1 for mentionning not to parse the output of ls.

          – rahmu
          Oct 7 '11 at 9:31






        • 2





          Option 2 breaks when the filenames contains ".

          – thiton
          Oct 7 '11 at 11:30






        • 2





          Very good point, thanks! I'll update accordingly.

          – rozcietrzewiacz
          Oct 7 '11 at 11:39











        • I want just mention, that if zsh is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done works in all cases.

          – jofel
          Mar 20 '12 at 19:56



















        +1 for mentionning not to parse the output of ls.

        – rahmu
        Oct 7 '11 at 9:31





        +1 for mentionning not to parse the output of ls.

        – rahmu
        Oct 7 '11 at 9:31




        2




        2





        Option 2 breaks when the filenames contains ".

        – thiton
        Oct 7 '11 at 11:30





        Option 2 breaks when the filenames contains ".

        – thiton
        Oct 7 '11 at 11:30




        2




        2





        Very good point, thanks! I'll update accordingly.

        – rozcietrzewiacz
        Oct 7 '11 at 11:39





        Very good point, thanks! I'll update accordingly.

        – rozcietrzewiacz
        Oct 7 '11 at 11:39













        I want just mention, that if zsh is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done works in all cases.

        – jofel
        Mar 20 '12 at 19:56







        I want just mention, that if zsh is used as shell (and SH_WORD_SPLIT is not set), all the nasty special cases (white spaces, " in the filename etc.) need not to be considered. The trivial for file in A/*.dat; do ./a.out < $file > ${file%.dat}.ans ; done works in all cases.

        – jofel
        Mar 20 '12 at 19:56















        2














        Try doing something like this (syntax may vary a bit depending on the shell you use):



        $ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done






        share|improve this answer


























        • That would not work. It would try to operate on stuff like somefile.dat.dat and redirect all output to a single file.

          – rozcietrzewiacz
          Oct 7 '11 at 8:54











        • You're right. I edited the solution to correct it.

          – rahmu
          Oct 7 '11 at 8:57











        • OK - Almost good :) Just somefile.dat.ans output stuff would look not so nice.

          – rozcietrzewiacz
          Oct 7 '11 at 9:07






        • 1





          Edited! I did not know about '%'. It works like a charm, thanks for the tip.

          – rahmu
          Oct 7 '11 at 9:23






        • 1





          Adding a -type file would be nice (can't < directory), and this makes the unusual-filename-fairy sad.

          – Mat
          Oct 7 '11 at 9:57
















        2














        Try doing something like this (syntax may vary a bit depending on the shell you use):



        $ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done






        share|improve this answer


























        • That would not work. It would try to operate on stuff like somefile.dat.dat and redirect all output to a single file.

          – rozcietrzewiacz
          Oct 7 '11 at 8:54











        • You're right. I edited the solution to correct it.

          – rahmu
          Oct 7 '11 at 8:57











        • OK - Almost good :) Just somefile.dat.ans output stuff would look not so nice.

          – rozcietrzewiacz
          Oct 7 '11 at 9:07






        • 1





          Edited! I did not know about '%'. It works like a charm, thanks for the tip.

          – rahmu
          Oct 7 '11 at 9:23






        • 1





          Adding a -type file would be nice (can't < directory), and this makes the unusual-filename-fairy sad.

          – Mat
          Oct 7 '11 at 9:57














        2












        2








        2







        Try doing something like this (syntax may vary a bit depending on the shell you use):



        $ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done






        share|improve this answer















        Try doing something like this (syntax may vary a bit depending on the shell you use):



        $ for i in $(find A/ -name *.dat); do ./a.out < ${i} > ${i%.dat}.ans; done







        share|improve this answer














        share|improve this answer



        share|improve this answer








        edited Oct 7 '11 at 9:22

























        answered Oct 7 '11 at 8:47









        rahmurahmu

        10.5k2070112




        10.5k2070112













        • That would not work. It would try to operate on stuff like somefile.dat.dat and redirect all output to a single file.

          – rozcietrzewiacz
          Oct 7 '11 at 8:54











        • You're right. I edited the solution to correct it.

          – rahmu
          Oct 7 '11 at 8:57











        • OK - Almost good :) Just somefile.dat.ans output stuff would look not so nice.

          – rozcietrzewiacz
          Oct 7 '11 at 9:07






        • 1





          Edited! I did not know about '%'. It works like a charm, thanks for the tip.

          – rahmu
          Oct 7 '11 at 9:23






        • 1





          Adding a -type file would be nice (can't < directory), and this makes the unusual-filename-fairy sad.

          – Mat
          Oct 7 '11 at 9:57



















        • That would not work. It would try to operate on stuff like somefile.dat.dat and redirect all output to a single file.

          – rozcietrzewiacz
          Oct 7 '11 at 8:54











        • You're right. I edited the solution to correct it.

          – rahmu
          Oct 7 '11 at 8:57











        • OK - Almost good :) Just somefile.dat.ans output stuff would look not so nice.

          – rozcietrzewiacz
          Oct 7 '11 at 9:07






        • 1





          Edited! I did not know about '%'. It works like a charm, thanks for the tip.

          – rahmu
          Oct 7 '11 at 9:23






        • 1





          Adding a -type file would be nice (can't < directory), and this makes the unusual-filename-fairy sad.

          – Mat
          Oct 7 '11 at 9:57

















        That would not work. It would try to operate on stuff like somefile.dat.dat and redirect all output to a single file.

        – rozcietrzewiacz
        Oct 7 '11 at 8:54





        That would not work. It would try to operate on stuff like somefile.dat.dat and redirect all output to a single file.

        – rozcietrzewiacz
        Oct 7 '11 at 8:54













        You're right. I edited the solution to correct it.

        – rahmu
        Oct 7 '11 at 8:57





        You're right. I edited the solution to correct it.

        – rahmu
        Oct 7 '11 at 8:57













        OK - Almost good :) Just somefile.dat.ans output stuff would look not so nice.

        – rozcietrzewiacz
        Oct 7 '11 at 9:07





        OK - Almost good :) Just somefile.dat.ans output stuff would look not so nice.

        – rozcietrzewiacz
        Oct 7 '11 at 9:07




        1




        1





        Edited! I did not know about '%'. It works like a charm, thanks for the tip.

        – rahmu
        Oct 7 '11 at 9:23





        Edited! I did not know about '%'. It works like a charm, thanks for the tip.

        – rahmu
        Oct 7 '11 at 9:23




        1




        1





        Adding a -type file would be nice (can't < directory), and this makes the unusual-filename-fairy sad.

        – Mat
        Oct 7 '11 at 9:57





        Adding a -type file would be nice (can't < directory), and this makes the unusual-filename-fairy sad.

        – Mat
        Oct 7 '11 at 9:57











        2














        For simple patterns, the for loop is appropriate:



        for file in A/*.dat; do
        ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
        done


        For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:



        find A -name '*.dat' -print | while IFS= read -r file; do
        ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
        done


        This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0 option to read so in those you can handle all characters, but unfortunately it's not POSIX.






        share|improve this answer






























          2














          For simple patterns, the for loop is appropriate:



          for file in A/*.dat; do
          ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
          done


          For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:



          find A -name '*.dat' -print | while IFS= read -r file; do
          ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
          done


          This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0 option to read so in those you can handle all characters, but unfortunately it's not POSIX.






          share|improve this answer




























            2












            2








            2







            For simple patterns, the for loop is appropriate:



            for file in A/*.dat; do
            ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
            done


            For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:



            find A -name '*.dat' -print | while IFS= read -r file; do
            ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
            done


            This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0 option to read so in those you can handle all characters, but unfortunately it's not POSIX.






            share|improve this answer















            For simple patterns, the for loop is appropriate:



            for file in A/*.dat; do
            ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
            done


            For more complex cases where you need another utility to list the files (zsh or bash 4 have powerful enough patterns that you rarely need find, but if you want to stay within POSIX shell or use fast shell like dash, you will need find for anything non-trivial), while read is most appropriate:



            find A -name '*.dat' -print | while IFS= read -r file; do
            ./a.out < "${file}" > "${file%.dat}.ans" # Never forget the QUOTES!
            done


            This will handle spaces, because read is (by default) line-oriented. It will not handle newlines and it will not handle backslashes, because by default it interprets escape sequences (that actually allows you to pass in a newline, but find can't generate that format). Many shells have -0 option to read so in those you can handle all characters, but unfortunately it's not POSIX.







            share|improve this answer














            share|improve this answer



            share|improve this answer








            edited Oct 7 '11 at 23:36









            Gilles

            544k12811021619




            544k12811021619










            answered Oct 7 '11 at 12:20









            Jan HudecJan Hudec

            27519




            27519























                2














                Use GNU Parallel:



                parallel ./a.out "<{} >{.}.ans" ::: A/*.dat


                Added bonus: You get the processing done in parallel.



                Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ






                share|improve this answer




























                  2














                  Use GNU Parallel:



                  parallel ./a.out "<{} >{.}.ans" ::: A/*.dat


                  Added bonus: You get the processing done in parallel.



                  Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ






                  share|improve this answer


























                    2












                    2








                    2







                    Use GNU Parallel:



                    parallel ./a.out "<{} >{.}.ans" ::: A/*.dat


                    Added bonus: You get the processing done in parallel.



                    Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ






                    share|improve this answer













                    Use GNU Parallel:



                    parallel ./a.out "<{} >{.}.ans" ::: A/*.dat


                    Added bonus: You get the processing done in parallel.



                    Watch the intro videos to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ







                    share|improve this answer












                    share|improve this answer



                    share|improve this answer










                    answered Oct 10 '11 at 13:34









                    Ole TangeOle Tange

                    12.8k1456107




                    12.8k1456107























                        1














                        I think you need at least a shell invocation in the xargs:



                        ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"


                        Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.






                        share|improve this answer


























                        • Have a look at why it is a bad idea to parse ls output.

                          – rozcietrzewiacz
                          Oct 7 '11 at 9:13











                        • @rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.

                          – thiton
                          Oct 7 '11 at 9:19






                        • 1





                          You are wrong about the shell expansion. ls outputs things like spaces without escaping and that is the problem.

                          – rozcietrzewiacz
                          Oct 7 '11 at 9:25











                        • @rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.

                          – thiton
                          Oct 7 '11 at 10:19











                        • Yes. You see now, parsing ls is no good. But xargs can be safely used with find - see the suggestion No 2 in my answer.

                          – rozcietrzewiacz
                          Oct 7 '11 at 10:53


















                        1














                        I think you need at least a shell invocation in the xargs:



                        ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"


                        Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.






                        share|improve this answer


























                        • Have a look at why it is a bad idea to parse ls output.

                          – rozcietrzewiacz
                          Oct 7 '11 at 9:13











                        • @rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.

                          – thiton
                          Oct 7 '11 at 9:19






                        • 1





                          You are wrong about the shell expansion. ls outputs things like spaces without escaping and that is the problem.

                          – rozcietrzewiacz
                          Oct 7 '11 at 9:25











                        • @rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.

                          – thiton
                          Oct 7 '11 at 10:19











                        • Yes. You see now, parsing ls is no good. But xargs can be safely used with find - see the suggestion No 2 in my answer.

                          – rozcietrzewiacz
                          Oct 7 '11 at 10:53
















                        1












                        1








                        1







                        I think you need at least a shell invocation in the xargs:



                        ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"


                        Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.






                        share|improve this answer















                        I think you need at least a shell invocation in the xargs:



                        ls A/*.dat | xargs -I file sh -c "./a.out < file > file.ans"


                        Edit: It should be noted that this approach does not work when the filenames contain whitespace. Can't work. Even if you used find -0 and xargs -0 to make xargs understand the spaces correctly, the -c shell call would croak on them. However, the OP explicitely asked for an xargs solution, and this is the best xargs solution I came up with. If whitespace in filenames might be an issue, use find -exec or a shell loop.







                        share|improve this answer














                        share|improve this answer



                        share|improve this answer








                        edited Oct 7 '11 at 10:23

























                        answered Oct 7 '11 at 8:58









                        thitonthiton

                        1,44899




                        1,44899













                        • Have a look at why it is a bad idea to parse ls output.

                          – rozcietrzewiacz
                          Oct 7 '11 at 9:13











                        • @rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.

                          – thiton
                          Oct 7 '11 at 9:19






                        • 1





                          You are wrong about the shell expansion. ls outputs things like spaces without escaping and that is the problem.

                          – rozcietrzewiacz
                          Oct 7 '11 at 9:25











                        • @rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.

                          – thiton
                          Oct 7 '11 at 10:19











                        • Yes. You see now, parsing ls is no good. But xargs can be safely used with find - see the suggestion No 2 in my answer.

                          – rozcietrzewiacz
                          Oct 7 '11 at 10:53





















                        • Have a look at why it is a bad idea to parse ls output.

                          – rozcietrzewiacz
                          Oct 7 '11 at 9:13











                        • @rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.

                          – thiton
                          Oct 7 '11 at 9:19






                        • 1





                          You are wrong about the shell expansion. ls outputs things like spaces without escaping and that is the problem.

                          – rozcietrzewiacz
                          Oct 7 '11 at 9:25











                        • @rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.

                          – thiton
                          Oct 7 '11 at 10:19











                        • Yes. You see now, parsing ls is no good. But xargs can be safely used with find - see the suggestion No 2 in my answer.

                          – rozcietrzewiacz
                          Oct 7 '11 at 10:53



















                        Have a look at why it is a bad idea to parse ls output.

                        – rozcietrzewiacz
                        Oct 7 '11 at 9:13





                        Have a look at why it is a bad idea to parse ls output.

                        – rozcietrzewiacz
                        Oct 7 '11 at 9:13













                        @rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.

                        – thiton
                        Oct 7 '11 at 9:19





                        @rozcietrzewiacz: In general, sure, but I assume someone trying to do xargs voodoo knows this. Since these filenames undergo another shell expansion, they have to be well-behaved anyway.

                        – thiton
                        Oct 7 '11 at 9:19




                        1




                        1





                        You are wrong about the shell expansion. ls outputs things like spaces without escaping and that is the problem.

                        – rozcietrzewiacz
                        Oct 7 '11 at 9:25





                        You are wrong about the shell expansion. ls outputs things like spaces without escaping and that is the problem.

                        – rozcietrzewiacz
                        Oct 7 '11 at 9:25













                        @rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.

                        – thiton
                        Oct 7 '11 at 10:19





                        @rozcietrzewiacz: Yes, I understand that problem. Now just suppose you would properly escape these spaces and get them into xargs: They are replaced in sh's -c string, sh tokenizes the -c string, and everything breaks.

                        – thiton
                        Oct 7 '11 at 10:19













                        Yes. You see now, parsing ls is no good. But xargs can be safely used with find - see the suggestion No 2 in my answer.

                        – rozcietrzewiacz
                        Oct 7 '11 at 10:53







                        Yes. You see now, parsing ls is no good. But xargs can be safely used with find - see the suggestion No 2 in my answer.

                        – rozcietrzewiacz
                        Oct 7 '11 at 10:53













                        1














                        There's no need to complicate it. You could do it with a for loop:



                        for file in A/*.dat; do
                        ./a.out < "$file" >"${file%.dat}.ans"
                        done


                        the ${file%.dat}.ans bit will remove the .dat filename suffix from the filename in $file and instead add .ans to the end.






                        share|improve this answer






























                          1














                          There's no need to complicate it. You could do it with a for loop:



                          for file in A/*.dat; do
                          ./a.out < "$file" >"${file%.dat}.ans"
                          done


                          the ${file%.dat}.ans bit will remove the .dat filename suffix from the filename in $file and instead add .ans to the end.






                          share|improve this answer




























                            1












                            1








                            1







                            There's no need to complicate it. You could do it with a for loop:



                            for file in A/*.dat; do
                            ./a.out < "$file" >"${file%.dat}.ans"
                            done


                            the ${file%.dat}.ans bit will remove the .dat filename suffix from the filename in $file and instead add .ans to the end.






                            share|improve this answer















                            There's no need to complicate it. You could do it with a for loop:



                            for file in A/*.dat; do
                            ./a.out < "$file" >"${file%.dat}.ans"
                            done


                            the ${file%.dat}.ans bit will remove the .dat filename suffix from the filename in $file and instead add .ans to the end.







                            share|improve this answer














                            share|improve this answer



                            share|improve this answer








                            edited Apr 12 '18 at 20:43

























                            answered Oct 7 '11 at 8:52









                            KusalanandaKusalananda

                            137k17258426




                            137k17258426






























                                draft saved

                                draft discarded




















































                                Thanks for contributing an answer to Unix & Linux Stack Exchange!


                                • Please be sure to answer the question. Provide details and share your research!

                                But avoid



                                • Asking for help, clarification, or responding to other answers.

                                • Making statements based on opinion; back them up with references or personal experience.


                                To learn more, see our tips on writing great answers.




                                draft saved


                                draft discarded














                                StackExchange.ready(
                                function () {
                                StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2funix.stackexchange.com%2fquestions%2f22229%2fxargs-with-stdin-stdout-redirection%23new-answer', 'question_page');
                                }
                                );

                                Post as a guest















                                Required, but never shown





















































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown

































                                Required, but never shown














                                Required, but never shown












                                Required, but never shown







                                Required, but never shown







                                Popular posts from this blog

                                濃尾地震

                                How to rewrite equation of hyperbola in standard form

                                No ethernet ip address in my vocore2