Yrurtj

I have data like this:

chr1    134901  139379  -   "ENSG00000237683.5";

chr1    860260  879955  +   "ENSG00000187634.6";

chr1    861264  866445  -   "ENSG00000268179.1";

chr1    879584  894689  -   "ENSG00000188976.6";

chr1    895967  901095  +   "ENSG00000187961.9";

I generated by parsing a GTF file

I want to remove the "'s and ;'s from column 5 using awk or sed if it possible. The result would look like this:

chr1    134901  139379  -   ENSG00000237683.5

chr1    860260  879955  +   ENSG00000187634.6

chr1    861264  866445  -   ENSG00000268179.1

chr1    879584  894689  -   ENSG00000188976.6

chr1    895967  901095  +   ENSG00000187961.9

Using gsub:

awk '{gsub(/"|;/,"")}1' file

chr1    134901  139379  -   ENSG00000237683.5

chr1    860260  879955  +   ENSG00000187634.6

chr1    861264  866445  -   ENSG00000268179.1

chr1    879584  894689  -   ENSG00000188976.6

chr1    895967  901095  +   ENSG00000187961.9

If you want to operate only on the fifth field and preserve any quotes or semicolons in other fields:

awk '{gsub(/"|;/,"",$5)}1' file 

Using sed to remove all instances of '";':
sed -i 's/[";]//g' file

To only remove from 5th column sed is probably not the best option.

If your data is formatted exactly as shown (i.e. no other " or ; in other columns that need to be preserved), then you can simply use tr to remove these characters:

tr -d '";' < input.txt > output.txt

I know the original post asked for sed or awk but if you want to remove the " and ; from only the fifth column I'd use regex and php. There's probably a way to do this in AWK but I like to use the easiest tools.

<?php



foreach(file($argv[1]) as $line){



    $matches = array();

    preg_match('/^(w+)s+(d+)s+(d+)s+(-|+)s+"(w+.d)";/',$line,$matches);

    $matched_line = array_shift($matches); // remove the first element

    vprintf("%st%st%st%st%sn",$matches);

}

this would output this

$ php /tmp/preg_replace.php /tmp/data

chr1    134901  139379  -   ENSG00000237683.5

chr1    860260  879955  +   ENSG00000187634.6

chr1    861264  866445  -   ENSG00000268179.1

chr1    879584  894689  -   ENSG00000188976.6

chr1    895967  901095  +   ENSG00000187961.9

A sed solution that makes sure we're only fiddling around with the fifth column:

sed -E 's/^(([^ ]+ +){4})"([^"]+)";$/13/' infile

chr1    134901  139379  -   ENSG00000237683.5

chr1    860260  879955  +   ENSG00000187634.6

chr1    861264  866445  -   ENSG00000268179.1

chr1    879584  894689  -   ENSG00000188976.6

chr1    895967  901095  +   ENSG00000187961.9

This works also without ERE (-E, or -r for some older sed), but requires a lot more backslashes. The +-quantifier is ERE-only according to the POSIX spec¹ and can be replaced by {1,} (or {1,} for BRE).

In case the columns aren't space-separated, the spaces can be replaced by the [:blank:] POSIX character class to also match tabs.

The regex in detail:

^               # Anchored at start of line

(               # Capture group 1 for first 4 columns

    (           # Capture group 2 for repeat count

        [^ ]+   # 1 or more non-spaces

         +      # 1 or more spaces

    ){4}        # 4 times "word plus spaces" (columns)

)               # End capture group 1

"               # Column 5 starts with double quote (not captured)

(               # Capture group 3 for column 5

    [^"]+       # One or more non-quote characters

)               # End capture group 3

";              # Quote and semicolon at end of column 5

$               # Anchored at end of line

¹ GNU sed, as an extension, allows + to be used in BRE as well.