Scripts and Utilities -- Sed/Ed lecture


  • Jim Plank
  • Directory: /home/cs494/notes/Sed
  • This file: http://www.cs.utk.edu/~plank/plank/classes/cs494/494/notes/Sed/lecture.html
  • Lecture links: http://www.cs.utk.edu/~plank/plank/classes/cs494/494/notes/Sed/links.html
  • Email questions and answers

    sed

    Sed is a ``stream editing'' command, which, like sort and grep is much more useful than you would at first believe. The syntax for sed is:
    UNIX> sed [ -n ] [ -e script ] [ -f sfilename ] [ filenames ]
    
    Usually you call sed in the following manner (omitting the -e):
    UNIX> sed script filename
    
    or you omit the filename and sed will work on standard input.

    Scripts have the following syntax:

              [ address [, address ] ] function [ arguments ]
    
    The addresses define the lines on which sed will act. The function defines what will get done to those lines. On the other lines (those not included in the address specification), sed either copies them to standard output unmodified, or if sed was called with the -n option, it does not copy them to standard output.

    If no addresses are defined, then sed acts on every line.

    Addresses

    You can specify an address by its line number, or as a range of line numbers, separated by a comma. Thus, you specify lines 6 through 10 as ``6,10''. The last line can be specified by the dollar sign.

    Instead of using a line number, you may use a regular expression. See the grep lecture for a primer on regular expressions. If you specify just one address with a regular expression, then sed will act on all lines that match that regular expression. If you specify a range of two regular expressions, then sed will act on all lines from the first line matching the first regular expression to the next line matching the second regular expression. Then it skips until it finds a line matching the first regular expression and acts on all lines from that line until the next line matching the second regular expression and so on. If it hits the end of the file before finding the a match for the second regular expression, then it simply goes to the end of the file.

    Functions

    The simplest function is d, which deletes the specified lines. Here are some examples:
    UNIX> cat usopen
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
         6  Olin Browne -- +2     -4    6
         7  Jim Furyk -- +2     -4    6
         8  Tommy Tolles -- +2     -4    6
         9  Jay Haas -- +2     -4    6
        10  Scott McCarron -- +3     -4    7
        11  Scott Hoch -- +3     -4    7
        12  David Ogrin -- +3     -4    7
        13  Loren Roberts -- +4     -4    8
        14  Stewart Cink -- +4     -4    8
        15  Billy Andrade -- +4     -4    8
        16  Bradley Hughes -- +5     -4    9
        17  Jose Maria Olazabal -- +5     -4    9
        18  Davis Love III -- +5     -4    9
        19  Nick Price -- +6     -4    10
        20  Lee Westwood -- +6     -4    10
    UNIX> sed d usopen                             # delete all lines
    UNIX> sed 6,19d usopen                         # delete lines 6 - 19
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
        20  Lee Westwood -- +6     -4    10
    UNIX> sed '2,$d' usopen                        # delete all but the first line
         1  Ernie Els -- -4     -4    0
    UNIX> sed '/+/d' usopen                        # delete all lines with +'s
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
    UNIX> sed '/^.....3/,/^.....9/d' usopen        # delete lines 3-9 and 13-19
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
        10  Scott McCarron -- +3     -4    7
        11  Scott Hoch -- +3     -4    7
        12  David Ogrin -- +3     -4    7
        20  Lee Westwood -- +6     -4    10
    UNIX> 
    
    The p command explicitly prints out a line. This is most often used in combination with the -n option. In this way, you can have sed strip out all but the specified lines. Note that
    UNIX> sed -n /RE/p
    
    is equivalent to:
    UNIX> grep RE
    
    When used without the -n option, the p command duplicates a line, which is sometimes convenient. Some examples:
    UNIX> sed -n 5p usopen                           # print just line 5
         5  Bob Tway -- +2     -4    6
    UNIX> sed -n 1,5p usopen                         # print lines 1-5
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    UNIX> sed -n '/J/p' usopen                      # print all lines with J's
         4  Jeff Maggert -- +1     -4    5
         7  Jim Furyk -- +2     -4    6
         9  Jay Haas -- +2     -4    6
        17  Jose Maria Olazabal -- +5     -4    9
    UNIX> sed -n '/J/,/J/p' usopen                  # Can you figure it out?
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
         6  Olin Browne -- +2     -4    6
         7  Jim Furyk -- +2     -4    6
         9  Jay Haas -- +2     -4    6
        10  Scott McCarron -- +3     -4    7
        11  Scott Hoch -- +3     -4    7
        12  David Ogrin -- +3     -4    7
        13  Loren Roberts -- +4     -4    8
        14  Stewart Cink -- +4     -4    8
        15  Billy Andrade -- +4     -4    8
        16  Bradley Hughes -- +5     -4    9
        17  Jose Maria Olazabal -- +5     -4    9
    UNIX> head -5 usopen | sed p                   # duplicate lines 1-5
         1  Ernie Els -- -4     -4    0
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
         5  Bob Tway -- +2     -4    6
    UNIX> 
    

    Substitution

    The most useful feature of sed is pattern substitution. The syntax for substitution is
    s/pattern1/pattern2/command
    
    In its simplest form, you give a regular expression for pattern1, a substitution string for pattern2, and no command. That means to substitute the substitution string when you encounter the regular expression in a line.

    Examples:

    UNIX> cat ussmall
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    UNIX> sed 's/--/XXXX/' ussmall
         1  Ernie Els XXXX -4     -4    0
         2  Colin Montgomerie XXXX -3     -4    1
         3  Tom Lehman XXXX -2     -4    2
         4  Jeff Maggert XXXX +1     -4    5
         5  Bob Tway XXXX +2     -4    6
    UNIX> sed 's/-/X/' ussmall
         1  Ernie Els X- -4     -4    0
         2  Colin Montgomerie X- -3     -4    1
         3  Tom Lehman X- -2     -4    2
         4  Jeff Maggert X- +1     -4    5
         5  Bob Tway X- +2     -4    6
    UNIX> 
    
    You'll note that it only performs the substitution on the first matching of pattern1. In other words, in the last command, only the first dash is replaced with an X, and not all of them. To replace all occurrences of pattern1, use a command of g.

    You can specify addresses on which to perform substitution. If pattern1 is not found on a line, then it performs no substitution.

    Examples:

    UNIX> sed 's/-/X/g' ussmall
         1  Ernie Els XX X4     X4    0
         2  Colin Montgomerie XX X3     X4    1
         3  Tom Lehman XX X2     X4    2
         4  Jeff Maggert XX +1     X4    5
         5  Bob Tway XX +2     X4    6
    UNIX> sed '/Lehman/s/--/XX/' ussmall
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman XX -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    UNIX> sed '1,/Maggert/s/Maggert/Sluman/' ussmall
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Sluman -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    UNIX> sed 's/Ernie/Burt/' ussmall
         1  Burt Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    UNIX> 
    

    \( and \)

    If you enclose a subpattern in \( and \), then you may use whatever matches the subpattern in the replacement string. The syntax in the replacement string is \n, where n identifies the subpattern. Here are some pretty contrived examples:
    UNIX> sed 's/^ *\([0-9]\)/\1 \1/' ussmall
    1 1  Ernie Els -- -4     -4    0
    2 2  Colin Montgomerie -- -3     -4    1
    3 3  Tom Lehman -- -2     -4    2
    4 4  Jeff Maggert -- +1     -4    5
    5 5  Bob Tway -- +2     -4    6
    UNIX> sed 's/^\(.*\) -- \(..\).*/\2  \1/' ussmall
    -4       1  Ernie Els
    -3       2  Colin Montgomerie
    -2       3  Tom Lehman
    +1       4  Jeff Maggert
    +2       5  Bob Tway
    UNIX> sed 's/\([A-Z][a-z]*\) /"\1 Baby" /' ussmall
         1  "Ernie Baby" Els -- -4     -4    0
         2  "Colin Baby" Montgomerie -- -3     -4    1
         3  "Tom Baby" Lehman -- -2     -4    2
         4  "Jeff Baby" Maggert -- +1     -4    5
         5  "Bob Baby" Tway -- +2     -4    6
    UNIX> 
    
    When performing the matching, sed finds the largest leftmost string that matches the pattern. The precedence is leftmost first, then largest. Thus, \(.*\) will match the entire string. \(.\) will match the leftmost character. And so on. Sometimes this gives you surprises when you use stars, because they can match nothing. However, you usually figure out how to get things working pretty quickly.
    UNIX> sed 's/\(.*\)/XXX \1 XXX/' ussmall
    XXX      1  Ernie Els -- -4     -4    0 XXX
    XXX      2  Colin Montgomerie -- -3     -4    1 XXX
    XXX      3  Tom Lehman -- -2     -4    2 XXX
    XXX      4  Jeff Maggert -- +1     -4    5 XXX
    XXX      5  Bob Tway -- +2     -4    6 XXX
    UNIX> sed 's/\(.\)/"\1"/' ussmall
    " "    1  Ernie Els -- -4     -4    0
    " "    2  Colin Montgomerie -- -3     -4    1
    " "    3  Tom Lehman -- -2     -4    2
    " "    4  Jeff Maggert -- +1     -4    5
    " "    5  Bob Tway -- +2     -4    6
    UNIX> sed 's/\([A-Za-z]*\)/"\1 Baby"/' ussmall
    " Baby"     1  Ernie Els -- -4     -4    0
    " Baby"     2  Colin Montgomerie -- -3     -4    1
    " Baby"     3  Tom Lehman -- -2     -4    2
    " Baby"     4  Jeff Maggert -- +1     -4    5
    " Baby"     5  Bob Tway -- +2     -4    6
    UNIX> sed 's/[^A-Z]*\([^ ]*\) \([^ ]*\) -- \(..\).*/\3: \2, \1/' ussmall
    -4: Els, Ernie
    -3: Montgomerie, Colin
    -2: Lehman, Tom
    +1: Maggert, Jeff
    +2: Tway, Bob
    UNIX> sed 's/[^A-Z]*\([A-Za-z ]*\).*/\1\1\1/' ussmall
    Ernie Els Ernie Els Ernie Els 
    Colin Montgomerie Colin Montgomerie Colin Montgomerie 
    Tom Lehman Tom Lehman Tom Lehman 
    Jeff Maggert Jeff Maggert Jeff Maggert 
    Bob Tway Bob Tway Bob Tway 
    UNIX> 
    
    You can call sed with -n, and use the p string as a command to the substituion, and then sed will only print out lines which performed the substitution:
    UNIX> sed -n 's/Maggert/Maggot/p' ussmall
         4  Jeff Maggot -- +1     -4    5
    UNIX> sed -n 's/-- -\(.\).*/: \1 under/p' ussmall
         1  Ernie Els : 4 under
         2  Colin Montgomerie : 3 under
         3  Tom Lehman : 2 under
    UNIX> 
    
    The g command works with replacement substrings too. Again, beware of the *:
    UNIX> sed 's/\([A-Z][a-z]*\)/X \1/g' ussmall
         1  X Ernie X Els -- -4     -4    0
         2  X Colin X Montgomerie -- -3     -4    1
         3  X Tom X Lehman -- -2     -4    2
         4  X Jeff X Maggert -- +1     -4    5
         5  X Bob X Tway -- +2     -4    6
    UNIX> sed 's/\([A-Za-z]*\)/X \1/g' ussmall      # can you figure out why???
    Output line too long.
    Output line too long.
    Output line too long.
    Output line too long.
    X X X X X X X X X X X X X X X X X X X....
    UNIX>
    
    Finally, you can use other characters besides the slash to delimit a substitution. In fact, you can use any character you want. This is useful if you want to mess with slashes:
    UNIX> sed 's#\([A-Za-z]\{1,\}\)#/\1/#g' ussmall
         1  /Ernie/ /Els/ -- -4     -4    0
         2  /Colin/ /Montgomerie/ -- -3     -4    1
         3  /Tom/ /Lehman/ -- -2     -4    2
         4  /Jeff/ /Maggert/ -- +1     -4    5
         5  /Bob/ /Tway/ -- +2     -4    6
    UNIX> 
    

    Scripts

    You can place multiple sed scripts into a file, and then run it with sed -f filename:
    UNIX> cat script
    s/[^A-Z]*//
    s/\([A-Za-z]*\) \([A-Za-z ]*\) /\2, \1/
    s/--/:/
    s/   .*//
    s/-\([0-9]*\)/\1 under/
    s/+\([0-9]*\)/\1 over/
    s/Maria Olazabal, Jose/Olazabal, Jose Maria/
    UNIX> sed -f script usopen
    Els, Ernie: 4 under
    Montgomerie, Colin: 3 under
    Lehman, Tom: 2 under
    Maggert, Jeff: 1 over
    Tway, Bob: 2 over
    Browne, Olin: 2 over
    Furyk, Jim: 2 over
    Tolles, Tommy: 2 over
    Haas, Jay: 2 over
    McCarron, Scott: 3 over
    Hoch, Scott: 3 over
    Ogrin, David: 3 over
    Roberts, Loren: 4 over
    Cink, Stewart: 4 over
    Andrade, Billy: 4 over
    Hughes, Bradley: 5 over
    Olazabal, Jose Maria: 5 over
    Love III, Davis: 5 over
    Price, Nick: 6 over
    Westwood, Lee: 6 over
    UNIX> 
    

    Ed

    Ed is the most basic Unix editor. True hatchet throwers always point out that there are still people at AT&T who edit with ed. Or, at least that's what they used to say when I was first in graduate school. Anyway, ed is pretty much an interactive version of sed. You'll find it very useful to know ed, because there are times when you have to use it, most notably when your system doesn't understand what kind of terminal you have, or if you're on a brutally slow modem.

    Basic ed functions you should know

    Ed gives you no prompt unless you ask for it. For that reason, I've alaised ed to ``ed -p"ed: "''. When ed doesn't like something, it gives you a question mark with no diagnostics. Ed is truly the ultimate Unix utility.

    Ed's syntax is like sed's:

              [ address [, address ] ] function [ arguments ]
    
    Another address you should know about is dot ('.'), which is the ``current'' line.

    Some basic ed functions you should know are:

    So, an example of copying ussmall and modifying it:
    UNIX> cp ussmall work
    UNIX> alias ed
    ed -p"ed: "
    UNIX> ed work
    191                           # This tells you how big the file is in bytes
    ed: 1,$p
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    ed: 1,$n
    1            1  Ernie Els -- -4     -4    0
    2            2  Colin Montgomerie -- -3     -4    1
    3            3  Tom Lehman -- -2     -4    2
    4            4  Jeff Maggert -- +1     -4    5
    5            5  Bob Tway -- +2     -4    6
    ed: 3d
    ed: 1,$n
    1            1  Ernie Els -- -4     -4    0
    2            2  Colin Montgomerie -- -3     -4    1
    3            4  Jeff Maggert -- +1     -4    5
    4            5  Bob Tway -- +2     -4    6
    ed: /Els/,3n
    1            1  Ernie Els -- -4     -4    0
    2            2  Colin Montgomerie -- -3     -4    1
    3            4  Jeff Maggert -- +1     -4    5
    ed: /Mont/,$d
    ed: 1,$n
    1            1  Ernie Els -- -4     -4    0
    ed: w
    36                        # This tells you how big the file is in bytes
    ed: q
    UNIX> cat work
         1  Ernie Els -- -4     -4    0
    UNIX> 
    
    You can do substitution commands in ed, just like in sed: Note that when you specify just one address in ed, it only does the first matching line:
    UNIX> cp ussmall work
    UNIX> ed work
    191
    ed: 1,$s/-- \([-+][0-9]*\).*/: \1/
    ed: 1,$p
         1  Ernie Els : -4
         2  Colin Montgomerie : -3
         3  Tom Lehman : -2
         4  Jeff Maggert : +1
         5  Bob Tway : +2
    ed: /M/s/[A-Z]/X/g
    ed: 1,$p
         1  Ernie Els : -4
         2  Xolin Xontgomerie : -3
         3  Tom Lehman : -2
         4  Jeff Maggert : +1
         5  Bob Tway : +2
    ed: q
    ?
    ed: q                  # don't save the changes
    UNIX> 
    
    You can use the g function to apply the command that follows to all lines that match the regular expression after the g:
    UNIX> ed work
    191
    ed: 1,$n
    1            1  Ernie Els -- -4     -4    0
    2            2  Colin Montgomerie -- -3     -4    1
    3            3  Tom Lehman -- -2     -4    2
    4            4  Jeff Maggert -- +1     -4    5
    5            5  Bob Tway -- +2     -4    6
    ed: 1,$g/M/d
    ed: 1,$p
         1  Ernie Els -- -4     -4    0
         3  Tom Lehman -- -2     -4    2
         5  Bob Tway -- +2     -4    6
    ed: q
    ?
    ed: q
    UNIX> ed work
    191
    ed: 1,$g/M/s/[A-Z]/X/g
    ed: 1,$p
         1  Ernie Els -- -4     -4    0
         2  Xolin Xontgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Xeff Xaggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    ed: w
    191
    ed: q
    UNIX> 
    
    You can add or lines of text in ed. The a function says to add the lines after the given line. The i function says to add the lines before the given line. The c command says to replace the specified line(s) with the ones that you type in. You complete the addition of text with a control-D, or by a line that has nothing but a period:
    UNIX> cp ussmall work
    UNIX> ed work
    191
    ed: 1a
    Ernie was the only player who didn't choke.
    The rest were total chokaramas.
    .
    ed: 1,$n
    1            1  Ernie Els -- -4     -4    0
    2       Ernie was the only player who didn't choke.
    3       The rest were total chokaramas.
    4            2  Colin Montgomerie -- -3     -4    1
    5            3  Tom Lehman -- -2     -4    2
    6            4  Jeff Maggert -- +1     -4    5
    7            5  Bob Tway -- +2     -4    6
    ed: $a  
    No one else
    .
    ed: 1,$p
         1  Ernie Els -- -4     -4    0
    Ernie was the only player who didn't choke.
    The rest were total chokaramas.
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    No one else
    ed: $c
    Definitely no one else
    .
    ed: 1,$n
    1            1  Ernie Els -- -4     -4    0
    2       Ernie was the only player who didn't choke.
    3       The rest were total chokaramas.
    4            2  Colin Montgomerie -- -3     -4    1
    5            3  Tom Lehman -- -2     -4    2
    6            4  Jeff Maggert -- +1     -4    5
    7            5  Bob Tway -- +2     -4    6
    8       Definitely no one else
    ed: 
    

    Marks, moving text, duplicating text

    You can mark a line with the k function. The mark is a one-letter abbreviation. If you mark a line with ka, then you can use that mark in addresses following a single quote:
    UNIX> cp ussmall work
    UNIX> ed work
    191
    ed: /Maggert/ka
    ed: 'ap
         4  Jeff Maggert -- +1     -4    5
    ed: 1,'an
    1            1  Ernie Els -- -4     -4    0
    2            2  Colin Montgomerie -- -3     -4    1
    3            3  Tom Lehman -- -2     -4    2
    4            4  Jeff Maggert -- +1     -4    5
    ed: 2
         2  Colin Montgomerie -- -3     -4    1
    ed: kb
    ed: 'b,'ap
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
    ed: 
    
    Note in the kb command, the mark is set to the current line, which was set in the previous command.

    The m function moves a set of lines to the line following the specified line. The t function copies a set of lines to the line following the specified line:

    ed: 1,$p
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    ed: 1,2m4
    ed: 1,$p
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         5  Bob Tway -- +2     -4    6
    ed: 1 
         3  Tom Lehman -- -2     -4    2
    ed: 1t1
    ed: 1,$p
         3  Tom Lehman -- -2     -4    2
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         5  Bob Tway -- +2     -4    6
    ed:
    

    Fancier addressing

    There are other ways to address lines. Enclosing a regular expression in question marks searched backwards from the current line to find the expression. You do any address +n or in do denote a positive or negative offset to the address. A lone comma stands for 1,$, and a semicolon stands for .,$.

    More commands

    The j command joins the specified lines (i.e. it takes out the newlines). The r command inserts a specified filename at the line following the given address. The u command undoes the previous command that modified the buffer (but you can only undo once).
    UNIX> cp ussmall work
    UNIX> ed work
    191
    ed: 1,$n
    1            1  Ernie Els -- -4     -4    0
    2            2  Colin Montgomerie -- -3     -4    1
    3            3  Tom Lehman -- -2     -4    2
    4            4  Jeff Maggert -- +1     -4    5
    5            5  Bob Tway -- +2     -4    6
    ed: 1,2j
    ed: 1,$p
         1  Ernie Els -- -4     -4    0     2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
    ed: 3,4j
    ed: 1,$p
         1  Ernie Els -- -4     -4    0     2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5     5  Bob Tway -- +2     -4    6
    ed: 0r ussmall
    191
    ed: 1,$p
         1  Ernie Els -- -4     -4    0
         2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5
         5  Bob Tway -- +2     -4    6
         1  Ernie Els -- -4     -4    0     2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5     5  Bob Tway -- +2     -4    6
    ed: u
    ed: 1,$p
         1  Ernie Els -- -4     -4    0     2  Colin Montgomerie -- -3     -4    1
         3  Tom Lehman -- -2     -4    2
         4  Jeff Maggert -- +1     -4    5     5  Bob Tway -- +2     -4    6
    ed: q
    ?
    ed: q
    UNIX> 
    

    Ed, sed and vi

    In the progression from total non-user-friendliness to semi-user-friendliness, the Unix folks next made vi, which is a screen-oriented ed. The nice thing about vi is that all (or almost all) ed commands work in vi if you start them with a colon. This means that you can do all sorts of fancy sed-like things in vi -- and it all comes for free once you understand sed.

    Of course, learning vi does try ones nerves if one is not used to ``edit'' and ``insert'' modes, but the interplay of sed, ed and vi is one of vi's big strengths.

    Ed in shell scripts

    The ``<< XX' indirection specification in the Bourne shell says to take standard input from the subsequent lines until you reach an ``XX''. This is usually how one puts ed into shell scripts. You usually call ed with a single dash so that it supresses all character counts. In standard input, environment variables are still expanded, which sometimes leads to surprises, but lets you do cool stuff.

    A simple example. Note the use of ',' to specify all lines, and the expansion of $1

    UNIX> cat shscript
    #!/bin/sh
    
    if [ $# -ne 1 ]; then
      echo "usage: shscript name" >&2
      exit 1
    fi
    
    cp ussmall work
    
    ed - work << EOF
    ,s/[^A-Z]*//
    ,s/ --.*//
    ,s/\([^ ]*\)/\1 "$1"/
    w
    q
    EOF
    
    UNIX> 
    UNIX> shscript Thumper
    UNIX> cat work
    Ernie "Thumper" Els
    Colin "Thumper" Montgomerie
    Tom "Thumper" Lehman
    Jeff "Thumper" Maggert
    Bob "Thumper" Tway
    UNIX> 
    

    Tr

    Finally, sometimes you want to put a newline in certain places, and sed and ed irritatingly don't let you do it easily. For that you use tr. Read the man page for a full description. The most useful incantation of tr is:
    tr char '\012'
    
    When char is a regular expression for a character. This will replace all instances of that char with a newline. So, for example, to replace all capital letters with newlines, you do:
    UNIX> tr '[A-Z]' '\012' < ussmall
         1  
    rnie 
    ls -- -4     -4    0
         2  
    olin 
    ontgomerie -- -3     -4    1
         3  
    om 
    ehman -- -2     -4    2
         4  
    eff 
    aggert -- +1     -4    5
         5  
    ob 
    way -- +2     -4    6
    UNIX>