Outline of Class 35: Regular Expressions, Revisited

Miscellaneous

• As most of you know, there is a lot of student discussion about student representatives to the presidential search board. In case you hadn't heard, there is an election from 8-4 today in the campus post office. You can find the candidates slates by fingering jones1 or on a web page at http://www.math.grin.edu/~rebelsky/Institution/student.presboard.html
• Reports! On Wednesday, I'd like you to give short status reports on your projects. You should plan to spend about five to ten minutes per group reporting on:
  • What your project is (so that your colleagues will know).
  • What the current status of your project is.
  • How that relates to your expect status.
  • What is left to do.
• I've extended the previous class notes to include a fuller discussion of that weird regular expression.

Grading

• I had allocated a lot of my weekend time to grading. However, I also participated in a half-day faculty discussion group on minority faculty, which I felt was important. The homework also took significantly longer to grade than I expected, partially because many of you mistakenly made files unreadable (at least for me).
• I've finished grading assignment seven. They were less good than I would have hoped, particularly given the extra time and comments I gave you. I've also written some general comments.
• I would like to better understand some of the problems you had.
  • I tried to emphasize the need for a peek() operation or its equivalent. Why didn't you add one?
  • I had tried to emphasize the need for unambiguity, both in the assignment and quiz. Why were some things still ambiguous?
  • I tried to give you examples that better illustrated the need for padding and a lot of freedom in padding. Why did you not provide more general techniques?
• If you're unhappy about your grade on this assignment, I'm willling to talk to you about ways to improve it for regrading.
• I have mixed feelings about grading the earlier assignments; for most of it, my main intent on those assignments was to make sure you thought about stuff. I'm not sure that grading will make a difference on HW3, particularly since you each commented on the strengths and weaknesses of each other's assignments. I will, however, grade your comments on other people's code.
• I'd appreciate it if you'd turn in printed copies of your comments from HW4 in class on Wednesday (that way I can grade them over break).
  • Some of you forgot to share them.
  • Some of you have moved them.
  • Some of you forgot to tell me where they are.
• Please remind me if you did assignment eight (Huffman coding, for extra credit).

Filters, Revisited

• I had told you that the main use of filters in Unix is through Unix's piping mechanism.
• However, there are certainly others.
• In Unix you can make the results of one command the parameters to another command by enclosing them in backquotes. For example, if coolfiles gave a list of cool files, I could edit just the cool files with
  vi `coolfiles`
  • Note that this is signficantly different from
    vi coolfiles
    which says to edit a file named coolfiles.
  • Note that this is particularly useful with grep, because you can quickly identify and edit all the files that contain a particular string. For example, when one of you noted that I had links to my .txt files rather than my .html files, I could quickly identify all such erroneous links.
• Unix provides a number of scripting languages that can take advantage of the output of filters and such.
  • In fact, you use a scripting language each time you log in to a Unix machine, since all the shells ( csh, bash, etc.) are actually little languages.
  • Suppose you wanted to do something to each file that contained the text "foobar", but the command you want won't take multiple filenames as arguments. In csh you can use

    foreach file (`grep -l "foobar" *`)
      something $file;
    end
    

• Note also that you can use regular expressions on the command line, although the form of these regular expressions is somewhat different. (In particular, "*" means "match anything" rather than "match repetitions of the previous regular expression".)

More grep Examples

• As we saw last class, there are many things you can do with grep.
• Unfortunately, the regular expressions you write may be difficult to understand without a reader's guide. When possible, try to provide some explanation of the components.
• Let's try to come up with a few more intersting grep problems. problems:
  • Edit all the files in the current directory that include the string ".txt".
  • List all uses of my course pages from within Grinnell, but not from the computer classroom.

sed

• sed's primary purpose is information modification -- it's used to modify files in batch mode.
• The most common use of sed is replacement (or substitution); replacing one form on line by another.
• We'll focus on that use today by considering a number of examples.
• The big one is to convert names in between first last and last,first formats.
  • The second should be somewhat easier.
  • We'll need to come up with some difficult examples.
  • Last names with multiple words (e.g., Werner von Braun)
  • Middle names (e.g., Samuel Alexander Rebelsky)
  • Suffixes (e.g., David J. Smith, Junior)
• Can you come up with your own? Here are some others ...
  • In a paper, convert all citations to hypertext links to the references in the back. You may assume that citations are of the form [#] and that the references are numbered with the number appearing at the beginning of the line.
  • Convert tables between different formats (e.g., tab-based, LaTeX, and HTML)
  • Evan's legendary "check out all the files in this RCS directory"
  • Build a table of contents for a set of HTML files, assuming that each heading is on a line by itself.