CSC 282.01, Class 13: A Unix Miscellany

Overview

• Preliminaries
  • Questions
• A problem
• Refresher: Selecting files
• Backticks
• Refresher: Shell variables
• The joy of sed
• Regular expressions
• Refresher: Loops
• Command-line history

Good things to do

• Today’s CS Extra! (Science 2022)

A problem

Used CamelCase rather than snake_style for field names.

Problem: Want to take every appropriate file (except db, which needs migration) and change TranslationLanguage to translation_language.

• Subproblem 1: How do I identify groups of files, not all of which are in the same directory?
• Subproblem 2: How do I transmit that list of files to another program to work with?
• Subproblem 3: How do I do the editing/modification efficiently?

Refresher: Selecting files

• *.txt means all the files that end with txt
• grep -l PATTERN *.txt means all the files with PATTERN in their bodies
• find . -name "*.txt" -print

Backticks

vi `find . -name "*.txt" -print`

Captures the output of one command, lets you use it as the command-line parameters to another command.

Refresher: Shell variables

name="Sam"
var="Hello $name"
echo $var

Most shell variables are only available to the shell, and not to subprocesses.

export var
export var=VALUE

makes the variable available to subprocesses.

We can use backticks with shell variables.

name=`command`

Detour: Substitution expressions in vi

• First instance on each line: :1,$s/Hello/Hi/
• Instance at the left margin on each line: :1,$s/^Hello/Hi/
• All instances on each line: :1,$s/Hello/Hi/g

The joy of sed

sed - Stream editor

sed -e "s/Hello/Hi" file > newfile

Note that he default behavior of sed is to keep the original file unchanged and send the modified version to stdout.

You can add -i for in-place, at least in GNU sed.

Regular expressions

This is the sed syntax, or parts thereof

• Most characters (except special characters, like ) match themselves
  • a matches the letter a
  • 3 matches the character 3
• ^ matches the start of line
• $ matches the end of the line
• \^ matches the caret character
• \\ matches the backslash character
• . matches any one character
• [abc] matches any one character in the group within the brackets
• [a-z] matches any lowercase letter. (ASCII ranges)
• * matches zero or more copies of the previous regular expression
  • a* is zero or more a’s
  • aa* is one or more a’s
• \( and \) serve the same role as parentheses in arithmetic
  • \(aa\)* matches even numbers of a’s
• In the replacement, \1 is the first parenthesized expression, \2 is the second, and so on and so forth

To duplicate any single letter that happens after H

s/H\(.\)/H\1\1/g

Refresher: Loops

for VAR in LIST; do
  COMMAND
done

while (TEST) do
  COMMAND
done

To rename all of my files that contain the word “Sam” to end with .sam rather than .txt.

for file in `grep -l Sam *.txt`; do
  newfile=`echo $file | sed -e 's/txt$/sam/'`
  mv $file $newfile
done

Sam’s favorite use of while: Track the status of downloads

while (1) do
  date
  ls -l FILE
  sleep 60
done

Command-line history