CSC362 2004S, Class 5: From STUPID to PAL

Admin:
* Public embarassment: Where are Daniel, Choed?
* DO NOT LEAVE THROUGH MR. WALKER'S CLASS
* Start reading chapter 3
* Project, phase 1, assigned
* We may get a "real compiler writer" here (MLton author)
* EBoards?
* Convo
* Mental health
* Sam not available this weekend

Overview:
* Stages of compilation and products thereof
* Appropriate design of intermediate representations
* Hand-coded lexical analysis
* Hand-coded syntactic analysis
* Hand-coded semantic analysis

The stages of compilations and their products

Sequence of characters
    |
LEXER / TOKENIZER
    |
Sequence of tokens
    |
PARSER
    |
Syntax tree
    |
SEMANTIC ANALYZER / TYPE CHECKER
    |
Annotated syntax tree
    |
CODE GENERATOR
    |
Intermediate code (PAL)
    |
OPTIMIZER
    |
Intermediate code (PAL)
    |
REGISTER ANALYSIS (based on target architecture)
    |
Intermediate code (PAL)
    |
TRANSLATION
    |
"Real" Assembly code
    |
OPTIMIZER
    |
"Real" Assembly code

How (at the interface level in Java) do we describe this translation?
* Interface or class for "input characters"
* Interface or class for "output tokens"
* Interface or class for "token"
* Interface or class for the tokenizer

What might we do with tokens?
* Print them to the screen
* Understand what purpose they serve
  * Need to determine the "type" of the token
* Need to be able to distinguish different tokens of the same "type"
  * Different identifiers
  * Different numbers
* Need to be able to associate information with particular tokens

Design question:
* Do tokens have parents and children, or do they belong in nodes
  that have parent's and children?

public class Token
{
  ??? tokentype;

  public String toString();
} // interface Token

public interface Tokenizer
{
  Token getNext()
    throws endOfInputException;
} // interface Tokenizer