Unit II - Context Free Grammar

Understanding Context-Free Languages, Grammars, Derivation Trees, Ambiguity, and Normal Forms

1. Introduction to Context Free Grammar (CFG)

Context-Free Grammar (CFG) is a formal grammar used to generate context-free languages. CFGs are more powerful than regular expressions and can describe languages that regular languages cannot.

Why do we need CFG?

Regular languages cannot handle:

  • Nested structures (like balanced parentheses)
  • Counting and matching (like aⁿbⁿ)
  • Recursive patterns (like palindromes)
  • Programming language syntax

Formal Definition of CFG:

A CFG G is defined as G = (V, T, P, S) where:

  • V: Finite set of variables (non-terminals) - usually uppercase A, B, S
  • T: Finite set of terminals (alphabet symbols) - usually lowercase a, b, c
  • P: Finite set of production rules of form A → α where:
    • A ∈ V (left side is a single variable)
    • α ∈ (V ∪ T)* (right side is string of variables and terminals)
  • S: Start symbol (S ∈ V)

Example 1: Simple CFG

Grammar G for language {0ⁿ1ⁿ | n ≥ 1}:

S → 01          (base case)
S → 0S1         (recursive case)
                

Derivations:

  • n=1: S ⇒ 01
  • n=2: S ⇒ 0S1 ⇒ 0011
  • n=3: S ⇒ 0S1 ⇒ 00S11 ⇒ 000111

Example 2: Balanced Parentheses

Grammar for valid parentheses strings:

S → ε           (empty string)
S → SS          (concatenation)
S → (S)         (nesting)
                

Examples generated: ε, (), (()), ()(), (()()), (()())

Derivation:

A derivation is a sequence of production rule applications to transform start symbol into terminal string.

  • Direct derivation (⇒): One step using one production
  • Derivation (⇒*): Zero or more steps
  • Positive derivation (⇒⁺): One or more steps

Language Generated by CFG:

L(G) = {w ∈ T* | S ⇒* w}

The set of all terminal strings that can be derived from start symbol.

Key Terminology:

Term Definition Example
Sentential Form Any string derived from start symbol (can have variables) S, 0S1, 00S11
Sentence Sentential form with only terminals 0011, 000111
Leftmost Derivation Always replace leftmost variable first S ⇒ 0S1 ⇒ 00S11 ⇒ 000111
Rightmost Derivation Always replace rightmost variable first S ⇒ 0S1 ⇒ 00S11 ⇒ 000111

2. Regular Grammars

Regular Grammars are restricted CFGs that generate exactly the class of regular languages. They bridge the gap between finite automata and context-free grammars.

Types of Regular Grammars:

1. Right-Linear Grammar:

All productions are of the form:

  • A → wB (variable on the RIGHT)
  • A → w

Where A, B ∈ V and w ∈ T*

Example: Grammar for L = (01)*

S → 01S
S → ε
                

2. Left-Linear Grammar:

All productions are of the form:

  • A → Bw (variable on the LEFT)
  • A → w

Where A, B ∈ V and w ∈ T*

Example: Grammar for L = (10)*

S → S10
S → ε
                

Relationship between Regular Grammars and Finite Automata:

Equivalence Description
Right-Linear ↔ NFA Every right-linear grammar can be converted to NFA and vice versa
Left-Linear ↔ NFA Every left-linear grammar can be converted to NFA and vice versa
Regular Grammar = Regular Language Regular grammars generate exactly regular languages

Converting Right-Linear Grammar to NFA:

Grammar:

S → 0A
S → 1B
A → 0S
A → 1B
B → 0B
B → 1B
B → ε
                

NFA Construction:

  1. Each variable becomes a state
  2. Add a final state F
  3. For production A → wB, add transition from state A to B on input w
  4. For production A → w, add transition from A to F on input w
  5. Start state is S

Key Difference: Regular vs Context-Free:

Feature Regular Grammar Context-Free Grammar
Production form A → wB or A → w (restricted) A → α (any string)
Variables on right At most one, at end/start Any number, anywhere
Recursion Tail recursion only Any recursion pattern
Power Generates regular languages Generates CFL (more powerful)
Can handle Finite memory patterns Nested structures, counting

3. Derivation Trees (Parse Trees)

A Derivation Tree or Parse Tree is a graphical representation of a derivation in a context-free grammar. It shows the hierarchical syntactic structure of a string.

Properties of Derivation Tree:

  1. Root: Start symbol of grammar
  2. Internal Nodes: Variables (non-terminals)
  3. Leaf Nodes: Terminals or ε
  4. Children: Correspond to right side of production
  5. Yield: Concatenation of leaf nodes from left to right

Example: Grammar and its Parse Tree

Grammar:

E → E + E
E → E * E
E → (E)
E → id
                

String: id + id * id

Parse Tree 1:

           E
          /|\
         E + E
         |   |\
        id  E * E
            |   |
           id  id
                

Interpretation: id + (id * id) - multiply first

Parse Tree 2:

           E
          /|\
         E * E
        /|\  |
       E + E id
       |   |
      id  id
                

Interpretation: (id + id) * id - add first

⚠️ This grammar is ambiguous (same string, multiple parse trees)

Relationship between Derivations and Parse Trees:

  • One parse tree can represent multiple derivations
  • Leftmost and rightmost derivations of same string have same parse tree
  • Parse tree abstracts away the order of derivation steps

Uses of Parse Trees:

  • Compilers: Syntax analysis phase
  • Interpreters: Understanding program structure
  • Natural Language Processing: Sentence structure
  • Ambiguity Detection: Multiple parse trees indicate ambiguity

Height of Parse Tree:

The height is the longest path from root to any leaf. It indicates the number of derivation steps needed.

4. Ambiguity in Context-Free Grammars

A CFG is ambiguous if there exists at least one string that has two or more distinct leftmost derivations (or equivalently, two or more parse trees).

Formal Definition:

Grammar G is ambiguous if ∃ a string w ∈ L(G) such that w has:

  • Two or more distinct leftmost derivations, OR
  • Two or more distinct rightmost derivations, OR
  • Two or more distinct parse trees

Example 1: Classic Ambiguous Grammar (Dangling Else)

S → if E then S
S → if E then S else S
S → other
                

Ambiguous String: if E then if E then other else other

Two Parse Trees possible:

  1. else matches with outer if
  2. else matches with inner if

Example 2: Arithmetic Expression Grammar

Ambiguous Grammar:

E → E + E | E - E | E * E | E / E | id
                

String "id + id * id" has multiple parse trees (different precedence)

Unambiguous Grammar (with precedence):

E → E + T | E - T | T
T → T * F | T / F | F
F → (E) | id
                

Now "id + id * id" has unique parse tree: id + (id * id)

Problems with Ambiguity:

  • Multiple meanings for same program
  • Compiler cannot decide which interpretation to use
  • Different execution results possible
  • Security vulnerabilities

Detecting Ambiguity:

Bad News: There is NO algorithm to determine if an arbitrary CFG is ambiguous (undecidable problem!)

Practical Approach:

  • Try to find two different parse trees for same string
  • Check for known ambiguous patterns
  • Use parser generators that detect conflicts

Resolving Ambiguity:

Method Description Example
Rewrite Grammar Create equivalent unambiguous grammar Add precedence levels (E, T, F)
Disambiguation Rules Add external parsing rules Yacc/Bison precedence declarations
Add Associativity Specify left or right association E → E + T (left associative)
Restrict Derivations Use only leftmost/rightmost Parser strategy

Inherently Ambiguous Languages:

Some context-free languages are inherently ambiguous - EVERY grammar for the language is ambiguous!

Example: L = {aⁿbⁿcᵐdᵐ | n,m ≥ 1} ∪ {aⁿbᵐcᵐdⁿ | n,m ≥ 1}

5. Simplification of Context-Free Grammars

Simplification removes useless components from a grammar without changing the language generated. This makes the grammar easier to understand and more efficient to parse.

Types of Useless Symbols:

  1. Non-generating Symbols: Variables that cannot derive any terminal string
  2. Non-reachable Symbols: Symbols that cannot be reached from start symbol

Step 1: Removing Non-Generating Symbols

A variable A is generating if A ⇒* w for some w ∈ T*

Algorithm to Find Generating Symbols:

  1. Initialize: Mark all terminals as generating
  2. Iterate: Mark variable A as generating if there exists production A → α where all symbols in α are already marked generating
  3. Repeat: Until no new variables are marked
  4. Remove: All variables and productions containing non-generating symbols

Example:

Original Grammar:
S → AB | a
A → b
B → CD
C → c
D → d | E
E → EF
                

Analysis:

  • Round 1: Terminals a, b, c, d are generating
  • Round 2: A → b (b is terminal), so A is generating
  • Round 2: C → c (c is terminal), so C is generating
  • Round 2: D → d (d is terminal), so D is generating
  • Round 3: B → CD (both C, D generating), so B is generating
  • Round 4: S → AB (both generating) and S → a, so S is generating
  • E and F are NOT generating (E → EF has E on right, circular)
After Removing Non-Generating:
S → AB | a
A → b
B → CD
C → c
D → d
                

Step 2: Removing Non-Reachable Symbols

A symbol X is reachable if S ⇒* αXβ for some α, β

Algorithm to Find Reachable Symbols:

  1. Initialize: Mark S (start symbol) as reachable
  2. Iterate: If A is reachable and A → α is a production, mark all symbols in α as reachable
  3. Repeat: Until no new symbols are marked
  4. Remove: All unreachable symbols and their productions

Example:

After removing non-generating:
S → AB | a
A → b
B → CD
C → c
D → d
E → e        (separate component)
                

Reachable from S: S, A, B, C, D, a, b, c, d

E is unreachable!

Final Simplified Grammar:
S → AB | a
A → b
B → CD
C → c
D → d
                

Step 3: Removing ε-Productions

An ε-production is a production of the form A → ε

A variable A is nullable if A ⇒* ε

Algorithm to Remove ε-Productions:

  1. Find all nullable variables
  2. For each production A → α containing nullable variables, add new productions with all combinations of including/excluding nullable variables
  3. Remove all ε-productions (except S → ε if ε ∈ L(G))

Example:

Original:
S → AB
A → aA | ε
B → bB | ε
                

Nullable: A, B, S

After Removing ε-Productions:
S → AB | A | B | ε
A → aA | a
B → bB | b
                

Step 4: Removing Unit Productions

A unit production is of the form A → B (single variable)

Algorithm:

  1. Find all pairs (A, B) such that A ⇒* B using only unit productions
  2. For each such pair and each non-unit production B → α, add A → α
  3. Remove all unit productions

Example:

Original:
S → A
A → B
B → a | b
                
After Removing Unit Productions:
S → a | b
A → a | b
B → a | b
                

Order of Simplification Steps:

  1. Remove ε-productions
  2. Remove unit productions
  3. Remove non-generating symbols
  4. Remove non-reachable symbols

⚠️ Order matters! Removing in wrong order may not fully simplify.

6. Normal Forms: Chomsky and Greibach

Normal Forms are standardized formats for CFGs that make certain operations and proofs easier while preserving the language generated.

A. Chomsky Normal Form (CNF)

A CFG is in Chomsky Normal Form if all productions are of the form:

  • A → BC: Two variables (no more, no less)
  • A → a: Single terminal
  • S → ε: Only if ε is in language, and S doesn't appear on right side

Theorem:

Every context-free language can be generated by a grammar in Chomsky Normal Form.

Uses of CNF:

  • Parsing algorithms (CYK algorithm)
  • Proving properties about CFGs
  • Simplifying grammar analysis
  • Determining if string is in language (O(n³) time)

Converting to CNF: Steps

  1. Start Symbol: Add new start symbol S₀ → S
  2. Eliminate ε-productions: Remove all A → ε
  3. Eliminate unit productions: Remove all A → B
  4. Replace terminals: For production with mixed terminals and variables, replace terminal a with new variable Ta and add Ta → a
  5. Break long productions: For A → B₁B₂...Bₙ (n > 2), introduce new variables:
    • A → B₁C₁
    • C₁ → B₂C₂
    • ...
    • Cₙ₋₂ → Bₙ₋₁Bₙ

Example: Convert to CNF

Original Grammar:
S → ASA | aB
A → B | S
B → b | ε
                

Step 1: Eliminate ε-productions (B is nullable)

S → ASA | aB | a
A → B | S | ε
B → b
                

Step 2: Eliminate ε-productions (A is nullable)

S → ASA | AS | SA | S | aB | a
A → B | S
B → b
                

Step 3: Eliminate unit productions

S → ASA | AS | SA | aB | a | b
A → b
B → b
                

Step 4: Replace terminals in mixed productions

S → ASA | AS | SA | TA B | a | b
A → b
B → b
TA → a
                

Step 5: Break long productions

S → AC₁ | AS | SA | TC₂ | a | b
C₁ → SA
C₂ → AB
A → b
B → b
TA → a
TC → a
                

Final CNF:

S → AC₁ | AS | SA | TC₂ | a | b
C₁ → SA
C₂ → AB
A → b
B → b
TA → a
                

Properties of CNF:

  • Derivation tree is a binary tree
  • For string of length n, derivation has exactly 2n-1 steps
  • Height of derivation tree ≥ log₂(n)
  • Useful for membership testing

B. Greibach Normal Form (GNF)

A CFG is in Greibach Normal Form if all productions are of the form:

  • A → aα: Terminal followed by zero or more variables
  • S → ε: Only if ε is in language

Formal Definition:

Every production is of the form:

A → aB₁B₂...Bₙ where:

  • a ∈ T (terminal)
  • B₁, B₂, ..., Bₙ ∈ V (variables, n ≥ 0)

Example: Grammar in GNF

S → aAB | bB | a
A → aA | bB | a
B → bB | b
                

✓ Every production starts with a terminal

✓ Followed by any number of variables (can be zero)

Uses of GNF:

  • Every derivation step generates exactly one terminal
  • For string of length n, exactly n derivation steps
  • Useful for constructing pushdown automata
  • One-to-one correspondence between PDA and GNF grammars

Comparison: CNF vs GNF

Feature Chomsky Normal Form Greibach Normal Form
Production form A → BC or A → a A → aα (α is variables)
Terminals Alone or at end of derivation First symbol in every production
Variables Exactly 2 per production Any number (≥0) after terminal
Derivation steps 2n-1 for string length n Exactly n for string length n
Conversion ease Easier to convert to More complex conversion
Main use Parsing algorithms (CYK) PDA construction

Important Results:

  • Every CFL without ε can be generated by grammar in CNF
  • Every CFL without ε can be generated by grammar in GNF
  • CNF and GNF do not increase expressive power
  • They are just convenient normal forms for analysis

⚡ Quick Revision (Unit II Cheat Sheet)

CFG Essentials

  • CFG: G=(V,T,P,S), productions A→α with A∈V, α∈(V∪T)*.
  • Language: L(G) = { w ∈ T* | S ⇒* w }.
  • Derivations: Leftmost vs Rightmost; same parse tree.
  • Ambiguity: ∃ w with ≥2 distinct parse trees/derivations.

Simplification Order

  1. Remove ε-productions (except S→ε if needed).
  2. Remove unit productions (A→B).
  3. Remove non-generating symbols.
  4. Remove non-reachable symbols.

Normal Forms

  • CNF: A→BC | a (plus S→ε optionally). Steps: new S₀, ε, unit, terminals→symbols, binarize.
  • GNF: A→aα (terminal first), no left recursion; each step yields 1 terminal.

Regular Grammars

  • Right-linear: A→wB | w; Left-linear: A→Bw | w. Generate exactly Regular Languages.
  • Convertible to NFA and vice versa.