Unit II - Context Free Grammar
Understanding Context-Free Languages, Grammars, Derivation Trees, Ambiguity, and Normal Forms
1. Introduction to Context Free Grammar (CFG)
Context-Free Grammar (CFG) is a formal grammar used to generate context-free languages. CFGs are more powerful than regular expressions and can describe languages that regular languages cannot.
Why do we need CFG?
Regular languages cannot handle:
- Nested structures (like balanced parentheses)
- Counting and matching (like aⁿbⁿ)
- Recursive patterns (like palindromes)
- Programming language syntax
Formal Definition of CFG:
A CFG G is defined as G = (V, T, P, S) where:
- V: Finite set of variables (non-terminals) - usually uppercase A, B, S
- T: Finite set of terminals (alphabet symbols) - usually lowercase a, b, c
- P: Finite set of production rules of form A → α where:
- A ∈ V (left side is a single variable)
- α ∈ (V ∪ T)* (right side is string of variables and terminals)
- S: Start symbol (S ∈ V)
Example 1: Simple CFG
Grammar G for language {0ⁿ1ⁿ | n ≥ 1}:
S → 01 (base case)
S → 0S1 (recursive case)
Derivations:
- n=1: S ⇒ 01
- n=2: S ⇒ 0S1 ⇒ 0011
- n=3: S ⇒ 0S1 ⇒ 00S11 ⇒ 000111
Example 2: Balanced Parentheses
Grammar for valid parentheses strings:
S → ε (empty string)
S → SS (concatenation)
S → (S) (nesting)
Examples generated: ε, (), (()), ()(), (()()), (()())
Derivation:
A derivation is a sequence of production rule applications to transform start symbol into terminal string.
- Direct derivation (⇒): One step using one production
- Derivation (⇒*): Zero or more steps
- Positive derivation (⇒⁺): One or more steps
Language Generated by CFG:
L(G) = {w ∈ T* | S ⇒* w}
The set of all terminal strings that can be derived from start symbol.
Key Terminology:
| Term | Definition | Example |
|---|---|---|
| Sentential Form | Any string derived from start symbol (can have variables) | S, 0S1, 00S11 |
| Sentence | Sentential form with only terminals | 0011, 000111 |
| Leftmost Derivation | Always replace leftmost variable first | S ⇒ 0S1 ⇒ 00S11 ⇒ 000111 |
| Rightmost Derivation | Always replace rightmost variable first | S ⇒ 0S1 ⇒ 00S11 ⇒ 000111 |
2. Regular Grammars
Regular Grammars are restricted CFGs that generate exactly the class of regular languages. They bridge the gap between finite automata and context-free grammars.
Types of Regular Grammars:
1. Right-Linear Grammar:
All productions are of the form:
- A → wB (variable on the RIGHT)
- A → w
Where A, B ∈ V and w ∈ T*
Example: Grammar for L = (01)*
S → 01S
S → ε
2. Left-Linear Grammar:
All productions are of the form:
- A → Bw (variable on the LEFT)
- A → w
Where A, B ∈ V and w ∈ T*
Example: Grammar for L = (10)*
S → S10
S → ε
Relationship between Regular Grammars and Finite Automata:
| Equivalence | Description |
|---|---|
| Right-Linear ↔ NFA | Every right-linear grammar can be converted to NFA and vice versa |
| Left-Linear ↔ NFA | Every left-linear grammar can be converted to NFA and vice versa |
| Regular Grammar = Regular Language | Regular grammars generate exactly regular languages |
Converting Right-Linear Grammar to NFA:
Grammar:
S → 0A
S → 1B
A → 0S
A → 1B
B → 0B
B → 1B
B → ε
NFA Construction:
- Each variable becomes a state
- Add a final state F
- For production A → wB, add transition from state A to B on input w
- For production A → w, add transition from A to F on input w
- Start state is S
Key Difference: Regular vs Context-Free:
| Feature | Regular Grammar | Context-Free Grammar |
|---|---|---|
| Production form | A → wB or A → w (restricted) | A → α (any string) |
| Variables on right | At most one, at end/start | Any number, anywhere |
| Recursion | Tail recursion only | Any recursion pattern |
| Power | Generates regular languages | Generates CFL (more powerful) |
| Can handle | Finite memory patterns | Nested structures, counting |
3. Derivation Trees (Parse Trees)
A Derivation Tree or Parse Tree is a graphical representation of a derivation in a context-free grammar. It shows the hierarchical syntactic structure of a string.
Properties of Derivation Tree:
- Root: Start symbol of grammar
- Internal Nodes: Variables (non-terminals)
- Leaf Nodes: Terminals or ε
- Children: Correspond to right side of production
- Yield: Concatenation of leaf nodes from left to right
Example: Grammar and its Parse Tree
Grammar:
E → E + E
E → E * E
E → (E)
E → id
String: id + id * id
Parse Tree 1:
E
/|\
E + E
| |\
id E * E
| |
id id
Interpretation: id + (id * id) - multiply first
Parse Tree 2:
E
/|\
E * E
/|\ |
E + E id
| |
id id
Interpretation: (id + id) * id - add first
⚠️ This grammar is ambiguous (same string, multiple parse trees)
Relationship between Derivations and Parse Trees:
- One parse tree can represent multiple derivations
- Leftmost and rightmost derivations of same string have same parse tree
- Parse tree abstracts away the order of derivation steps
Uses of Parse Trees:
- Compilers: Syntax analysis phase
- Interpreters: Understanding program structure
- Natural Language Processing: Sentence structure
- Ambiguity Detection: Multiple parse trees indicate ambiguity
Height of Parse Tree:
The height is the longest path from root to any leaf. It indicates the number of derivation steps needed.
4. Ambiguity in Context-Free Grammars
A CFG is ambiguous if there exists at least one string that has two or more distinct leftmost derivations (or equivalently, two or more parse trees).
Formal Definition:
Grammar G is ambiguous if ∃ a string w ∈ L(G) such that w has:
- Two or more distinct leftmost derivations, OR
- Two or more distinct rightmost derivations, OR
- Two or more distinct parse trees
Example 1: Classic Ambiguous Grammar (Dangling Else)
S → if E then S
S → if E then S else S
S → other
Ambiguous String: if E then if E then other else other
Two Parse Trees possible:
- else matches with outer if
- else matches with inner if
Example 2: Arithmetic Expression Grammar
Ambiguous Grammar:
E → E + E | E - E | E * E | E / E | id
String "id + id * id" has multiple parse trees (different precedence)
Unambiguous Grammar (with precedence):
E → E + T | E - T | T
T → T * F | T / F | F
F → (E) | id
Now "id + id * id" has unique parse tree: id + (id * id)
Problems with Ambiguity:
- Multiple meanings for same program
- Compiler cannot decide which interpretation to use
- Different execution results possible
- Security vulnerabilities
Detecting Ambiguity:
Bad News: There is NO algorithm to determine if an arbitrary CFG is ambiguous (undecidable problem!)
Practical Approach:
- Try to find two different parse trees for same string
- Check for known ambiguous patterns
- Use parser generators that detect conflicts
Resolving Ambiguity:
| Method | Description | Example |
|---|---|---|
| Rewrite Grammar | Create equivalent unambiguous grammar | Add precedence levels (E, T, F) |
| Disambiguation Rules | Add external parsing rules | Yacc/Bison precedence declarations |
| Add Associativity | Specify left or right association | E → E + T (left associative) |
| Restrict Derivations | Use only leftmost/rightmost | Parser strategy |
Inherently Ambiguous Languages:
Some context-free languages are inherently ambiguous - EVERY grammar for the language is ambiguous!
Example: L = {aⁿbⁿcᵐdᵐ | n,m ≥ 1} ∪ {aⁿbᵐcᵐdⁿ | n,m ≥ 1}
5. Simplification of Context-Free Grammars
Simplification removes useless components from a grammar without changing the language generated. This makes the grammar easier to understand and more efficient to parse.
Types of Useless Symbols:
- Non-generating Symbols: Variables that cannot derive any terminal string
- Non-reachable Symbols: Symbols that cannot be reached from start symbol
Step 1: Removing Non-Generating Symbols
A variable A is generating if A ⇒* w for some w ∈ T*
Algorithm to Find Generating Symbols:
- Initialize: Mark all terminals as generating
- Iterate: Mark variable A as generating if there exists production A → α where all symbols in α are already marked generating
- Repeat: Until no new variables are marked
- Remove: All variables and productions containing non-generating symbols
Example:
Original Grammar:
S → AB | a
A → b
B → CD
C → c
D → d | E
E → EF
Analysis:
- Round 1: Terminals a, b, c, d are generating
- Round 2: A → b (b is terminal), so A is generating
- Round 2: C → c (c is terminal), so C is generating
- Round 2: D → d (d is terminal), so D is generating
- Round 3: B → CD (both C, D generating), so B is generating
- Round 4: S → AB (both generating) and S → a, so S is generating
- E and F are NOT generating (E → EF has E on right, circular)
After Removing Non-Generating:
S → AB | a
A → b
B → CD
C → c
D → d
Step 2: Removing Non-Reachable Symbols
A symbol X is reachable if S ⇒* αXβ for some α, β
Algorithm to Find Reachable Symbols:
- Initialize: Mark S (start symbol) as reachable
- Iterate: If A is reachable and A → α is a production, mark all symbols in α as reachable
- Repeat: Until no new symbols are marked
- Remove: All unreachable symbols and their productions
Example:
After removing non-generating:
S → AB | a
A → b
B → CD
C → c
D → d
E → e (separate component)
Reachable from S: S, A, B, C, D, a, b, c, d
E is unreachable!
Final Simplified Grammar:
S → AB | a
A → b
B → CD
C → c
D → d
Step 3: Removing ε-Productions
An ε-production is a production of the form A → ε
A variable A is nullable if A ⇒* ε
Algorithm to Remove ε-Productions:
- Find all nullable variables
- For each production A → α containing nullable variables, add new productions with all combinations of including/excluding nullable variables
- Remove all ε-productions (except S → ε if ε ∈ L(G))
Example:
Original:
S → AB
A → aA | ε
B → bB | ε
Nullable: A, B, S
After Removing ε-Productions:
S → AB | A | B | ε
A → aA | a
B → bB | b
Step 4: Removing Unit Productions
A unit production is of the form A → B (single variable)
Algorithm:
- Find all pairs (A, B) such that A ⇒* B using only unit productions
- For each such pair and each non-unit production B → α, add A → α
- Remove all unit productions
Example:
Original:
S → A
A → B
B → a | b
After Removing Unit Productions:
S → a | b
A → a | b
B → a | b
Order of Simplification Steps:
- Remove ε-productions
- Remove unit productions
- Remove non-generating symbols
- Remove non-reachable symbols
⚠️ Order matters! Removing in wrong order may not fully simplify.
6. Normal Forms: Chomsky and Greibach
Normal Forms are standardized formats for CFGs that make certain operations and proofs easier while preserving the language generated.
A. Chomsky Normal Form (CNF)
A CFG is in Chomsky Normal Form if all productions are of the form:
- A → BC: Two variables (no more, no less)
- A → a: Single terminal
- S → ε: Only if ε is in language, and S doesn't appear on right side
Theorem:
Every context-free language can be generated by a grammar in Chomsky Normal Form.
Uses of CNF:
- Parsing algorithms (CYK algorithm)
- Proving properties about CFGs
- Simplifying grammar analysis
- Determining if string is in language (O(n³) time)
Converting to CNF: Steps
- Start Symbol: Add new start symbol S₀ → S
- Eliminate ε-productions: Remove all A → ε
- Eliminate unit productions: Remove all A → B
- Replace terminals: For production with mixed terminals and variables, replace terminal a with new variable Ta and add Ta → a
- Break long productions: For A → B₁B₂...Bₙ (n > 2), introduce new variables:
- A → B₁C₁
- C₁ → B₂C₂
- ...
- Cₙ₋₂ → Bₙ₋₁Bₙ
Example: Convert to CNF
Original Grammar:
S → ASA | aB
A → B | S
B → b | ε
Step 1: Eliminate ε-productions (B is nullable)
S → ASA | aB | a
A → B | S | ε
B → b
Step 2: Eliminate ε-productions (A is nullable)
S → ASA | AS | SA | S | aB | a
A → B | S
B → b
Step 3: Eliminate unit productions
S → ASA | AS | SA | aB | a | b
A → b
B → b
Step 4: Replace terminals in mixed productions
S → ASA | AS | SA | TA B | a | b
A → b
B → b
TA → a
Step 5: Break long productions
S → AC₁ | AS | SA | TC₂ | a | b
C₁ → SA
C₂ → AB
A → b
B → b
TA → a
TC → a
Final CNF:
S → AC₁ | AS | SA | TC₂ | a | b
C₁ → SA
C₂ → AB
A → b
B → b
TA → a
Properties of CNF:
- Derivation tree is a binary tree
- For string of length n, derivation has exactly 2n-1 steps
- Height of derivation tree ≥ log₂(n)
- Useful for membership testing
B. Greibach Normal Form (GNF)
A CFG is in Greibach Normal Form if all productions are of the form:
- A → aα: Terminal followed by zero or more variables
- S → ε: Only if ε is in language
Formal Definition:
Every production is of the form:
A → aB₁B₂...Bₙ where:
- a ∈ T (terminal)
- B₁, B₂, ..., Bₙ ∈ V (variables, n ≥ 0)
Example: Grammar in GNF
S → aAB | bB | a
A → aA | bB | a
B → bB | b
✓ Every production starts with a terminal
✓ Followed by any number of variables (can be zero)
Uses of GNF:
- Every derivation step generates exactly one terminal
- For string of length n, exactly n derivation steps
- Useful for constructing pushdown automata
- One-to-one correspondence between PDA and GNF grammars
Comparison: CNF vs GNF
| Feature | Chomsky Normal Form | Greibach Normal Form |
|---|---|---|
| Production form | A → BC or A → a | A → aα (α is variables) |
| Terminals | Alone or at end of derivation | First symbol in every production |
| Variables | Exactly 2 per production | Any number (≥0) after terminal |
| Derivation steps | 2n-1 for string length n | Exactly n for string length n |
| Conversion ease | Easier to convert to | More complex conversion |
| Main use | Parsing algorithms (CYK) | PDA construction |
Important Results:
- Every CFL without ε can be generated by grammar in CNF
- Every CFL without ε can be generated by grammar in GNF
- CNF and GNF do not increase expressive power
- They are just convenient normal forms for analysis
⚡ Quick Revision (Unit II Cheat Sheet)
CFG Essentials
- CFG: G=(V,T,P,S), productions A→α with A∈V, α∈(V∪T)*.
- Language: L(G) = { w ∈ T* | S ⇒* w }.
- Derivations: Leftmost vs Rightmost; same parse tree.
- Ambiguity: ∃ w with ≥2 distinct parse trees/derivations.
Simplification Order
- Remove ε-productions (except S→ε if needed).
- Remove unit productions (A→B).
- Remove non-generating symbols.
- Remove non-reachable symbols.
Normal Forms
- CNF: A→BC | a (plus S→ε optionally). Steps: new S₀, ε, unit, terminals→symbols, binarize.
- GNF: A→aα (terminal first), no left recursion; each step yields 1 terminal.
Regular Grammars
- Right-linear: A→wB | w; Left-linear: A→Bw | w. Generate exactly Regular Languages.
- Convertible to NFA and vice versa.