PGD01C02
Module 5 · Data Structures and Algorithm Design

Backtracking

Core Titles
Key headlines and terms for quick recall
  • Backtracking — systematic search of solution space with pruning
  • Built on recursion + DFS of a state-space tree
  • Pruning — abandon a branch when constraints violated
  • Classics: N-Queens, Sudoku, subset sum, permutations, maze solving
  • Branch and Bound = backtracking + cost bound for optimisation
  • Worst-case still exponential, but pruning makes it tractable in practice
Basic Idea
What it is, why it matters, how it works

What is backtracking?

Backtracking explores a state-space tree of partial candidates one branch at a time (DFS) and prunes branches that cannot lead to a valid / better solution. When a branch is rejected, it backs up and tries another.

It is a refined brute force — smart enough to skip obviously hopeless subtrees.

Template

def backtrack(partial):
    if is_complete(partial):
        record(partial); return
    for choice in candidates(partial):
        if is_valid(partial + [choice]):
            backtrack(partial + [choice])
            # undo choice if you're tracking it explicitly

Key ingredients

  1. State — the partial solution.
  2. Choices — possible extensions of the state.
  3. Constraints — conditions that any complete solution must satisfy.
  4. Pruning — abandon a branch when constraints can't be satisfied.
  5. Goal check — when partial = full solution.

Classic backtracking problems

1. N-Queens. Place NN queens on an N×NN \times N board so none attacks another.

def solve(col, board):
    if col == N: yield board.copy()
    for row in range(N):
        if not attacked(row, col, board):
            board[col] = row
            yield from solve(col + 1, board)

Pruning: don't place a queen on a square already attacked.

2. Sudoku. Recursively fill empty cells; backtrack on conflicts.

3. Maze / path finding. DFS with backtracking when dead-end.

4. Subset sum. Find subset summing to target.

def subset_sum(items, target, partial=[]):
    if sum(partial) == target: yield partial
    if sum(partial) > target: return
    for i, x in enumerate(items):
        yield from subset_sum(items[i+1:], target, partial + [x])

5. Permutations / combinations. Generate by swapping or DFS.

6. Knight's tour. Visit every square on a chessboard exactly once.

7. Crossword puzzle generation.

8. Graph colouring. Assign colours such that adjacent vertices differ.

Pruning strategies

  • Constraint propagation — early check of constraints.
  • Forward checking — eliminate impossible future choices.
  • Heuristic ordering — try most-constrained variable first (MRV in CSP).
  • Memoisation — cache already-explored states.

Branch and Bound

For optimisation problems: in addition to feasibility constraints, keep the best-so-far cost and bound the best achievable cost down a branch. If the bound is worse than the current best, prune.

Used for 0/1 Knapsack, TSP, ILP.

Complexity

Worst-case still exponential (O(bd)O(b^d) where bb = branching factor, dd = depth). But pruning often makes it work in practice on instances that would defeat brute force.

Backtracking vs DP vs Greedy

ParadigmWhen
GreedyLocally best = globally best
DPOverlapping sub-problems + opt substructure
BacktrackingSearch a state space with feasibility constraints; no overlap
Branch & BoundBacktracking with cost bound for optimisation

Why in Data Science / AI

  • Constraint satisfaction problems (timetabling, scheduling).
  • Game-tree search — chess, Go (Minimax + α-β pruning is backtracking with bound).
  • SAT solvers — DPLL is backtracking with unit propagation.
  • Hyperparameter search trees — sometimes use backtracking-style pruning.
Mind Map
Visual structure of the concept
BACKTRACKING
├── DFS through state-space tree
├── Recursive template
│   ├── Try each choice
│   ├── Recurse
│   └── Undo (back up)
├── Pruning
│   ├── Constraint propagation
│   ├── Forward checking
│   └── MRV heuristic
├── Classics
│   ├── N-Queens
│   ├── Sudoku
│   ├── Maze
│   ├── Subset sum
│   ├── Permutations
│   ├── Knight's tour
│   └── Graph colouring
└── Branch & Bound (with cost bound)
    ├── 0/1 Knapsack
    ├── TSP
    └── ILP
Exam Q&A
Part A (2 marks) and Part B (20 marks) style questions

Part A (2 marks each)

Q1. Define backtracking. A systematic search of the solution space by building partial solutions, abandoning ("backing up") any partial solution that cannot be completed into a valid full solution.

Q2. Give two classical backtracking problems.

  1. N-Queens — place NN queens so none attacks another.
  2. Sudoku — fill empty cells respecting row, column, and box constraints.

Q3. What is the difference between backtracking and brute force? Brute force enumerates every candidate; backtracking explores the same space but prunes branches that violate constraints early — saving exponential work in practice.


Part B (20 marks)

Q. Explain backtracking as an algorithm design technique. Discuss its template, illustrate with the N-Queens problem, and relate it to Branch and Bound for optimisation.

Backtracking idea.

Backtracking is a smart brute force. It builds partial solutions by exploring a tree of choices depth-first; whenever a partial solution violates a constraint, the algorithm abandons that subtree and backs up to try a different choice.

Template.

backtrack(partial):
    if complete(partial):
        record(partial); return
    for choice in candidates(partial):
        if feasible(partial + choice):
            backtrack(partial + choice)
            # undo choice if needed

Key ingredients.

  1. State — partial solution so far.
  2. Choices — candidate extensions.
  3. Constraints — must-hold for any complete solution.
  4. Pruning — drop infeasible branches early.
  5. Goal check — complete solution found.

Example — N-Queens.

Problem: place NN queens on an N×NN \times N chessboard so that no two attack each other (no two share row, column, or diagonal).

State. List board[col] = row of the queen in column col.

Choices. For each column, try each row.

Constraints.

  • No two queens in the same row.
  • No two on the same diagonal.

Pseudo-code.

def attacked(row, col, board):
    for c in range(col):
        r = board[c]
        if r == row or abs(r - row) == abs(c - col):
            return True
    return False

def solve(col=0, board=[None]*N, solutions=[]):
    if col == N:
        solutions.append(board[:]); return
    for row in range(N):
        if not attacked(row, col, board):
            board[col] = row
            solve(col + 1, board, solutions)

Example for N=4N = 4. Two solutions exist: [1,3,0,2] and [2,0,3,1]. Without backtracking we would explore 44=2564^4 = 256 placements; pruning cuts this to a handful.

Pruning strategies.

  • Constraint propagation — at each step check constraints.
  • Forward checking — eliminate impossible future choices.
  • MRV (Minimum Remaining Values) — assign the most-constrained variable next.
  • Memoisation — cache visited states.

Branch and Bound.

For optimisation problems, backtracking is extended with a cost bound:

  • Keep best-so-far solution cost.
  • Down each branch, compute a lower bound on the achievable cost.
  • If the lower bound is worse than the current best, prune the branch.

Used for:

  • 0/1 Knapsack — lower bound = current value + fractional knapsack of remaining.
  • Travelling Salesman Problem (TSP) — lower bound via MST or assignment relaxation.
  • Integer Linear Programming (ILP) — solve LP relaxation as bound.

Comparison with other paradigms.

ParadigmUse when
Brute forceSmall nn, no structure
GreedyLocally best = globally best
DPOverlapping sub-problems
BacktrackingState-space search with feasibility constraints; no overlap
Branch & BoundBacktracking + cost bounds for optimisation

Complexity. Worst-case still exponential (O(bd)O(b^d) where bb = branching factor, dd = depth) — but pruning often makes real-world instances tractable.

Applications.

  • Constraint Satisfaction Problems (CSP) — timetables, sudoku.
  • SAT solvers — DPLL is backtracking with unit propagation.
  • Game-tree search — minimax + α-β pruning is backtracking with cost bound (used in chess, Go).
  • AI planning — STRIPS-style planners.
  • Hyperparameter trees in AutoML.

Take-away. Backtracking + pruning is one of the most general algorithmic paradigms — used wherever a problem requires searching a combinatorial space with constraints. Branch and Bound extends it to optimisation by adding cost-based pruning.