Backtracking
Core Titles
Key headlines and terms for quick recall- Backtracking — systematic search of solution space with pruning
- Built on recursion + DFS of a state-space tree
- Pruning — abandon a branch when constraints violated
- Classics: N-Queens, Sudoku, subset sum, permutations, maze solving
- Branch and Bound = backtracking + cost bound for optimisation
- Worst-case still exponential, but pruning makes it tractable in practice
Basic Idea
What it is, why it matters, how it worksWhat is backtracking?
Backtracking explores a state-space tree of partial candidates one branch at a time (DFS) and prunes branches that cannot lead to a valid / better solution. When a branch is rejected, it backs up and tries another.
It is a refined brute force — smart enough to skip obviously hopeless subtrees.
Template
def backtrack(partial):
if is_complete(partial):
record(partial); return
for choice in candidates(partial):
if is_valid(partial + [choice]):
backtrack(partial + [choice])
# undo choice if you're tracking it explicitly
Key ingredients
- State — the partial solution.
- Choices — possible extensions of the state.
- Constraints — conditions that any complete solution must satisfy.
- Pruning — abandon a branch when constraints can't be satisfied.
- Goal check — when partial = full solution.
Classic backtracking problems
1. N-Queens. Place queens on an board so none attacks another.
def solve(col, board):
if col == N: yield board.copy()
for row in range(N):
if not attacked(row, col, board):
board[col] = row
yield from solve(col + 1, board)
Pruning: don't place a queen on a square already attacked.
2. Sudoku. Recursively fill empty cells; backtrack on conflicts.
3. Maze / path finding. DFS with backtracking when dead-end.
4. Subset sum. Find subset summing to target.
def subset_sum(items, target, partial=[]):
if sum(partial) == target: yield partial
if sum(partial) > target: return
for i, x in enumerate(items):
yield from subset_sum(items[i+1:], target, partial + [x])
5. Permutations / combinations. Generate by swapping or DFS.
6. Knight's tour. Visit every square on a chessboard exactly once.
7. Crossword puzzle generation.
8. Graph colouring. Assign colours such that adjacent vertices differ.
Pruning strategies
- Constraint propagation — early check of constraints.
- Forward checking — eliminate impossible future choices.
- Heuristic ordering — try most-constrained variable first (MRV in CSP).
- Memoisation — cache already-explored states.
Branch and Bound
For optimisation problems: in addition to feasibility constraints, keep the best-so-far cost and bound the best achievable cost down a branch. If the bound is worse than the current best, prune.
Used for 0/1 Knapsack, TSP, ILP.
Complexity
Worst-case still exponential ( where = branching factor, = depth). But pruning often makes it work in practice on instances that would defeat brute force.
Backtracking vs DP vs Greedy
| Paradigm | When |
|---|---|
| Greedy | Locally best = globally best |
| DP | Overlapping sub-problems + opt substructure |
| Backtracking | Search a state space with feasibility constraints; no overlap |
| Branch & Bound | Backtracking with cost bound for optimisation |
Why in Data Science / AI
- Constraint satisfaction problems (timetabling, scheduling).
- Game-tree search — chess, Go (Minimax + α-β pruning is backtracking with bound).
- SAT solvers — DPLL is backtracking with unit propagation.
- Hyperparameter search trees — sometimes use backtracking-style pruning.
Mind Map
Visual structure of the conceptBACKTRACKING
├── DFS through state-space tree
├── Recursive template
│ ├── Try each choice
│ ├── Recurse
│ └── Undo (back up)
├── Pruning
│ ├── Constraint propagation
│ ├── Forward checking
│ └── MRV heuristic
├── Classics
│ ├── N-Queens
│ ├── Sudoku
│ ├── Maze
│ ├── Subset sum
│ ├── Permutations
│ ├── Knight's tour
│ └── Graph colouring
└── Branch & Bound (with cost bound)
├── 0/1 Knapsack
├── TSP
└── ILP
Exam Q&A
Part A (2 marks) and Part B (20 marks) style questionsPart A (2 marks each)
Q1. Define backtracking. A systematic search of the solution space by building partial solutions, abandoning ("backing up") any partial solution that cannot be completed into a valid full solution.
Q2. Give two classical backtracking problems.
- N-Queens — place queens so none attacks another.
- Sudoku — fill empty cells respecting row, column, and box constraints.
Q3. What is the difference between backtracking and brute force? Brute force enumerates every candidate; backtracking explores the same space but prunes branches that violate constraints early — saving exponential work in practice.
Part B (20 marks)
Q. Explain backtracking as an algorithm design technique. Discuss its template, illustrate with the N-Queens problem, and relate it to Branch and Bound for optimisation.
Backtracking idea.
Backtracking is a smart brute force. It builds partial solutions by exploring a tree of choices depth-first; whenever a partial solution violates a constraint, the algorithm abandons that subtree and backs up to try a different choice.
Template.
backtrack(partial):
if complete(partial):
record(partial); return
for choice in candidates(partial):
if feasible(partial + choice):
backtrack(partial + choice)
# undo choice if needed
Key ingredients.
- State — partial solution so far.
- Choices — candidate extensions.
- Constraints — must-hold for any complete solution.
- Pruning — drop infeasible branches early.
- Goal check — complete solution found.
Example — N-Queens.
Problem: place queens on an chessboard so that no two attack each other (no two share row, column, or diagonal).
State. List board[col] = row of the queen in column col.
Choices. For each column, try each row.
Constraints.
- No two queens in the same row.
- No two on the same diagonal.
Pseudo-code.
def attacked(row, col, board):
for c in range(col):
r = board[c]
if r == row or abs(r - row) == abs(c - col):
return True
return False
def solve(col=0, board=[None]*N, solutions=[]):
if col == N:
solutions.append(board[:]); return
for row in range(N):
if not attacked(row, col, board):
board[col] = row
solve(col + 1, board, solutions)
Example for . Two solutions exist: [1,3,0,2] and [2,0,3,1]. Without backtracking we would explore placements; pruning cuts this to a handful.
Pruning strategies.
- Constraint propagation — at each step check constraints.
- Forward checking — eliminate impossible future choices.
- MRV (Minimum Remaining Values) — assign the most-constrained variable next.
- Memoisation — cache visited states.
Branch and Bound.
For optimisation problems, backtracking is extended with a cost bound:
- Keep best-so-far solution cost.
- Down each branch, compute a lower bound on the achievable cost.
- If the lower bound is worse than the current best, prune the branch.
Used for:
- 0/1 Knapsack — lower bound = current value + fractional knapsack of remaining.
- Travelling Salesman Problem (TSP) — lower bound via MST or assignment relaxation.
- Integer Linear Programming (ILP) — solve LP relaxation as bound.
Comparison with other paradigms.
| Paradigm | Use when |
|---|---|
| Brute force | Small , no structure |
| Greedy | Locally best = globally best |
| DP | Overlapping sub-problems |
| Backtracking | State-space search with feasibility constraints; no overlap |
| Branch & Bound | Backtracking + cost bounds for optimisation |
Complexity. Worst-case still exponential ( where = branching factor, = depth) — but pruning often makes real-world instances tractable.
Applications.
- Constraint Satisfaction Problems (CSP) — timetables, sudoku.
- SAT solvers — DPLL is backtracking with unit propagation.
- Game-tree search — minimax + α-β pruning is backtracking with cost bound (used in chess, Go).
- AI planning — STRIPS-style planners.
- Hyperparameter trees in AutoML.
Take-away. Backtracking + pruning is one of the most general algorithmic paradigms — used wherever a problem requires searching a combinatorial space with constraints. Branch and Bound extends it to optimisation by adding cost-based pruning.