Initially I was just using a template file, but I was getting weird results with some pages having no header/footer, but then later on the header/footer would reappear with correct page numbers.
So I switched to trying to specify the header/footer as was referenced here.