I know what bay means but this combination of bay+window is confusing to me. This is the expression: someone walks with his bay window going ahead of him
This is the exterior view of a bay window, and here is what a bay window looks like from the inside. It is basically a multi-paned window that projects outward from the main walls of a building.
There is a link concerning the matter in question.
bay window: Definition from Answers.com