The whole unit is called a tap. The part which you turn the water on is called a handle. Handles come in all shapes. There are the long lever handles called "tap levers". There are the the classic cross handles and rounded knob-shaped handles(knob handles?).
In AmE all three options would work colloquially, although formally only handle or knob would work.
While the valve is technically the internal component which regulates the water flow, it's not uncommon to refer to the handle as a valve, since turning it regulates the valve.