Here in the US I hear people use 'down' and 'up' referring to the geographical locations, for example
How's the weather up there?
How was your trip down to Florida?
Is it common for Americans to think of a place being north or south of their current location?
If you can't tell if the place is south or north, do you replace it with 'over here/there'?
Also, I noticed that 'down' is used more often than 'up'. As an example, you can walk down the street but I rarely hear about walking up the street. Does this has anything to do with travelling down being historically more important then travelling up?
Yes, "up" and "down" are often used to refer to geographical north and south. "East" and "west" are often referred to as "over" there. But there are exceptions. In Chicago "Downtown" refers to the central business district. One goes "Downtown" no matter what direction it is. In New York City, "uptown" and "downtown" usually refer to north and south. "Crosstown" usually is used for east and west.
We walk "up the street" as well as "down the street", but "down the street" is more common.