Sit = be in a position in which the lower part of the body is resting on a seat or other type of support, with the upper part of the body vertical:
>> We sit on the chairs in the dining room.
Sit down = to move your body to a sitting position
>> When I come home, I sit down on the sofa to watch TV
They can really be used interchangeably.