You could, but I'd suggest you use "It's up to you" which most natives will understand.So, could I say to a new employee, who is learning his job: "William, you´re in charge of this group, so it´s down to you to cater to their needs, and do this, and this, and this,...? I mean, would it mean the same as it´s your responsibility to cater to their needs..?