1- If it is ok with him.... (OK) with = he agrees with it.
2- If it is ok by him.... (OK) by = he agrees to (do) it.
3- It is ok with me. (OK)
4- It is ok by me. (OK)
5- There is not a problem in starting on the twelfth. (OK)
=> There's no problem with starting on the twelfth.