I am not a professional teacher. The prolem here is the proper usage of inifinitive. hsb will do well to visit a site on gerund and infinitive. My observations are as follows.
A- I was to go. It is not correct to use an infinite after an auxiliary verb.
B- I was about to go. It is ok, because of the preposition ‘about’ which has an infinitive as its object. To make it clear, let us take the example of a complete sentence:
Last month I was about to go on a pilgrimage.
C - I was going to go. It is also ok as the infinitive is used after the main verb ‘going’. The meaning can be derived from a complete sentence:
Yesterday, I was going by train to go for fishing.
D- I was to have gone. This is grammatically incorrect.