In Diane Abbot's case, there are two obvious possibilities - she is black and female. Those are two groups which have in the past suffered discrimination and, more recently, have been the subject of positive discrimination. However, when "token" is used, it's in a negative or derogatory sense.
A political party may be criticised for only choosing white candidates to stand in a local election. Political parties don't like criticism so sometimes they will add an extra candidate who is of an ethnic minority, in order to placate those criticising them. When a person has been included simply because they are a particular gender, religion, sexual orientation or colour, rather than on merit, then they are frequently considered to be the "token" ... (woman, Hindu, homosexual, black person etc etc).
Of course, generally it is not the case that they have been selected as the token anything, but selected on merit. However, an accusation of tokenism is frequently thrown at companies, political parties, schools, and many more.