Both the phrases are true and used in English. Essence is a noun and the phrase Essence of means: The most important ingredient; the crucial element or the sum and substance of something:
The essence of his talk remains unclear to most the audience.
In essence is an idiom meaning: by its very nature; essentially.
He is in essence an honest person