"Out of the way" describes a place that is not easy to find; it is not located on a main street or in a largely populated area. It is somewhat remote and quiet.
"Hole in the wall" describes a place that is small and not fancy or elegant; it might be downright shabby. For example, my favorite Chinese restaurant in my hometown can be described as a hole in the wall - from the outside it looks like a tiny, abandoned factory; inside it's a little dark and the furniture is old, the carpet is worn. But the food is delicious, so you can't always judge a place on appearance alone.