In that sentence, gaze is much better. You gaze at something lovingly or with great fondness or interest; you look at that thing steadily for a period of time because you find it attractive or interesting. If you stare at something, you look at it steadily for a period of time, but usually in disbelief, anger, a complete lack of comprehension, etc. Using stare in the example above is so awkward I'd have to say it's wrong, especially with the adverb lovingly. That's a dead giveaway for a native speaker, because gaze lovingly is a common collocation. I can't think of any situation where I'd say someone stared lovingly at something. It's virtually an oxymoron.
Student or Learner