Some time ago I picked up an anthology called "The Best Ghost Stories", to get some of HPL's stories that I didn't have. Now, this anthology features stories by Edgar Allan Poe, Algernon Blackwood, Sheridan Le Fanu, E.F. Benson, and M.R. James.
M.R. James is considered the originator of the "antiquarian ghost story", and his protagonists share many similarities with those of Lovecraft: bachelors with an antiquarian bent, and who's curiosity causes some ancient evil/curse to be unleashed/uncovered.
For those unfamiliar with his work, I'd like to recommend "Count Magnus" & "The Ash-Tree" to get you started.
The late, great Sir Christopher Lee read four of James' stories in a candle-lit room in King's College in a BBC production for Christmas in 2000, so if you can find them floating around online, that should be worth checking out as well.