What's going on with my toilet

Hi folks. Have done a little plumbing but not sure what happening here. Moved into new place and guest bathroom toilet was silently spilling into the overflow. I adjusted the refill mechanism so it would stop refilling about 2 inches below the overflow. A few days later I checked and it was overflowing again. When I flush it only refills to the 2 inches below the overflow but after a few hours its overflowing again. I dont hear any water refilling or anything so not sure what's going on. Do I just need to replace the refill mechanism

Yes! The fill valve is likely leaking water into the tank.
They're are relatively cheap and easy to replace. But don't adjust it to 2" below the overflow. Instructions will likely suggest a 1/2" or so.

Thanks Diehard! I've replaced them before at my old house but wasn't sure if it was the same root cause. Thanks again for confirming

Use a fluidmaster ball cock. They are easy to install and you can replace the gasket easily if needed.

Everybody forgets (Which is GREAT for Fluidmaster sales)
There's no need to replace the assembly - Cure the leaky valve with a replacement rubber seal - no need to drain toilet tank, etc., 5-minute fix.
Link, complete with instructions below, most any local hardware store will have it.

If he doesn't already have a fluidmaster in the tank then some of these cheap fill valves are not that easy
to repair. Cheaper to replace. That's what I do and then no call backs that way.