Update-
I replaced the diaphragm on one of the toilets. The water doesn't continuously run now, but now it doesn't get a full flush. I got a 3.5 replacement kit, could it be it's the wrong one and that's why the flush is weak?
I'm not going to try the other one, instead wait for a...