Both sides went into the game looking to bounce back from first defeats suffered under their new managers.
It was Hearts who impressed first, although Lawrence Shankland fired wastefully wide from in front of goal as the Scotland striker looked to end his recent barren run.
However, it was Craig Gordon who was first in action, the visiting goalkeeper pushing wide Clark's curling free-kick.
Just when the game looked to be turning Saints' way, the latest in a series of dangerous Blair Spittal corners to the front post flicked off Clark's head and goalkeeper Ross Sinclair was left flying, Superman-style punching thin air as the ball found the far corner.
Striker Adama Sidibeh squandered a couple of chances to level before the break, midfielder Jason Holt's curler struck the face of the bar shortly after, and Gordon had to push the former's head flick wide.
St Johnstone were level midway through the second half when referee Chris Graham was called to the monitor by VAR to review a challenge by Hearts midfielder Cammy Devlin on Kyle Cameron and Clark sent Gordon the wrong way from the resulting spot-kick.
Hearts burst back into life and Baningime weaved his way to the edge of the penalty box and set up forward Vargas to find the far corner with a low drive.