Kelleher, who impressed when deputising for Alisson Becker for Liverpool this season, produced a strong save to deny Roland Sallai after the restart, and another stop followed from Schafer's shot from the resulting corner.
Multiple substitutions from both sides took the flow out of the game - including a debut for Lyon defender Jake O'Brien - and fellow replacement Liam Scales got a crucial touch on Szoboszlai's delivery to deny Hungary taking the lead with 15 minutes remaining.
Sallai's low effort was easily held by Kelleher and captain Seamus Coleman fired a shot wide as the clock ticked into injury-time.
However, there was late drama as substitute Parrott beat Callum Styles to a loose ball and sprinted clear on goal.
Despite having team-mate Sammie Szmodics to his left, he elected to shoot and the power on his effort beat Hungary keeper Denes Dibusz.
It may only have been a friendly for the Irish but O'Shea was delighted with the win as he celebrated with players and staff in the dugout.
O'Shea will take charge of his fourth match as interim manager away to Portugal on 11 June while Hungary take on Israel on Saturday before their Euro 2024 campaign against Switzerland, hosts Germany and Scotland in Group A.