Ipswich started well at a packed Portman Road and made Southampton pay for an early mistake.
The home side took the lead inside the opening few minutes and it all came from Ipswich's press, which forced an error out of the Saints defence.
Hirst nicked the ball off Joshua Quarshie and raced in behind, cutting the ball across the box where Jack Clarke was waiting.
The ball then flicked up before hitting Harwood-Bellis, who unintentionally pushed the ball into the back of the net off his chest, past the despairing watch of keeper Gavin Bazunu.
After a slightly nervy first 20 minutes, Southampton started to take some control, knocking the ball around confidently and getting forward through the likes of Mateus Fernandes and Robinson.
They got right back into the game just before the half-hour mark.
Ryan Fraser delivered an outswinging cross from the right which was met by the onrushing Robinson, and the 18-year-old academy graduate headed the ball downwards and into the net.
Saints nearly shot themselves in the foot again moments later, when Harwood-Bellis decided to drive forward with the ball but got caught in possession.
Hirst then drove forward, beating challenges from Jack Stephens and Quarshie before shooting and forcing a good save out of Bazunu.
Ipswich started the second half with a big opportunity to retake the lead when Hirst placed a ball through for Szmodics to run onto beyond the Southampton backline.
The forward was forced wider into the left hand-side of the box but sent a low effort towards the goal that hit the inside of the post and bounced back into the grateful arms of Bazunu.
The final 20 minutes brought plenty of end-to-end entertainment as both sides chased a late winner.
Ipswich nearly nicked one in added time when substitute Conor Chaplin headed over the bar from close range.