The home side thought they had taken the lead after just 40 seconds when Duku tucked the ball home after Tom Tonks had disguised a long throw before going short but the Lambs' joy was cut short once they saw the flag had been raised.
Tamworth were the more threatening side in the first half and after a corner was only partially dealt with, Beck-Ray Enoru hit a shot from distance which was saved by visiting keeper Killian Cahill, who celebrated his 22nd birthday with a clean sheet.
The best chance of the first half came in additional time when Stefan Mols rose highest and his header looked destined for the top corner but Cahill clawed the ball out of his goal and away from danger.
Until their goal, Orient had been unable to sustain a period of purposeful possession on their opponents' artificial 3G pitch and an over-hit cross from Aaron Connolly summed up their attacking efforts in the opening hour.
Idris El Mizouni did spurn a half-chance for Orient when he scuffed wide on the spin before Tamworth's Enoru completely mistimed his effort following a long Tonks throw in to the area - one of several during the evening.
With 11 minutes remaining, Orient made the breakthrough when substitute Sonny Perkins' deft cross was flicked into the far corner of his goal by Digie - who had until then been a rock at the back for the home side.
With the match deep into injury time, Tamworth keeper Jas Singh went up for a corner to make himself a nuisance but the delivery was dealt with resolutely by the visitors.
Orient could have wrapped up the tie with the final act but substitute Alfie Lloyd didn't get hold of his effort properly, finding the grateful arms of Singh.