It was Nicolas Raskin who struck first against his Belgian compatriots. The midfielder timed his run to the back post perfectly to meet Tavernier's excellent delivery and net just inside the far post.
But Union showed plenty of threat and thought they had levelled, but VAR deemed Franjo Ivanovic had strayed marginally offside before finishing past Jack Butland.
Cyriel Dessers was effective again. From a corner just before half-time, he finished with real composure but was judged to be offside in the tightest of calls.
Moments later, Mohammed Fuseini got on the end of a cross for the visitors and somehow put it behind from a yard.
That was a let-off and more would follow as Union pushed for an equaliser.
Ivanovic missed from six yards. Fuissini thought he had scored but Tavernier blocked. Then Ivanovic was off target on the follow-up when it seemed a goal was odds-on.
But Rangers relieved all that pressure on the counter. Dessers bobbed and weaved down the left, and pulled back for Cerny, who netted his 13th of the season.
That seemed to knock the stuffing out of the visitors and Cerny came close again with a shot off a post.
However, in a thrilling finale, Kevin Mac Allister pulled one back with a header that looped over Jack Butland and was awarded after a lengthy video assistant referee (VAR) check.
Substitute Kevin Rodriguez then, off balance, scooped over with the goal gaping and Ross Sykes blazed over deep into stoppage time, but Rangers clung on for their coveted prize.