They had also been held to a surprise goalless draw at home by Hibernian on Friday as they fell two points behind Rangers five games into the new Scottish Women's Premier League season.
KuPS, who lie second behind HJK Helsinki midway through their own domestic season, started the sharper side and Finland-born Nigerian striker Ariyo pounced after a slack pass in midfield to fire low past goalkeeper Kelsey Daugherty after 18 minutes for her sixth goal in five games.
Celtic's Hana Kerner fired wide just before the break, but Daugherty had to produce two fine saves before the Scottish champions got back into the game.
Goalkeeper Ellan Arpiainen saved superbly from Caitlin Hayes but soon after failed to hold the defender's cross and it fell kindly for Noonan to nod in from close range just after the hour mark.
Celtic were by now the side in the ascendency, but the Finns resisted doggedly until Kit Loferski held off a challenge and laid it into the path of summer signing Noonan to slot the Scots into the lead.
The Republic of Ireland forward then pounced on a wayward pass at the edge of the area to poke home her sixth goal of the campaign as KuPS suffered their first defeat - other than three at the hands of HJK - in 22 outings this season.
Now only Gintra or Anenii Noi stand between Celtic and a two-legged round-two tie to decide who reaches the 16-team group stage.