David McGoldrick scored for the visitors inside four minutes - but the Pilgrims levelled six minutes after the break through ex-Magpie Sam Osborne.
Mitch Roberts was red-carded for a last man challenge midway through the second half - but sub Pemi Aderoju scored a brilliant winner after a flowing United move five minutes from time.
United: Cameron Gregory, Cameron Green, Mitch Roberts, Michael Bostwick, Jacob Hazel (Pemi Aderoju 60), Adam Marriott (Tony Weston 60), Jai Rowe, Jordan Richards, Sam Osborne (Dan Mooney 60), Frankie Maguire (Keaton Ward 77), Dylan Hill (Tom Leak 60). Subs (not used): Martin Woods, Zak Mills.
Referee: Jamie O'Connor.
Attendance: 1,086 (284 visitors).
PIC: Chris Bray.