Forest striker Lewis Grabban came off the bench to grab an 82nd-minute winner and set up a fourth-round clash against holder Leicester City.
Sunday's result was just the second time in 26 years that Arsenal have been knocked out in the third round -- Forest previously beat Arsenal at this stage of the competition in 2018 in what was former manager Arsene Wenger's final FA Cup game.
Arsenal, who have won the FA Cup 14 times, most recently in 2020, wore an all-white kit in their clash Sunday as part of a "No More Red" campaign to spread awareness of the high levels of knife crime among young people in London.
The visitors failed to dominate in a scrappy first half, and Nuno Tavares, who appeared uninjured, was visibly frustrated to be substituted on 35 minutes following a sloppy performance.
Forest had a chance to take the lead in the second half when winger Brennan Johnson led a counter attack and set up forward Philip Zinckernagel, who forced opposing goalkeeper Bernd Leno into a save.
Mikel Arteta's side failed to record a shot on target while Forest pressed on, and Grabban came on as a substitute on 68 minutes and soon made an impact, racing onto the end of a cross from midfielder Ryan Yates and slotting home to earn a 1-0 victory.