In what was a double header with both the men's and women's teams competing to raise money for the charity Mind, Arsenal's women's side beat their London rivals 2-1 in the earlier game.
Havertz opened the scoring in the first half before Granit Xhaka pulled one back for Arsenal after the break. However, the lead only lasted four minutes before Abraham got Chelsea's second.
There was an injury worry for Arsenal on 40 minutes when Thomas Partey was forced off. Arsenal coach Mikel Arteta said after the game that Partey would have a scan on Monday but "at the moment it is not looking good."
While the Partey situation is concerning for Arsenal, the match also saw the introduction of new £50 million signing Ben White.
Xhaka scored Arsenal's equaliser on 69 minutes when he headed home from a Nicolas Pepe cross.
The Switzerland international had been linked with a move to Roma but Arteta said after the match that he would be staying with Arsenal this season.
"Granit is going to stay with us," Arteta said. "He is a key member of our squad."
Chelsea weren't down for long, though, and Abraham scored a powerful right-footed shot into the bottom corner to give them the lead on 73 minutes.
In the early kick-off between the women's sides, Arsenal beat Chelsea 2-1 with goals from Katie McCabe and Freya Jupp.
Reanna Blade had levelled things for Chelsea on 80 minutes but 17-year-old Jupp -- who was making her senior debut -- was the star as she scored the winner for Arsenal on 83 minutes.
McCabe had given Arsenal the lead on 27 minutes but it was the younger players who impressed.
There were injury worries for both sides with Jordan Nobbs and Pernille Harder going off with knocks.