Mikel Arteta's struggling Arsenal side drew level in the 31st minute with a fine diving header from Alexandre Lacazette after an excellent cross from Brazilian Gabriel Martinelli but City took charge after the break.
A mistake by Arsenal Icelandic goalkeeper Runar Alex Runarsson gifted the visitors a 2-1 lead in the 54th as he let a relatively harmless Riyad Mahrez free-kick through his arms.
Aymeric Laporte then headed in a Foden cross to wrap up the win for Pep Guardiola's side as they added to the troubles of the Gunners who are languishing in 15th spot in the Premier League, only four points above the relegation zone.
Arteta had declared before the game that he wanted to see "fighters" and not "victims" and he said that despite the result he saw the fighting spirit he needs to turn the season around.
Martinelli sets up Lacazette for Arsenal equalizer
Carabao Cup: Alexandre Lacazette (31') Arsenal 1-1 Manchester City. Watch Carabao Cup on ESPN+.
"I will focus on the fighters we have in the squad and the spirit we showed tonight against a really difficult opponent. I see a lot of fighters," he said.
Arteta was assistant at Manchester City before he joined Arsenal a year ago and Guardiola defended him from his critics.
"What can I say -- in our position as a manager we'll be judged by the results. I can only say I was with him a long period, the most successful period our club had.
"Sometimes in our job we need time. It's time to be patient. He's an outstanding manager, incredible work ethic and puts the club in front of any decisions he has to make," Guardiola said of his fellow Spaniard.
City joined Brentford in the semi-finals after the second-tier (Championship) side enjoyed a 1-0 win over Premier League Newcastle United earlier on Tuesday.