Voth, 31, has not pitched for the Orioles since June 13 because of right elbow discomfort.
The reliever was 1-2 with a 4.94 ERA in 31 innings over 23 appearances before the injury.
Fellow right-hander Dean Kremer will move up in the rotation, leaving a vacancy for Thursday night's series finale.
No reason was provided for the removal of Flaherty, who allowed seven runs in three innings against the San Diego Padres in his last start Aug. 15.
Flaherty, 27, is 1-2 with a 7.07 ERA in three starts with Baltimore since being acquired in an Aug. 1 trade with the St. Louis Cardinals. He was 7-6 with a 4.43 ERA in 20 starts for the Cardinals this season before the trade.
Kremer, 27, is 11-5 with a 4.50 ERA in 25 starts this season. He is 2-3 with a 4.93 ERA in nine games (eight starts) in his career against Toronto.