Golesh was the primary playcaller for the Volunteers, who led the country in scoring this season with 47.3 points per game. He came to Tennessee with Josh Heupel in 2021 from UCF and also coached the Volunteers' tight ends.
The 38-year-old Golesh was named last month as one of five finalists for the Broyles Award, which goes annually to the top assistant coach in college football.
"We are thrilled to welcome Coach Golesh as the new leader of our program. He is a dynamic coach and recruiter who is relentless in pursuit of excellence, as evidenced by his outstanding track record of elevating numerous programs. He also happens to be one of the most creative and successful offensive minds in college football," Kelly said in a statement. "Alex has worked his way up through the college ranks, been a key part of successful program-building at numerous stops and has developed perhaps the most fearsome offensive attack in college football today."
A news conference to introduce Golesh is scheduled for Monday.
Tennessee has been one of the nation's most explosive offenses each of the past two seasons. The Volunteers, using an up-tempo system, scored 38 or more points in 10 of 12 games this season. They were third nationally in passing with 332.3 yards per game while also rushing for 205.7 yards per game.
A 19-year coaching veteran in the collegiate ranks, Golesh replaces Jeff Scott at South Florida. Scott was fired Nov. 6 after compiling a 4-26 record in two-plus seasons with the Bulls. South Florida finished its season last week with a 46-39 loss to rival UCF.
Golesh is a graduate of Ohio State and has spent time at Illinois, Iowa State and Toledo. Golesh was also a candidate for the Cincinnati head-coaching job this year.