Madrid took the lead through Vinicius Junior, who raced in behind the opposition backline on 25 minutes and converted a one-on-one, before Barca striker Luuk de Jong levelled the score just before half-time. Karim Benzema put Madrid back in front midway through the second half, but Barca equalised again when forward Ansu Fati headed home on 84 minutes, forcing extra time.
Valverde came off the bench and scored the winner in the 98th minute, capping a stunning team move as Carlo Ancelotti's side outnumbered Barca in a counterattack and supplied Valverde with a comfortable finish.
"This Barca can compete against anyone. We didn't get the result but it was step forward despite the loss," Barca boss Xavi Hernandez said in a postmatch news conference.
"I'm sad and angry because it's a Clasico defeat and we miss out on a trophy. But we dominated Madrid for large spells. We lack some experience, patience, responsibility. We made mistakes. We could have stopped their counters and been more responsible with the ball. We're close. We pushed Madrid back. It's a shame."
Wednesday's result was Madrid's 100th win in El Clasico since the two sides first met in 1902. Barca have 96 wins in the fixture.
"I don't know if it's the best [Clasico I've played in], but it was one with a lot of anxiety and tension," Benzema told reporters.
"We played until the end, an important game, a difficult game, Barcelona are always a strong opponent."
Madrid struck the first blow as Vinicius opened the scoring, collecting a through ball from Benzema before striking past goalkeeper Marc-Andre ter Stegen.
Barca pulled level just before half-time through De Jong, who was in the right place at the right time when Madrid defender Eder Militao attempted to clear a low cross from Dembele but was blocked by De Jong and into the goal.
Benzema fired Madrid back into the lead on 72 minutes, first forcing a save from ter Stegen before prodding home from a cutback from teammate Dani Carvajal.
But Xavi's side fought back, and Fati, who was playing for the first time since November due to injury, found space in Madrid's 6-yard box and provided a glancing header to send the two sides to extra time.
Ancelotti's side held Barca at bay in the extra period before going in front for a third time following a breakaway that culminated in four Madrid players queuing up to supply finish as the ball fell to Valverde to slot home.
Winger Rodrygo had a golden opportunity to end hopes of a third Barca comeback in the final moments of the match when Benzema set him in for a one-on-one, but he fired wide.
Ancelotti said he disagreed with Xavi's assessment that Barca controlled the game.
"I don't agree [with Xavi]. The game was even, they had more possession, and we used the counter," Ancelotti told reporters.
"In the first half we deserved to win, we conceded with some bad luck. The second half was more even, but I don't think Madrid dominated Barca or that Barca dominated Madrid."