And City looked to have made it safe just before the hour as Kevin de Bruyne played a corner to Bernardo Silva on the edge of the box.
His shot looked harmless but Kompany stuck out a boot to wrong foot Ospina and send the fans behind that goal wild.
What a moment for the skipper who has endured such a nightmare few years due to injuries.
And just minutes later it was all over as Danilo slipped through David Silva and he turned Calum Chambers superbly before firing low past Ospina.
Pep Guardiola said: “We're so happy, and a big congratulations to all of Manchester City, and our fans. This trophy is for Manchester City and not for me.
"The first half we were not good, there were a lot of mistakes, in the second half we played with a lot more courage and personality - we were outstanding."
Replay of Sergio Aguero push on Shkodran Mustafi
Arsene Wenger said: "You have to accept being criticised when you lose games. But I believe we have to recover quickly. We got to the final, we lost, so now we have to focus on the next Premier League game."
"When you lose 3-0 of course you have to look at defending better, but I still feel like for large periods we defended well, but we lost concentration sometimes and against good teams you pay for it.
"It's not the application that was missed it was the concentration at key periods that was missed. The second goal was a big goal in this game and for me, it was 100 per cent offside.
"We analyse all the games in a very mathematical way, we didn't give too many chances away, but we gave away easy chances, and didn't create enough for ourselves."
How to win £1MILLION with Sun Six
SUN SIX is the new predictor game where you can win £1million - just by correctly picking the results of six games.