Adam Lallana's last-minute strike gave Liverpool an astonishing first Barclays Premier League win of 2016 in a nine-goal thriller at Norwich.
Jurgen Klopp's men had trailed 3-1 with under 30 minutes to go, then led 4-3 before Sebastien Bassong's stoppage-time strike levelled matters.
But there was still time for substitute Lallana to mis-hit a shot into the ground and secure a 5-4 victory as Liverpool's own defensive failings were eclipsed by Norwich's at Carrow Road.
Klopp celebrated on the pitch with a shirtless Lallana, but there is much work to do if the Reds are to advance to the Capital One Cup final at the expense of Stoke, visitors to Anfield for the semi-final second leg on Tuesday night.
It had looked like new signing Steven Naismith would be an instant hero for the Canaries after Dieumerci Mbokani cancelled out Roberto Firmino's opener with a brilliant back-heel.
Naismith scored his first Everton goal against Liverpool and, on his debut following his move from Goodison Park, the Scot struck again to give the Canaries a 2-1 half-time lead.
Poor defending allowed Norwich to further extend their advantage when Alberto Moreno felled Naismith in the box and Wes Hoolahan converted the resulting penalty.
Jordan Henderson reduced the deficit from a gaping hole in the Norwich defence and another appeared allowing Firmino to equalise with 27 minutes remaining.
James Milner converted for 4-3 after capitalising on Russell Martin's dreadful pass back before the late drama.
The Canaries looked lively early on, with Naismith, Nathan Redmond and Hoolahan floating around in spaces between Liverpool's defence and midfield. But it was Liverpool who struck first.
Moreno and Milner combined well on the left and the latter fed Firmino.
The Brazilian was played onside by Robbie Brady and his scuffed shot went in via the boot of Declan Rudd and the post.
Brady recovered well to deny Milner inside the area moments later before Norwich exposed Liverpool's aerial vulnerability to draw level when the visitors twice failed to clear a corner.
Graham Dorrans headed back in and Mbokani did the rest, controlling the ball with his back to goal before his improvised finish left Sakho sprawling on the floor.
Liverpool's defence was again found wanting when Hoolahan beat Henderson and found Naismith, who finished brilliantly across goal for an interval lead.
Naismith next stole in on the wrong side of Moreno before the left-back lunged in from behind.
It might have been a foul, but Moreno made certain by recklessly bundling Naismith over from behind. Hoolahan converted the penalty.
Liverpool pulled one back within moments as Firmino flicked Nathaniel Clyne's cross on to an unmarked Henderson, who swept in.
The Reds equalised when Milner found Lallana, who centred for Firmino to control the ball once and prod it past Rudd.
It went from bad to worse for Norwich as Martin inexplicably passed back in the direction of Rudd without looking.
Milner intercepted and finished calmly to restore Liverpool's advantage only for Bassong to finish superbly from 22 yards for what he thought was the equaliser.
There was another twist to come, though, as Norwich failed to clear and Lallana's shot looped over Rudd.
TWEET OF THE MATCH
"FULL-TIME: Norwich 4-5 #LFC. No words." - Unsurprisingly, Liverpool's official Twitter feed (@LFC) were lost for words at the final whistle as the Reds came out victorious in a nine-goal thriller.
Declan Rudd: 5/10
Ivo Pinto: 5
Robbie Brady: 5
Russell Martin: 5
Sebastien Bassong: 5
Jonny Howson: 6
Nathan Redmond: 6
Graham Dorrans: 6
Dieumerci Mbokani: 7
Wes Hoolahan: 7
Steven Naismith: 7
Matt Jarvis (for Hoolahan, 70 minutes): 5
Martin Olsson (for Redmond, 70): 5
Cameron Jerome (for Naismith, 82): 5
Simon Mignolet: 5
Nathaniel Clyne: 6
Alberto Moreno: 5
Kolo Toure: 5
Mamadou Sakho: 5
Emre Can: 6
James Milner: 7
Roberto Firmino: 7
Jordan Henderson: 7
Jordan Ibe: 6
Adam Lallana (for Ibe, 59): 7
Christian Benteke (for Henderson, 76): 6
Steven Caulker (for Moreno, 90): 5
Naismith. It could not be a defender. There was an unfamiliar look about Naismith, who for so long wore blue and white with Kilmarnock, Rangers and Everton before his reported £8.5million move to Norwich. He made an instant impression in yellow and green and could not be faulted for the goals.
MOMENT OF THE MATCH
Mbokani's goal. The Congo striker improvised brilliantly to equalise. With his back to goal, he controlled a bouncing ball and scored with a deft back-heel which bamboozled Sakho and Mignolet.
VIEW FROM THE BENCH
Naismith and Pinto made their Norwich debuts as two of four changes, while Liverpool made nine changes following their FA Cup replay defeat of Exeter. Both sides are in desperate need of defensive reinforcements, but the substitute who made the most impact was Lallana, the match winner.
MOAN OF THE MATCH
Plenty of mistakes at the back on show in this game. Alan Hansen will be glad he does not have to analyse the defending having now retired from the Match of the Day sofa.
WHO'S UP NEXT
Norwich v Tottenham, Barclays Premier League, Tuesday, February 2
Liverpool v Stoke, Capital One Cup semi-final second leg, Tuesday, January 26
Lallana said it was "massive" result for his side but admits he feared it was not going to happen when Bassong levelled.
He told BT Sport 1: "A strange afternoon but we're just delighted to get the three points in the end. When they scored in the 92nd or 93rd minute, you think it's going to be one of them days.
"But Benteke nodded it down, next thing you know it's in the back of the net. It's three points which is massive for us after a few disappointing results."
Milner said: "(The game) was ridiculous at times. Going up and then it's disappointing coming in at half-time, we weren't at the races. But we showed character to come back especially after conceding the goal early in the second half.
"It's disappointing to concede from another set-piece. It would have been easy to roll over but we went straight back on the front foot. A great finish from the lads."
Klopp said: "Unbelievable game. It was not easy today. I think we had a lot of problems in the game. I don't know what it was but it wasn't enough at the start of the game against a team searching for confidence.
"But we played football, we made it 1-0, we should have made it 2-0. But set-play, set-play, set-play and they got more confidence.
"Second half we came back with brilliant goals and at the end when I saw five minutes (stoppage-time) I thought...I couldn't understand why.
"I don't understand this and I asked but it was not positive. But these two goals, we were a deserved winner. Four goals from set-plays, that must be a new record, I think you have to make a new list for British football. I think it is crazy what we are doing."
A disappointed Alex O'Neil, the Norwich manager, said: "I think we were our own downfall, that best describes the second half defensively. We score four goals and you don't manage to win the game, that's a big, big problem.
"We got ourselves in a 3-1 situation, defended reasonably well but there's nothing to say and nothing constitute the situation."