I am one of the biggest Peyton Manning fans in the world. He's a great player and he is one of the most consistent leaders in the history of the game. The Colts typically have not been a great postseason team. For some reason, they dominate during the regular season and get into the playoffs and lose to a team that is more physical and seemingly (I know this is hard to imagine) more prepared.
I think back to 2005, the year before the Colts won their first Super Bowl. They were the #1 seed, first round bye, and if I remember, they had a similar start, like a 13-0 run. The Steelers marched into the then RCA dome, ran the ball, shoved their foot down their throat, harrassed Peyton Manning, and kicked their arse on the way to winning Super Bowl XL in Detroit.
I don't see the Patriots as a threat to the Colts this year. There won't be another New England, Indianapolis AFC title tilt. The Patriots just aren't that good. Unlike years past, their defense is not what it was years ago. They can't get pressure on the quarterback and their secondary is very shady. Tom Brady looks better, engineering the NFL's 2nd ranked offense. However, this isn't the team of year's past. There is no Willie McGinest, no Richard Seymour, no Asante Samuel. It's not the same group.
The San Diego Chargers, I can't believe I'm saying this, but the Cincinnati Bengals, and if the Steelers get in, which they should, with a chance to defend their title, they have the running game, defense, quarterback, and pedigree, to knock the Colts off again.
Make sure to keep your eyes on the AFC playoffs. Watch closely who will represent the AFC in the Super Bowl. It won't be the Colts -- not this year...