Joe Gallen hailed Pompey’s strength of character after surviving a tough Memorial Stadium examination to claim victory.
Ronan Curtis’ goal four minutes from full-time secured a fourth-successive win at the start of their League One campaign.
Yet Kenny Jackett’s men had to dig deep for the majority of the match as they were forced to defend for long periods.
Gareth Evans put them ahead in the 32nd minute before Bristol Rovers deservedly levelled from the penalty spot through substitute Chris Lines.
Three minutes later Lines was sent off following a challenge on substitute Brett Pitman – then Curtis was the matchwinner in the 86th minute.
And Pompey remain in second spot behind Peterborough.
Assistant manager Gallen said: ‘Over the years it has been such a tough fixture here, so to come here and get any points is well played to all our team.
‘I think we had the better of the start of the game, the opening 10 to 15 minutes we were on top but even in the first half they came into it pretty strongly really.
‘Their midfield dominated with Kyle (Bennett) coming off the line and Liam Sercombe coming off the line, we were unable to stop them.
‘However, there weren’t loads of clear-cut chances – apart from Sercombe’s shot just before we scored.
‘In the second half it was wave after wave of attacks at times but we did come through it and showed a lot of guts, character and organisation.’