The fact that Pochettino would appear to be Manchester United's first choice to replace the sacked Jose Mourinho is testament to how well Spurs are doing, but the Argentinian reckons his side do not get the positive coverage they deserve.
They were heavily criticised for not making any summer signings, despite going on to make their best ever start to a Premier League season, have had fun poked at them for not winning any trophies in recent years and have attracted negative headlines for work on their new stadium overrunning.
Pochettino, who earlier this season said he was enduring his "worst feeling" as Spurs boss, claims people are quick to jump on board when things go wrong.
"We are on a good run, and playing well, but the moment we lose games it's going to turn quick. I know that very well. This is normal," he told reporters.
"My experience in football, because I try to understand everything, in this football club if some bad results arrive it's going to turn again.
"All the positives today will be reversed and become negatives, 'We are still not solid in our idea, we believe, but not really believe. We are happy, but not so happy'.
"I think there's still a massive work to do. But when I say 'worst feeling' I don't speak about inside of the club. I talk about the perception outside."
Spurs are through to the Champions League knockout stages, are outside contenders for the Premier League title and reached the last four of the Carabao Cup with a 2-0 win at Arsenal in midweek.
That is against the backdrop of playing at Wembley, making no summer signings and having a long injury list.
Yet Pochettino insists the default perception of his club is a negative one.
"We are doing fantastic but if we don't sign in January people say we are going to struggle - it's all negative," he added.
"Rather to say Tottenham didn't sign anyone is because they believe that they are going to - or have the tools - perform as well and maybe challenge for the title.
"It's always negative. The doubt is always negative, never positive that's because of the history. Because of what happened in the past. We need to fight with this perception.
"All the decisions are always taken in the negative way, and in that it's so difficult to build something special and strong structure for the future to win, but we are fighting about that perception in the last four years and a half.
"It's about timing, to convince the people, to believe and Tottenham is the most important, any name is [no] bigger than the club, and the fans, and the soil of the club, I think we are changing that perception."