The Three Lions boss feels that his side should be capable of matching their performance at the World Cup in 2018 and reach the semi-finals and would consider anything less as a failure.
Asked if anything less than a last-four place would be deemed as underperformance, Southgate said: "Yeah, it probably will.
"I think we're realistic about that, we have to live with that expectation. We know the excitement around the team and it's great, we're now relevant."
Southgate continued: "Expectations were lower (in Russia). We've got some exciting players and I don't think these players are at their peak by any means. I think when I look at some of the players working with us it's so exciting for England in the future.
"I've got to try to manage the expectations for the players. I accept the situation as a manager, there's expectation, I have to deliver.
"We have to deliver as a group of staff, it's not about me in terms of if we can be successful. But, of course, it's about me if we fail. No problem, that's the gig.
"But for the players, I've still got to bear in mind they are a young group, they have still got a lot of progression to make, some are now learning how to win things, being involved in big matches. It's brilliant for their progress and opportunities for England, that's what we want.
"Are we ready to win? Well, we've been to two semi-finals so the next step is to try to go further."