The Three Lions boss watched as a mistake from Michael Keane gifted Valon Berisha a goal after just 34 seconds of the Group A clash at Southampton's St Mary's on Tuesday night.
However, Raheem Sterling, Harry Kane, a Mergim Vojvoda own goal and a Jadon Sancho double put England into a commanding 5-1 lead by half-time.
England then let Kosovo back into the contest after the break as Berisha and Vedat Muriqi made it 5-3, before Kane had a penalty saved and Sterling hit the post as the hosts missed chances to kill the game off.
Southgate was quick to praise the threat his side displayed going forwards, but was concerned at the goals they conceded.
"A bizarre game really," he told ITV. "There were obvious poor errors for goals but I'm really pleased with the composure we showed. We used the ball well.
"The errors, we have to be better - the players don't need me to tell them that. The outstanding parts of our play were obvious, the errors were obvious."
Southgate suggested that he will look more in-depth at whether the individual mistakes were down to poor concentration or if there is something more underlying.
The 49-year-old could not decide if it was the "intensity" of the game or "pressure" but insisted that his players must learn from the "basic errors".
He also added that it was a positive that his players managed to recover from their mistakes to hold on for a fairly comfortable win.
The victory leaves England top of Group A with four wins from four games, three points above the Czech Republic, having scored 19 goals.
Next up is a trip to Prague to face the Czechs on October 11, before meeting Bulgaria in Sofia three days later.