Craig Tanner had opened the scoring from a 16th-minute angled drive, cancelled out by Craig Slater's superb 25-yard free-kick on 31 minutes.
Adams, who side maintain a four-point lead, said: "We pressed the ball and looked like we wanted to win the match.
"It was a difficult match because they set their stall out early in the game and played with five at the back and four in midfield with one striker up.
"We found it difficult to break them down. As soon as we scored the first goal they went to four at the back, then changed their midfield.
"It was a game where we were well on top for that period of the game, we lose a goal and they get a bit of belief from that. To win a game in this division is very difficult but it was a good way to finish.
"The first goal was a well-worked goal with Craig Tanner getting on the end of a good move and then the second goal was from Jimmy Spencer doing really well to get the flick-on, and Ryan Donaldson running down the goalkeeper forcing him into the mistake.
"I think that being at home there is a number of teams that come and sit in, play deep and come for a point, and it looked for large spells today that was what Colchester were trying to do, and sometimes that happens."
The decisive goal came from Luke Prosser's back-pass but manager John McGreal defended his player.
He said: "Luke has come in and held his hands up. He has been fantastic for us, he leads by example and takes it on the chin."
Reflecting on the game as a whole, he said: "We got the goal we deserved and I wanted them to get to half-time at 1-1 because they were finding little pockets of space between our two centre-halves.
"We were playing top of the league and we needed to tighten things up.
"There is controversy about the winning goal. I am sure the throw-in should have been taken in their defensive third but it got taken on the halfway line.
"I know things got in a muddle and we had that bit of a break but that is something we should have sorted out.
"The games we have lost we haven't really deserved to lose. Hopefully it will change soon. You could see how disappointed the lads were at the end."