Scott Sinclair crashed a shot off the post in the 23rd minute before home captain Lars Stindl, who had opened the scoring in their 2-0 win in Glasgow, volleyed in after 32 minutes to give the Bundesliga side the interval lead.
However, the Hoops did not let their heads drop and in the 76th minute Monchengladbach defender Julian Korb was shown a straight red card by Portuguese referee Manuel De Sousa for pulling Moussa Dembele down in the box and the French striker slotted in the penalty.
With two fixtures remaining against Barcelona and Manchester City, the Parkhead side have two points from four games and sit at the bottom of the table.
However, they could still qualify for the knockout stages of the tournament permutations permitting or, more likely but not easily, get into the Europa League.
"I thought we should have won the game," Rodgers said. "I thought we were the best team. I am really proud of the team tonight.
"I said to them that in order for us to improve we need to play with personality and the personality we showed in the game was outstanding.
"We were very brave on the ball, opened up the pitch to play, secured and built the game, created chances, with speed on the counter attack and we had chances to win.
"We hit the crossbar, we should have scored in the end. What was important was to get a result."