A GFCI outlet should trip if there is a voltage difference between the ground wire and neutral. Any flow there indicates a potential problem and should cause it to trip off. Without a ground connected, this effectively prevents the GFCI from working properly. The holes at the rear of the box are for ground screws. You should connect the ground to there. Also, it looks like the ground wire is "grounded" to the box by being folded back into the entry hole with contact at that point. This installation was not likely done by an electrician. Proper grounding is necessary for proper functioning of the GFCI outlet.

Dan