I believe Enid's reference about the "we" and "too many blind spots" was actually about how she and Carl had snuck in and out of the place without being detected. I guess she figured if she could do it, so could others.
Perhaps it's time to set up some perimeters within the fence itself, ring it with razor wire or place some claymores (land mines) or something that just makes a hell of a lot of noise penetrating.
Makes sense. Might not have access to claymores/razor wire. They could build more lookout towers, but I guess the question is do they have enough people to man them now?