Mine has/had the same problem when I first bought it. It's kinda like a "crunchy" sound isn't it? The deeper you press the pedal the more it makes the sound. It is in fact the master cylinder. This isn't the only car it happens to. I know 3 other people, personally, who have had the same thing happen. All 3 had the master cylinder replaced - problem gone.
Regardless, mine only did it towards the end of the day, after long periods of driving -- like running errands or something all day. I happened to be at Sears Town Auto getting a flat tire fixed and I saw that they changed brake fluid -- So i thought to myself..."for $30, I'll try it." Changed out the brake fluid and no more sound to this day -- that was over a year ago. Brakes works perfect.
Joe's knowledge is ridiculously superior to mine. So if he tells you something, I'd probably listen to him. That, and he's a super nice, helpful guy.
But I'm just saying from my experience, that's what I've come across. Something about the seals going bad inside the master cylinder causes that sound to reverberate into the cabin and sound like it's the arm that pushes on the pushrod inside the booster. I'm not a car rocket scientist, just a do-it-yourselfer kinda guy.
Check this thread out:
http://www.lincolnvscadillac.com/showthread.php?t=35522
Pay attention to post 22. I actually did a good job of explaining it.
Joe, what do you think? Possibly master cylinder issue, or still leaning towards something else?