I think in practice there is a lot of common sense and sportsmanship involved. People's good judgment help keep the sport fun and fair. Light air conditions where everyone is floating into one another at a mark, no one says protest, then goes to room to get exonerated. Touching a mark when no one sees? Even if you told me that God Himself was asleep and didn't see, I would take the penalty. Port boat crosses starboard boat and thinks, yikes, that was closer than I thought, they wait to see the flag (assuming its too hard to hear). Port boat crosses starboard boat and starboard boat obviously takes a huge evasive action, probably they do their turns without a flag -- indeed I would hope they would even if the other boat didn't use "Protest" when shouting.
While I agree with the self-regulating spirit, I think the "everyone takes a penalty every time regardless of protest" ignores the common sense complexity. (Not to disparage our friends on the links, but an everyday mark rounding with boats coming in on different tacks and angles requires a lot more thinking than the rules questions in an everyday round of golf, and you get a lot more time to think about it.)
That said, I still think balance of the equities is off with letting a foul slide if the wrong words were shouted in the heat of the moment. And it certainly undoes the spirit of self-regulating.
Ang, to get back to your proposal, there's another factor at the amateur level: timing. Skippers brains can get pretty busy in a rounding. I have certainly crewed on boats where things are shouted in the heat of the moment. Then on a bit of reflection (and sometimes a word from the crew), the skipper realizes that, say, coming in at that angle we probably weren't overlapped or were overlapped, etc. In the 1980s, a lot of #$%@# was said, then a minute or two later there was a flag... or not. Sometime cooler heads prevailed.
Rather than eliminate the flag requirement, I would be more forgiving on the form of the hail and the timing of the flag. Strategically waiting 2 minutes to protest rarely gives anyone an edge they didn't already have -- like super rare. But a more forgiving hail rule and a more forgiving flag timing rule would mean 1) you don't get away with a flagrant foul if someone's flag isn't ready (way more common), 2) you get a chance to take a breath and be sure before you make someone do turns or drag them into the room, 3) we don't have rules that have the contradiction of saying "we expect you to act reasonably" and "if you use the wrong word, it is like it never happened."
That said, I'm not sure if what you have heard over and over is that the timing requirement was too strict or that club-level racers didn't have a flag at all. The above would help meet people where they are on timing, but not on having a flag at all.
Full disclosure, I had a Kodak film canister on my El Toro, it was fine. ;)