First, the RC has to score the boat in it's finish position/time.  Since the flag is part of the mark,  the RC has the option to protest the boat for breaking RRS 31 while racing, but it's not mandatory.   Sailing is a self policing sport and boats should really protest other boats for RRS violations.
 

Peter beat me to it by a minute.

The RC may protest, but is not obligated to.  If you are absolutely sure the boat touched the mark, then a friendly chat with the competitor after racing, but well before the protest time limit, will often result in an RET.  This would also satisfy the notice requirements of RRS 61.1(b).

Even after the protest time limit, suggesting to the competitors that you saw them touch the mark (which they may not have noticed) and that the corinthian thing to do would be to retire would not be inappropriate.

They did finish according to the definition, but also (given your description) broke rule 31. But absent a protest they can't be penalized. 

I agree with Peter. Sailing is a self policing sport. The other sailors should be protesting.
If, however, there are no other sailors in the vicinity to be able to witness the incident, I would follow with the chat as suggested by Matt, giving the opportunity to the sailor to retire. If they choose not to, then think about protesting. I think it is courtesy to give the sailor the choice to retire in 99% of times. 
There are always other factors that may play into the circumstances to take into account also.

I recall seeing SIs which said you could touch the flag.

Charles, in order to do that, the SI would need to change the definition of mark.  RRS 86.1(a) prohibits that.  It would be an invalid instruction.

Yes, but I think rule 31 can be changed.  

There's one point worth mentioning.  If the boat had cleared the line and the sail hit the mark after the boat (and of course the sail) had cleared it, then there's been no breach and so, of course, no penalty.  That can happen as the boat finishes, clears the line and then sails for home.

Charles - Rule 31 can be changed, but it relies on the definition of mark.  You can't carve off a physical piece of the mark without changing the definition. The real lesson here is - don't use a pin with a flag - use an inflatable mark, preferably a sphere or cylinder (appropriately counter-weighted so it stands up).

UK Team Racing generally has a SI which permits touching the flag, but not always it is on the tick list of things to be checked in SI's like are you retired for capsizing and how far do you have to capsize. It gets around the objections by modifying 44.1 to say there is no penalty for touching the flag, providing it stays on the mark.

Who uses marks with flags these days?  :)

Matt, would such an SI really need to change the definition of "mark"? The definition starts with "an object." Could the sailing instruction just say that the object/mark is the buoy & spar but not the flag?

Clubs in my area frequently use pins with flags. It's generally understood that the flag is part of the mark (no different from a M flag temporary mark) and it's up to the boats to avoid touching the flag. As long as the flag is appropriately sized it's not much more difficult than avoiding any other part of the mark.

I think rule 31 can be changed without changing the definition mark.  Here's what the SI could say: 

"Rule 31 is changed as follows:  While racing, a boat shall not touch a starting mark before starting, a mark that begins, bounds or ends the leg of the course on which she is sailing, or a finishing mark after finishing, except that, if the mark has a flag, she may touch the flag but not its staff."  

You can. But why?
Just do not touch the flag.

Far from me to pick an argument with the Chair of the US Sailing Rules Committee, but . . .

Mark  - An object the sailing instructions require a boat to leave on a specified side, and a race committee vessel surrounded by navigable water from which the starting or finishing line extends. An anchor line or an object attached accidentally to a mark is not part of it.

The flag is not an anchor line or an object attached accidentally to the mark, ergo it is part of the mark.  I don't see how you can separate it from the staff or the buoy without changing the definition.

I do agree with the, "There shall be no penalty for touching the flag" method of getting around this, though. 

A flag/staff on a buoy makes a poor sight line since it's never vertical.  At what point on the pin are you calling the line?  This is why spheres (balls, tomatoes) make the best start finish pins - if you're using a buoy at all.  An anchored boat with a flag / staff / extra set(s) of eyes is the best. 

Gareloch Class and Freundeskreis Klassische Yachten team racing in Dragons on the Starnbergersee

Gareloch Class and Freundeskreis Klassische Yachten team racing in Dragons on the Starnbergersee

Sorry for the double post.  A still for if the video does not work.

Thank you for the replies.  As always, the discussion on this forum has been interestng and informative.

As Chair of a local regatta in which I also raced...   I touched the finish mark, came onto the course did a turn then refinished.   The RC came to me after the race and informed me that I had only done a 270 and not the required 360...  so I promptly retired after finishing.

My point is Peter is right...  but, you mush also inform a boat if you intend to protest.  If you inform a competitor of their error, quite often they will admit to the mistake and learn a lesson and opt for a RAF.

If you sailed back onto the course (side of the line) and then did a full one turn penalty before finishing, would you have satisfied the requirement to do your turn promptly? 
Similarly in a downwind finish, if you jibe, sail back to the pin, tack around it, and finish, have you done a proper one turn penalty?

I don't think it's required to perform the penalty turn on the course side, is it? Only to get well clear, do the turn and to return entirely to the course side before re-finishing. 

Depending on where you start I think you can complete a one-turn penalty with as little as about 210 degrees of course change. The penalty is complete as soon as one tack and one jibe (in the same direction but not necessarily in that order) have been completed, yes? 

The penalty is actually “one turn”. Turns must include one tack and one gybe. Doing “a gybe, sailing back to the pin, tackIng around it, and then finishing” is two separate manoeuvres and does not fulfil “one turn”.

Tracy, as long as it's prompt, it's possible in a downwind finish to gype at the pin, immediately tack and then bear away and finish. Not as applicable in fleet racing but "spin & win" is a common match racing move (where the penalty turn doesn't have to be taken immediately, just prior to finishing). 

And in team racing, with a windward finish, it is often a problem when the boat gybes around the pin but then is in the process of completing their tack when the bow crosses the finishing line - penalty not completed before finishing so no finishing position at that point.
Also, in Team racing we use flags because in shifting winds we can set a number of marks and just move the flag from mark to mark to keep a true beat, similarly with the leeward mark to preserve the run, the reaches tend to be less critical.