Bobb, thank you for detailed suggestion. It seems to me that for the most cases you and Josh mentioned, there is already solution: just mark it with “there is no face here” command. And this face (or animal, or toy) will be gone for good.
The challenge for most part is with legitimate faces of the people that you don’t want (at this point of time) to add to your database, but may want to do that later, but they keep showing up as suggestions here and there. This is is the part I am struggling to come up with reasonable UI approach.
By the way, you mentioned other programs are writing something to the file. Can you please be more specific, or even give an example?