Faces found at home

Ask people what they want from a photo library and it's rarely a better way to sort by date. It's "show me every photo of Grandad", or "the one of the dog on the beach". Answering that means looking at the photos, and that has usually meant sending them to somebody else's computer to be looked at.

Collectoh does the looking on your own server. It can find the faces in your photos and gather them for you to name, and it can tag each photo with what's in it. Neither sends a single photo, face or name anywhere else. Here's how it works and, just as important, what it doesn't do.

#Off until you switch it on

Both features are off by default, and they're switched on one photo library at a time (a Memories library, as the app calls it), because they're decisions about people's pictures. An admin turns them on in the library's settings: Recognise faces for one, Tag photos by what is in them for the other.

The first time either is needed, Collectoh fetches the small models it uses, about 40 MB for faces and about 150 MB for tags, and keeps them on the server from then on; a server with no internet connection can be given the files by hand. The face models are checked against a known checksum before they're used. Both features run on the server's own processor, one photo at a time, in the maintenance window at night, and they step aside whenever anyone is watching something. New photos are queued when a scan finishes, and a nightly pass picks up anything missed.

#How faces are found and gathered

Two models do the work. The first finds faces. It looks at each photo whole, which catches the faces that fill a portrait, and again in overlapping tiles, which catches a family lined up across a room. A face too small or too uncertain to be any use is left out.

The second describes each face as 128 numbers: a sort of fingerprint of how that face looks, useful only for comparing it with other faces. It doesn't guess anyone's age, mood or anything else about them. The only question Collectoh asks of those numbers is whether two faces look like the same person.

Faces that look alike are gathered into groups, and two faces from one photo never go into the same group. Any group seen more than once appears in the library's People section under Who's this?, as a tile of faces waiting for a name.

#You do the naming

Open a group and type a name. If they're already on your server, perhaps from another photo library or a film's cast (it's all one list of people), Collectoh offers them as you type; otherwise the name becomes someone new. Then three things happen. Every photo with a face from that group is credited to them. Other unnamed groups that clearly look like them take the name too. And from then on, each new photo whose face comes near is named as it arrives.

A person can be any number of groups. That matters more than it sounds: a child at two and the same child at ten can look like two people to a model, and both belong to them. A group only takes a name by itself when it's clearly nearer that person than anyone else, so a group that sits between two brothers stays unnamed rather than guessed, and it never takes the name of someone who's already in one of its photos.

When it gets something wrong, you put it right. On a photo's page, a face can be named, or marked as not the person it was put with, which takes it out of their group altogether. A named group can be un-named the same way, and a group of strangers, such as the people behind you at a wedding, can be put away.

Once named, people are credits like any other: each photo's page lists who's in it, the People section leads to each person's photos, and search and the person filter find them. They're shown only to people who can see the photo, so nobody learns about a picture through its faces. Naming and correcting is for the server's admins; everyone else simply sees the names.

#Photos tagged by what's in them

The second switch gives each photo keywords from a vocabulary of about a hundred scenes, occasions, animals and things, such as beach, sunset, dog and birthday. The most common appear as chips at the top of the library, each opening the photos that carry it. The model's keywords are marked as its own, but they're ordinary tags that the tag filter finds and a metadata refresh keeps. The moment you edit a photo's keywords they become yours, and the model leaves that photo alone for good.

#What it doesn't do

• It doesn't send photos, faces, names or keywords off your server. There's no cloud service behind any of it.
• It doesn't name anyone by itself. Every name starts with you.
• It doesn't look anyone up. A person known only from your photos is never asked about at a film catalogue or anywhere else.
• It doesn't change your photos. What it learns is kept by Collectoh, not written into your files.
• It doesn't look at videos. Faces are found in photos only.
• It isn't perfect. On a test household of 2,000 photos made from a public research collection, naming each person's largest group was enough to credit their photos correctly at least 97 times in 100. Those were news photographs of grown-ups, though. Your own, with children growing up, faces in profile and dim living rooms, are harder, and you'll be the best judge of how it does.

#Deleting it all

Switching face recognition off for a library deletes every face found there, along with the groups, the credits and the names that were only ever given to faces; the settings dialog warns you before you save. To clear everything at once, Settings → Faces has Delete all face data. It asks twice, then deletes every face, every group and every name given only to faces, and switches face recognition off in every library. Your photos themselves aren't touched. It can't be undone, which is rather the point.

There's more about photo libraries in the docs, and about who can see what in sharing.