Pictures with an 8-Year Old: A Failure of Citizen Science


I’ll start this at the beginning.  Recently Friend of LWON, Chris Arnade, posted a picture of himself — which itself was not unusual because Chris is, among other things, a photographer and posts pictures of himself right along with pictures of other people. But the picture was unusual.  Chris is a very serious guy and always looks it.  This recent picture, though, was something else.

Chris by a friendChris by an 8-year oldHere are the pictures. The one on the left is the serious Chris, taken by another adult, the way I’m used to seeing him. But the picture on the right, the recent one?  “Chris!” I said. “Look how delighted you look!”

“The picture was taken by an 8-year old,” he said.

I had an epiphany.  I’d take a bunch of pictures of people, an 8-year old would take pictures of the same people.  The ones I took would look sternly grownup.  The ones the 8-year old took would look like they’d seen full-on blooming lilacs for the first time.

I couldn’t find an 8-year old, but Nora was nearby and she’s 10 — close enough.  Then oh joy! Cameron generously said she’d add data; she has a 7-year-old.  We conducted citizen science

We were careful about protocol.  We wouldn’t say, “Smile,” and we wouldn’t explain why we wanted the pictures, we’d just say, “It’s a scientific experiment and if we told you, we’d spoil it.”  And the kids would have to be careful to shoot the people straight-on, so they wouldn’t be looking down.

So. To see whether this data supports the epiphany, I would like to ask you, dear reader, to vote on which of these pictures was taken by a grownup and which by a kid. (Please, keep reading after the ads, please? When you get free survey software, you get ads.)

Yeah, me neither. I got 66% and I knew the answers. One problem is the number of uncontrolled variables. Maybe we should have taken the pictures on different days so people would respond to each of us more freshly; or maybe we should have controlled who shot first; or maybe we should have used the same official or non-official-looking camera; or maybe, as Nora’s father said, I was so beloved that people smiled at me the way they did at Nora; and I’m pretty sure Cameron is beloved too, in which case we’d have to also find an adult photographer who was unbeloved.

The other problem is statistics, which I don’t and never will understand except to say, if you flip a coin ten times, you could easily get heads two-thirds of the time; but flip it 100 times, and you’ll get heads half. Cameron and I and our kid-photographers got only eleven flips. People who understand these things say, “small number statistics,” and tell you to go find another place to play.  So here’s my point:  the only reason citizen science accomplishes wonders (those are serious scientific papers, people) is because the citizen scientists number in the zillions and the objects they’re sciencing number the same; and all those uncontrolled variables wash out.

This therefore leaves the epiphany in the hands of you, dear reader.  I need a zillion-minus-eleven more face-pairs; and a zillion-minus-four more photographers.  Who’s with me here?  Anybody?  We could even quantify the width of smiles and the crinkling of eyes. Nora’s father said if we get this right, we could have the database for computer-identification of faces and affect national security.  Anybody at all?  No?

Or, of course, the epiphany could just be wrong.   Nora took one of these pictures, I took the other, the cat didn’t care, he just hated both of us.



I could in no way have posted these results, such as they are, without considerable time and effort by Nora, by Cameron, by Cameron’s 7-year old, by Nora’s father, and by LWONian Rose Eveleth who figured out how to embed that survey.  I thank them all with all my heart.  My epiphany wishes that for their sakes, it had worked out better.  Two things did work though:  Nora’s father taught her to use his fancy camera and she loves it; and Cameron’s 7-year old learned how to use her camera phone and has therefore quit bugging her about it.


The Zooniverse, linked to above, has a new citizen science project in which people look at images of the Equadorian earthquake to help coordinate rescue work on the ground.  Here you go: you’d be doing good.


Chris Arnade’s photos used with his kind permission though I bet he didn’t ask the actual photographers.  The test photos of the face-pairs are used with the kind permission of the owners of the faces.

Share Button

17 thoughts on “Pictures with an 8-Year Old: A Failure of Citizen Science

  1. Well I got 72% and there were a few that I wasn’t sure of and ended up looking at other things in the photo, like exposure, depth of field & focus. Adults it would appear take a “better” photo, I guess because they have more experience. Great fun, thanks for the interactive post!

  2. If I was the adult photographer, I was shooting with a cell phone and Nora was using her dad’s fancy camera. So the differences you saw might have been due to my idiot-proof camera.

  3. 63% – I looked for the most genuine smile … one that lit up the eyes as well as the mouth. But on some of the people, it was hard to tell. And there’s that one guy who’s smiling less than the cat!

    So much fun. I love citizen science!

  4. I got 45%. I should have just closed my eyes and guessed. I started out trying to do it by the quality of the picture. And when that failed, I started looking at their expressions. That failed too.

  5. Erm yeah… Where I couldn’t make up my mind I judged the one with better composition to be the adult one… wrong every time!! Do I have odd standards of composition in my head? Do cameras already ‘do’ that for you now? Just point in the general direction, shoot and out comes a decent photograph?

  6. I think three of the four photographers were using a cell phone. Nora was the only one using a fancy digital camera. So yes, cameras “do” make up for all kinds of photographer-related idiocies. But not composition, I think. Except in this case, I fooled with the photos to get the faces of each pair roughly the same size, and that would have messed up the composition. I’ve taken this test a number of times (to make sure the software worked) (also I know but never remember who took what picture} and my best scores were when I concentrated on the width of the smile and how smiley the eyes looked.

  7. I got only 45%! This is fun. And all the more reason we all need to support Bethany Brookshire (@scicurious) and her project to make videos of science *experiments*! So much of what’s out there for kids is science demos. She’s going to explain how to get a real experiment going, with statistics and everything!

  8. “Or, of course, the epiphany could just be wrong.” As a scientist, this was my first thought after taking the quiz. Yes, your sample size might be too small to see the effect or there could be some flaw in the experimental design (e.g., not everyone used the same camera)… but it is always important to remember that your hypothesis could simply be wrong! Too often people set out with the mindset to *prove* their hypothesis, when the goal should always be to *test* it.

  9. I got 54% by picking the ones that appeared to have been taken from below (picture taker was shorter than the subject).

  10. Yeah, I tried to control for that point of view problem by asking the kid(s) to ask the people to sit down. And clearly they did, but clearly they didn’t always; and they were all doing me enough favors already that I didn’t want to ask for a re-take.

  11. Ha! I even got wrong one of the pairs that I took. We were wondering if there is a difference between women and men, or between people who have kids or like kids and people who don’t. And/or grumpy people and less-grumpy people.

  12. I got 90% – the one I missed was the guy who didn’t smile for either photo. I think there is a definite difference and I guess I might be imagining this but I definitely got a sense that the difference in each, while super subtle, was that the person being photographed wanted to project a certain image in your photos and was less guarded/more honest in the photos taken by the 10 year old. Except the non-smiling guy. I like this experiment! Super interesting and I think very valid results!

  13. That’s exactly what I thought too, Shannon, that the people would be more self-conscious or guarded with another adult but more open with the kids. For some reason, I think that’s delightful. And the non-smiling guy, I can see in one photo a difference around the eyes, they’re softer, just by a little.

Comments are closed.

Categorized in: Uncategorized