Your Personal Black Box


It's a cliché to say that electronics are changing our lives, but some of these changes will be especially profound. We're already used to using smartphones and other gadgets to record the special occasions in our lives (and the funny things our cats do). But with sensor and storage costs plummeting, we're quickly moving towards devices that will quietly record our lives 24-7, just like the black boxes on aircraft and other vehicles. The privacy and security concerns are significant, especially if someone else is doing the recording (or if you're recording someone else). But if you're in control of the data, you'll be able to revisit the past in ways that have never been possible before.

Salesman
Sadly, most black boxes aren't actually black.  Via Aviation News

A recent New York Times article tells the story of two cyclists who were able to investigate their own bike accidents, even when one of them had blacked out from his injuries and couldn't remember anything about it. They were able to do this because they were using the latest generation of cycling computers, which continuously record speed, cadence (pedal RPM), GPS position, and even heart rate to allow you to analyze your ride afterward. This is great data to have when training, but it's even more useful when the unthinkable happens. One of the cyclists, Ryan Sabga, was hit by a car while riding in Denver, which the driver denied. Ryan was able to reconstruct the accident from the data in his Garmin bicycle computer, which led to the driver's insurance company paying his claim. He says on his blog that he won't ride without a "black box" again, and recommends that all cyclists do the same.

This trend is increasing. We've seen similar devices that continuously record OBD-II or video data from your car, so that if you have an accident, the data will help in court or with insurance claims (note that this goes both ways!). In fact, it's possible that this functionality is already built into your vehicle's computer system. Digital audio and video recorders increasingly have "continuous record" features, which allow you to leave them running in case something spectacular happens. The Livescribe pen continuously records both audio and your handwritten notes for later downloading to a larger computer. And as always, "there's an app for that," with smartphone applications for all kinds of black-box purposes, such as monitoring your sleep patterns.

SparkFun's family of dataloggers

Anyone familiar with SparkFun knows that we love dataloggers. The uLog, OpenLog, and Logomatic are ideal for pairing with various sensors, and the Package Tracker (currently undergoing revision) is effectively a personal black box in itself; recording GPS and environmental data to a SD card for later analysis. Enabling continuous recording on these devices (deleting the oldest data when the memory is full) is only a matter of programming. If accident data is what you're interested in, you could even program the logger to stop recording shortly after a major acceleration shock etc. to prevent losing the "interesting" data later. This is technology you can use now, not the mythical "five to ten years" from now.

But the next generation of devices will be very interesting. Imagine wearing a device that will let you treat your life like a TiVo™, allowing you to "rewind reality" whenever you miss something or want to take a second look. We've been steadily moving towards this goal. In 1945, Vannevar Bush imagined a machine called the "Memex", which would record one's lifetime of media and communications, and "which is mechanized so that it may be consulted with exceeding speed and flexibility." Computer science pioneers Douglas Englebart and Ted Nelson expanded on this theme, eventually leading to the hyperlinked structure of the world wide web. In the early 2000s, then-MIT grad student Brian Clarkson attempted to continuously videotape his entire life. His panoramic audio and video recording backpack captured such moments as the first time he met his girlfriend (who apparently wasn't scared off by all the hardware Brian was wearing). The 100 day experiment resulted in 500GB of data... a lot back then, but much more reasonable today. His backpack, which contained five cameras for 360º recording was a bit cumbersome, but it won't be long before Moore's law and the economies of mass production make this device pocket-sized and entirely practical. Clarkson envisioned a device he called a "familiar," which would sit perched on your shoulder, watching and listening to everything you do, building a multimedia diary for later "reality mining."

The hardware to do this is pretty much ready today. The real problem won't be generating and storing these vast amounts of data, but providing useful access to it. An active research area is automated indexing, allowing you efficiently search for the needle of information you want without having to manually tag everything in your digital haystack (this is bad enough with an MP3 collection, much less your entire life). The ultimate familiar would use AI to not only record, but analyze what it sees, letting you ask it simple questions about the past, rather than having to manually rewind to an event to review it.

The ability to access any moment of your past will have a profound impact on our lives. Will this be useful, or dangerous? Will it make us smarter, or dumber? How will you apply this technology? For me, it would be nice to not have to write anything down (just go back to that meeting or conversation), and be able to retrace my steps to see where I left my keys. Better yet, I'll just ask my familiar where I left them.

Get ready to rewind!


Comments 14 comments

  • I have been considering buying one of those Veho muvi cameras and mounting it on my bike in case something awesome or tragic happens. And an airhorn; I almost got hit by an SUV when I had the green light (fortunately, another driver saw me and honked (at the SUV of course ;]). You can NOT turn left on red, woman!

  • This already exists… its not for our personal lives but for our Collective life… it records humanity… its indexed and you can access any part of it by simple search…(google mostly, according to studies)

    we call it the internet…

    It is a black box if you think about it, just at a HUGE scale.

    • I dare someone to try and reverse engineer the Internet.

      C'mon hackers, crack this baby! I want my data back!

    • first thing i’m thinking of is the wayback machine… Looking at what the original google looked like is strangely satisfying.

  • Props for the Vannevar mention!

  • On the more mundane side of things, speaking of OBD dataloggers and such… I built one a couple of years ago, real time display of everything engine related, GPS, camera, wifi, etc.etc. Lots of fun stuff. Had it mounted in my truck, took a trip from the upper midwest out to the west, Yosemite Natl Park, etc. On the way back, 6 miles from my house (after a 5,000 mile long trip!), the water pump seized up. It kinda had me wondering if I’d overheated the engine beforehand and hadn’t noticed. Pulled up the datalog in a simple spreadsheet, saw a constant 198F (+/- 2F) up until the exact point of me shutting the engine off after loosing the serpentine belt. Took a big load off my mind.

  • I had a cyclometer that was so smart that like a self-ejecting tomato, it punched out and landed intact on the pavement I collided with a metal gate (gimme a break; it was really dark at the time, and now I have a much better headlight).

    But on the subject of black boxes, how about one for your house? Using the Sparkfun weather meter and pressure sensor I built a black box for my house that can record wind speeds up to 1500 MPH (in theory, at least). So when my house is annihilated by a microburst, all they have to do is find the SD memory card in the wreckage and they can see the wind speed, temperature, rain, barometric pressure, humidity, light level, and power consumption at the time of my demise.

    • Rather than “finding it in the wreckage”, why not have it upload the data to “the cloud”? This way you can also retrieve it from anywhere in the world (technically.)

  • I’m sorry, but recording every inane moment of your life so that you can replay it later sounds like some sad excuse to try and justify your existence. It’s like people photographing themselves at clubs like they need to show others that they had a good time.

    I see all this as a degeneracy of our collective society. We should be inventing and experiencing, not reminiscing about the past.

    • To me the value in recording every moment is (or will be) that you’ll automatically capture the things that you would want to revisit, especially those things you’re not expecting, or didn’t realize at the time would turn out to be important. Recording everything in order to save those few moments is wasteful, but storage is becoming cheap enough to seriously consider it.

    • Vain people will always be vain. As you describe it they’re abusing a rational technology (the camera) to fulfill their need for extrinsic validation.

      I agree that the need to collect, archive, and store all the events and information that we experience is a degeneracy. People need to be not afraid to live in the moment and forget it ever happened later, even if it is a really weird thing your cat does. That said, I also think that automatically recording everything with a “Press to keep the last 5 minutes” button might not be all bad. How many times have you witnessed an historic or personally singular event that you just wished you could share?

      The other side of the coin exists too. If only you and I record our entire lives then the future will have a quaint window into an era. If everyone or even just 100 people do the same you end up with a pile of trash. So many moments in life are wholly uneventful it would be tragic to keep them in hopes that someday someone might relive your mediocrity, even if you are POTUS. You end up needing to destroy the data.

      I think that, personally, I’d choose to live life without a “familiar”. But then…I also light my room with candles and prefer a house without central heating.

      • I agree that the need to collect, archive, and store all the events and information that we experience is a degeneracy. People need to be not afraid to live in the moment and forget it ever happened later, even if it is a really weird thing your cat does.

        I can relate to this view, but I kind of wonder if we’re that far from a place where what seems like obsessive or shallow behavior now will just become part of the furniture of everyday mental life.

        You and I (plus most of the people reading this) conduct a good part of our lives in technological channels that literally didn’t exist a generation ago. A lot of them already involve radical forms of memory augmentation.

        For example: My code and writing lives in version control, I have persistent undo in my text editor, and my command line keeps effectively infinite history. IRC, IM, SMS, and Twitter are all trivial to log indefinitely. I probably haven’t deleted an e-mail from a real human being in better than a decade.

        None of this stuff represents obsessive effort on my part. It’s just in the tooling, it’s easy enough I don’t really have to think about it, and it turns out to have benefits. With access to a computer, the boundaries of my recall are shifted outside of my own skull in some fairly surprising ways. Google, Wikipedia, et al. are becoming something very like a collective memory for our entire culture.

        To borrow a phrase from Alfred Korzybski by way of Heinlein, humans are time-binding animals. Memory distinguishes us from other organisms as much as do upright locomotion and opposable thumbs. We keep coming up with ways to augment these abilities, just like we keep coming up with mechanical multipliers for our other strengths. Cheap storage, highly portable computing, and digital cameras are incredibly disruptive technologies, and they’re going to produce a lot of grotesque excess, but not all that long ago (in the grand scheme of things) you could have said the same about written language.


This Week

Heartbleed

This Month

Happy Arduino Day!

SXSW Create 2014