How would you react if I was recording you right now?

Exactly, that’s why I wasn’t going to tell you

For a while I’ve been obsessed with human augmentation using technology. When I say "human augmentation," I imagine you're thinking about wearables that monitor health or exoskeletons that give us superhuman strength, but no, I'm thinking about productivity tools.

Productivity tools may seem like the least interesting on this list but I promise you, they're not, and they're going to change all of our lives.

Shout out to my teammates Parker McKee, Dave Husak, and Tony Kulesa for the feedback and ideas.

---

I try out every new tool I come across and because of all the amazing new productivity tools my work style has completely changed. I voice-to-text all my emails using wispr flow and have chatGPT deep research read out loud to me using speechify, much to the chagrin of my colleagues.

Hey. I don't do it at my desk. Mostly.

I believe there is going to be a massive shift in the way people exist, interact, and work, and it will come in the form of ambient recording technology.

To give an example.

The moment I tried Granola, I knew it was going to be big. Everybody wants a note-taker, but nobody wants that note-taker bot in the room with them. Nobody wants that blaring record button making them uncomfortable and unable to talk freely. Nobody wants to ask permission. We just want the memory augmentation of someone who is taking notes for us. That is exactly what granola did.

Companies like Rewind, Granola, and Wispr Flow are pushing the boundaries in human augmentation via recording. They are proving there is significant productivity to be driven tapping audio, augmenting memory, and intelligence. But they're limited to the bounds of a computer.

I want this technology to be a part of my everyday life, not just the hours I'm at my computer.

There's a startup called Omi which is a personal recording device that records all of your interactions throughout the day and takes notes.

I have the Meta Ray-Bands, and I use them to talk on the phone. They don't do much else. I suppose it's because big companies like Meta are too afraid to push the boundaries in the ways that are necessary to drive this technology forward. They won’t lead this revolution but they could be a great platform for its deployment (if they would just put an app store onto their platform). Come on Meta, don't be a jerk.

No one in the ambient recording space has figured it out, and I think that's because success requires significantly more information capture than we've seen so far.

The Big Picture

Wearable productivity tools are the wedge through which the human augmentation market will emerge, but it will be so much bigger. A platform comparable to mobile or the internet with its own set of applications, experiences, and monetization pathways.

This would be technology in the least obtrusive form we've ever seen. Even more, the weaving of information into the world around you in real time unlocks applications that can't exist today.

It's the world we've dreamed up in every sci-fi book and movie.

But, this world can't exist unless we design a framework for how these devices can capture the world around us. This requires us to rethink recording laws with a new level of granularity.

What’s stored?

The critical argument I’d like to make is: a platform that is live processing and deleting is different from a platform that stores everything it captures. Whether your modality is audio or video, you can provide significant value by processing without storing, or storing only consented information (for example storing only your own speech and related material, rather than other participants in the conversation).

Storing would entail that all of your ambiently recorded audio & video was saved, so that it could be pulled up and referenced back to.

Live Processing would entail that information is constantly extracted from the audio & video stream, in real time and then the content is deleted or filtered to only save key components.

I think companies that aim to process rather than store should only require single party consent in all states and should be considered different from traditional recording devices.

Think about self-driving cars. When driving a Tesla, you’d see a stream of people and cars that are being captured by Tesla's recording system. This is a perfect example of ambient recording technology– streaming the surrounding world and augmenting our ability to monitor surrounding vehicles. What I'm proposing is that we extrapolate this paradigm to other devices.

There is still a place for regulation, but that regulation should focus on different parts of the product.

Rather than limiting how and where people can record, regulations should concern–

1. What data is stored vs. processed in real-time and then deleted

2. Who can access the recordings and consent management (the user, not the company)

3. How it can be used (Inadmissible in court).

Laws.

When I was using these platforms, I was of course curious about the legal ramifications. So I spent some time digging into U.S. recording privacy laws.

At the federal level, there is a law called the Wiretap Act (18 U.S.C. § 2511) which states that recording a conversation is legal if at least one party consents. As a participant of a conversation, you can record without notifying others; however, recording with criminal intent or eavesdropping is illegal. Most states have one-party consent laws which align with the federal law. A small group of states (about 10) require all-party consent to record conversations (California, Florida, Pennsylvania, Maryland, Massachusetts, Illinois, and Washington, Connecticut, Oregon, Montana, and Hawaii).

Gauging reactions.

When discussing this technology I tend to see a generational divide in reaction.

Older generations tend to get angry about the prospect of more recording.

Younger generations are generally open to it. Excited about the benefits and less concerned about the privacy implications.

Given the generational increase in acceptance of recording technology, it's likely that people will embrace these ambient monitoring tools. That being said, there are obvious risks.

I don't want to end up in a Black-Mirror-episode big-brother-surveillance-state either.

It's imperative we design a regulatory framework that makes sure this is done in a privacy-first way.

The Ambient Recording Revolution is coming

This is how I hypothesize it will play out.

1. Productivity tools will rule - Audio only.

Unobtrusive personal augmentation tools, like recording and memory support devices, will begin to assimilate people to the idea that being recorded is okay.

Key Challenges:

• The regulatory environment will need to re-calibrate to the idea of processing vs storing.

• Significant value must be demonstrated to perpetuate adoption of these systems.

2. Audio and visual personal augmentation will arrive

When people feel comfortable with audio, video will arrive and unlock significantly more productivity gains. In this phase we will use hardware to collect data from the world around us and use our phone to process and surface useful information in real time.

Key challenges:

• Hardware advances will be necessary. Today we don't have the compute capacity to process a 4K video in real-time at the edge. We don't have the network bandwidth to process real time on a local device. We don't have the battery capability to support a constantly streamed input, and we don't have small enough, light enough cameras to make these systems unobtrusive.

• Consent networks will come into play as we apply the idea of social networks to the world around us.

  • You can limit these tools to only processing and memorializing your own information, but there will emerge a consent framework similar to follows or friends that allow your information to be viewable & savable to others.

• Capture of biometric data (e.g., using someone's face to identify them) will also be a critical component of value creation with its own set of regulatory challenges.

  • Biometric identification laws differ state by state. The overarching precedent is that individual consent is required before being biometrically identified by face/fingerprint/eye/etc.

Facebook was sued in 2015 by the state of Illinois for their AI auto-tagging feature that used faces to identify and tag who was in a photo with you. The lawsuit settled for $650M in 2021. At the same time, companies like Clear and Apple are using facial recognition with users' consent across the nation.

Just imagine you're walking around and you see someone you know but you forget their name. You could have their face recognized and their LinkedIn profile automatically surfaced on your phone without you having to do anything.

3. Augmented Reality will finally get good enough

Eventually the technology will get good enough to be a closed loop system, using your wearable to collect, process and surface the information in real time.

Key challenges:

• All the hardware challenges from above and more. AppleVision Pro had a pretty amazing interface but it was too clunky to wear for extended periods of time. We could be 5 years away with the Orion glasses from Meta which still look quite big and clunky, or it could take longer.

• Augmented reality won't be successful unless we have killer use cases to put on top of it, such as social networks and personal augmentation technologies.

I don't think we need to wait for them to drive the ambient recording revolution.

We recently invested in two Harvard undergrads Caine Ardayfio and AnhPhu Nguyen who are building the first facial recognition social network. I am so excited to see this type of technology enter the world, and I encourage you to check out their video in which they talk about how it all works.

If you're excited about this too, please reach out. If this makes you angry, I'd love to hear why or what unforeseen consequences I am missing (I’m sure there are many).

I wish I could make this a more comprehensive poll but I cant.