Reflect | How A.I. Will Unlock Our Minds (2/2)

"AI-assisted Synthesis" will unlock new learning, innovation and self-discovery

This article is Part II of a 2-part series on Reflect. Part I was an interview with Reflect’s founder and CEO, Alex MacCaw where we explored the future of note-taking; one powered by AI. In this article, I expound on that vision in more detail, exploring the implications (good and bad), of a personal A.I. that knows your most private thoughts.

---

Ever since seeing Iron Man in 2008, I've wanted my own J.A.R.V.I.S. A secure, private A.I. companion that can "see" what I see and "hear" what I hear, turning me into a real life Iron Man (minus the billions of dollars of net worth).

So when Humane previewed their AI-powered wearable just a few weeks ago, needless to say, I got excited about where the future was heading. I even wrote about a concept called Ambient Computing that posits that technology will eventually fade into the backgrounds of our lives, only surfacing when needed and allowing us to finally move beyond the screen and engage with the world around us in a more present way.

But there are limitations to this sort of wearable. While it can see and hear the world around you, it doesn’t know what's going on in your own head. For some, that's a comfort. Your mind is the final frontier, unknown by an ever-present A.I. But if the goal was for it to be the most useful it can be, like J.A.R.V.I.S., that's quite a handicap. After all, if you wanted an intimate thought-partner, you'd want it to know you, to know how you thought about the world and how you wanted to process it.

In the end, our A.I. companion is meant to be an extension of ourselves, not another being entirely.

So how do we open up our minds to an A.I.? Well… barring some breakthrough advancements in brain-machine interfaces by Neuralink, most of that knowledge, if it were to ever leave our brains, would probably end up in something like a notes app, right?

How many of us have opened up Apple Notes on our phone and just did a brain dump to organize our thoughts? Or to make a list of tasks? Or journal about a memory we don't want to forget? Writing it down or talking through it with someone is just the natural ways we offload the weight of our thoughts.

These notes are often the most vulnerable things we own so it's important that they're private. But also, it's important that they're accessible by our A.I. self.

That's the future that Alex MacCaw, founder of Reflect is trying to build.

Let's dive in.

Welcome to Making Product Sense
Join thousands of product leaders and founders from the top companies in the world, who are learning how to hone their product sense.

The Evolution of Notes

If our notes contain the private thoughts of our lives, it’s important to understand how A.I. is bringing our notes to life. Any good note taking system needs to have three parts:

1. Ingestion - how do you get content into the system?

2. Synthesis - how do you remix and combine ideas within the system?

3. Retrieval - how do you extract the information you need from the system?

Phase I: Analogue Everything

The OG note taking system - the Zettelkasten - would use a system of index cards, meticulously organized with references to other cards in the system, effectively allowing you to follow a path of interconnected ideas across your note taking system. Ingestion, synthesis and retrieval was all incredibly manual and tedious, but it got the job done.

Phase II: Digital Ingestion

Thankfully, by the time I started taking notes, software existed. Around 2010, as Evernote was coming into its own, I got into this idea of a "second brain." A place where I could dump everything in my life and know I could track it down later as needed. Over 4,000 notes later, my Evernote truly was an extension of my own life.

It was incredible at ingestion. It had a web clipper, a mobile app with a page scanner and support for all sorts of file types like images or PDFs.

But synthesis and retrieval were difficult. I had to come up with elaborate systems for organization to ensure I could track down a thought. And trying to pull ideas together from across multiple notes was cumbersome to say the least. I couldn't organize notes within notes without manually copying a note link and pasting it inside of another note. So everything existed on the same plane of thought.

Phase III: Graph-based Synthesis

Then, around 2017, along came powerful outliners like Roam Research and Logseq (and now, Reflect) that made fluid connections between notes through nest-able nodes and backlinks. It allowed me to connect ideas without the need for rigid organizational structures like folders and my brain exploded. My wife could tell you, probably with some degree of annoyance, just how excited I was about this new way of taking notes. I could finally write in the same way I thought! And with Roam's daily notes page, I found myself starting to journal for the first time.

The unfortunate downfall of these outliners was that, while they were incredible for synthesis, retrieval still lacked a certain fluidity because it would require traversing a web of nodes and backlinks to track down all of the thoughts I had on a particular subject.

Phase IV: Conversational Retrieval

Then, late last year, A.I. exploded onto the scene, poised as the perfect companion to these incredible tools for thought.

It could understand meaning and it could converse in natural language. If it had access to our notes, in theory, it could be the perfect ingestion, synthesis and retrieval mechanism. But as with any new technology, there are limitations.

1. Context: For one, the context is too small. You can't pass in every single note you've taken so it's limited in what it can "understand."

2. Privacy: Additionally, with notes being as personal as they are, sending your raw thoughts to a third-party API like OpenAI's GPT-4 feels too... vulnerable. We need a more private option.

3. Size: If you're running a model locally, it has to be small enough (while still being powerful) to be able to run on a device like a smartphone.

What we didn't anticipate last Fall was just how rapidly things would advance in the world of A.I. Several models have now been released with a 100K+ context window, powerful open source models are being released every week that could bypass the need to send our notes to a third-party API, and now there are even decent models that can run on a mobile phone.

^ That paragraph is probably already out of date

By the end of the year, a lightweight, open source model with a large enough context window that it can ingest our entire notes archive could be available offering the privacy, speed and memory that we need to have a truly robust A.I. companion within our notes.

---

Personal A.I.

Suddenly, I can't help but revisit the idea of a personal A.I. that I envisioned after Humane's announcement. I can see two paths for a “Personal A.I.”, each serving different purposes. Whether or not they’re the same A.I. or two different ones, we don’t know yet.

Humane’s device ingests audio and video and could help you traverse the world around you. But a truly personal A.I. that ingest your notes could help you traverse the world within you.

Imagine being able to analyze your activities to find out what you were doing on the days when you were happiest so you could make time for more of those things. Or find out who are the five people you spend the most time with to make sure you're being influenced by the right friends.

I journal just about everything I do, tagging the people, places, things and activities so that my notes are a living record of my life. I can already answer many of these sorts of questions with the power of queries and a little elbow grease. But there are some questions that still aren't easy to answer.

For example, imagine being able to identify and process unhealthy patterns in your life and have your personal A.I. suggest ways to adjust your thinking or habits to embrace a healthier way of life.

Or, imagine being able to ask it what sort of job would suit you best. It knows what activities give you joy and align with your deepest desires, what skills you have and how you like to work. It would probably even know your salary needs. It can then go and find jobs that meet those criteria that you might not even know exist.

This sort of deep, personal analysis with an A.I. thought partner unlocks a lot of new capabilities including some that have been previously-unknown to us.

For example...

---

AI-assisted Synthesis

A new superpower has been unlocked... I'm calling it AI-assisted Synthesis.

It's a 3-part cycle that turns the three elements of a good note taking tool - ingestion, synthesis and retrieval - into a flywheel, assisted by A.I.

1. You add some seed material (ingestion).

2. You ask a question of that material and the A.I. responds with a pre-synthesized answer (synthesis and retrieval).

3. You then draw new conclusions and respond, adding new material back into the source context (synthesis and ingestion).

4. As you continue to talk with the A.I., the conversation would eventually yield both breakthroughs (synthesis) and documentation of said breakthroughs (ingestion) for later use (retrieval).

For thousands of years, people have had to spend lifetimes learning about a subject, understanding it, collaborating with others, and tinkering with the knowledge until the right things are in their head at the right time and they can finally make a breakthrough.

In the early days of the internet, the idea of having the world's knowledge catalogued and searchable was almost overwhelming. Rather than manually combing through volumes of books and scientific journals, we could type a simple topic into a search box using natural language and be served up the most relevant information human kind had on the subject. That was an exponential leap in progress for knowledge synthesis and retrieval.

And the fact that you could then contribute to that body of knowledge by publishing your own thoughts and findings onto the internet made the ingestion of knowledge almost trivial. You still had to comb through the endless list of websites manually, but this time, they were filtered and sorted by relevance.

Now, if the insights you had 10 years ago can be surfaced alongside the insights someone else had 100 years ago and the insights you just had 10 minutes ago, you can find connections in near real-time that may have taken a decade before.

Suddenly, the beautiful art of note taking gets enveloped by a bigger, more important paradigm. AI-assisted Synthesis can give each of us the ability to make connections we would have never made before, personal, professional or otherwise. Our note taking tool is no longer just a repository of personal thoughts and feelings. It truly becomes our second brain, storing the cast away ideas, the random snippets of thought we never thought we’d need, ready to pull them together into something new.

---

How “future” are we talking?

So where are we with regards to this sort of fanciful view of the future?

Honestly... not far off.

Let's take a tool like Reflect. By the end of this year, they'll probably have something resembling an MVP of what I've illustrated above. A local A.I. model that's private and encrypted could run on your device offering you a chatbot that can help you parse through your own thoughts. It would be a place where you can read, write and chat with your notes.

But now imagine 5 years down the road. If Reflect could run a companion app on Humane's wearable device, you could have access to your personal A.I. whenever you needed, and your personal A.I. would have access to the sights and sounds of the world around you, understanding you and your life better with each day.

And since it could be connected to the internet, it could inject knowledge and insights from the world wide web to help you learn and synthesize even more ideas while keeping your personal knowledge repo private and secure.

This hybrid system of real-time ingestion, AI-assisted synthesis and conversational retrieval is a level of super-intelligence that we've never seen before. A personal J.A.R.V.I.S. - a genius thought partner - for every mind.

Call me crazy, but I'm confident that more breakthroughs will happen over the next 10 years as a result of A.I. than we've seen in the last 50.

---

But aren’t there risks?

I would be naive if I didn't see the potential drawbacks to these sorts of advancements. This could turn into some sort of dystopian Black Mirror episode where we can play back and obsess over negative memories. Or we ask the A.I. about our unhealthy behavior and it responds with an absolutely soul-crushing, deeply perceptive monologue that sends us into a spiral of depression. Who knows what might happen.

Needless to say, there will need to be guardrails. Like with any technology, there will always be bad with the good. I'm a bit of a techno-optimist so I believe the good will far outweigh the bad, but I would understand if you took the opposite position. Some days, I could go either way. We're living on the brink of a new world and we have no idea what it will hold.

But I'm curious to hear your thoughts.

• Is it good or bad if an A.I. can access your private notes?

• Will conversations with A.I.s lead to real breakthroughs like I believe it will?

• What do you think a personal A.I. should be like?

That’s all for this one - I’ll catch ya next week.

—Jacob ✌️