Taste Profile

I send myself Instagram Reels constantly: a skincare product to try out, a pour-over recipe, an edit that I found creative, and then I never open them again. I then realized that this was valuable data going to waste. Sending something to yourself is a much stronger signal than liking it. There's an intention behind every one: I found this interesting enough to want to come back to it at some point. It got to a point where I had a few hundred reels sitting in a DM thread that I knew I would do nothing about, so I built a product that would. Taste Profile turns those Instagram Reels into something I can actually ask questions of and utilize.

Capture runs entirely off the iOS share sheet (the menu iOS slides up when you tap Share in any app). The process is, I hit share on a reel, pick the shortcut, and it asks one question before anything else happens: why are you saving this? That answer records the intent rather than content itself. For example, if I send a reel to myself, answering the question with "More things I want" versus "coffee products to buy" send the same video to completely different places.

I. The only question it asks

II. Four actions, no app to install

The shortcut POSTs the URL and the note to a Next.js route on Vercel. That route canonicalizes the link, since share sheets hand over URLs with tracking parameters (the ?igsh= tail Instagram appends to a shared link so it can tell where the share came from), so the same reel shared twice collapses into one row instead of two. It then drops the reel onto a queue in Supabase and replies with whether the share was successful.

Here's why I set up a queue and not some other solution: the video processing can't run on Vercel at all. There's no ffmpeg binary, the filesystem is read-only, and functions on Vercel's Hobby plan are capped at 60 seconds, while transcribing a single Instagram reel can take tens of seconds on its own. So the split between processing and queuing is fixed: Vercel queues Reels in under a second, and a separate long-running Python worker drains the queue. This Python worker is automated to run every 6 hours.

In terms of what's coming out of the reels, there are three signals: audio, on-screen text, and the caption. The Python worker reads each reel in all three ways, because a reel's actual content might live in only one of them. Audio goes through Whisper, OpenAI's open-source speech-to-text model, running locally on my own machine, so there's no API key and no per-reel cost. On-screen text comes from keyframes pushed through a vision model, which I sample on scene changes rather than on a fixed interval because a 30-second reel holding one static shot gives ten near-identical frames while a fast-cut reel misses every actual cut, and each frame costs a vision call. All frames captured are bundled into one request, which is roughly twelve times cheaper than per-frame calls and this lets the model use continuity between frames to read text that's occluded in any single one. The model's smart, so it can fill in the gaps. Captions come along with the post. With the note I typed on my phone when initially prompted by the shortcut, the classifier has all the information it needs.

The classifier proposes a fit, but the taxonomy decides. This was the failure mode I was most worried about: when left alone, a model will happily label reels "coffee," "espresso," "Coffee Gear," and "brewing," producing four separate clusters for the same overall idea, and the Obsidian graph would fragment into something useless. Existing labels go into the prompt with an instruction to reuse them, but every label that comes back with no fit into existing labels is then pushed through a deterministic resolver, which normalizes it and fuzzy-matches against labels that already exist before it's allowed to create an entirely new one. Every new label that does get created is then logged to a drift file, so fragmentation of the graph is something I can measure by counting events.

The output of the classifier is plain markdown files. Each reel becomes a note with YAML frontmatter, the extracted products, the caption, the on-screen text, the keyframes copied in so the note isn't a wall of text about a video I can't see, and links to its clusters. Obsidian's backlinks do the rest, and the visual graph below is just what falls out of 88 reels linking to 16 cluster notes.

III. One cluster, and everything filed into it

What it's actually for. The whole point of this product was to make use out of all of these reels, and the ability to query through this graph allows that. Using Claude Code with the working folder pointed at my Obsidian folder, I can ask: "What skincare products have I sent myself in the past week to try? What coffee methods should I attempt this weekend? Give me the links to the reels I saved because I wanted to make something similar." Before this, the only way to answer those was scrolling my DMs for twenty plus minutes, which I never did.

IV. Whether the worker choked on it

Built with Cursor and Claude Code, on Supabase and Vercel, with a Python worker doing everything the serverless half can't.