Hey HN, I built Decaf which is a Chrome extension that rewrites webpage comments using the Gemini Nano via the Prompt API which was being discussed yesterday here I saw that the top comment in the thread (https://news.ycombinator.com/item?id=47917026) was floating an idea for a “de-snarkifier”. The idea was to rewrite mean comments into civil ones, preserve the facts and stance and keep the original a click away So this extension sends the request to the Gemini Nano model in chrome which is under a feature flag - Few-shot examples loaded as user/assistant turns in initialPrompts is a trap with Nano really. At low temperature with a JSON schema constraint, the model literally just copies the last assistant example. I had it answering "Type your search here" to questions about composing posts. Moving the examples out of initialPrompts and into the system prompt as plain guidance text, plus bumping temperature to 0.6, fixed it.
Caveats according to the documentation of Prompt API (https://developer.chrome.com/docs/ai/prompt-api) Chrome 138+ desktop, ~22 GB free disk, 4 GB VRAM, 16 GB RAM. English-strong (en/es/ja). First rewrite per session is slow because the model warms up while the subsequent are sub-second on an M1.
Hey HN, I built Decaf which is a Chrome extension that rewrites webpage comments using the Gemini Nano via the Prompt API which was being discussed yesterday here I saw that the top comment in the thread (https://news.ycombinator.com/item?id=47917026) was floating an idea for a “de-snarkifier”. The idea was to rewrite mean comments into civil ones, preserve the facts and stance and keep the original a click away So this extension sends the request to the Gemini Nano model in chrome which is under a feature flag - Few-shot examples loaded as user/assistant turns in initialPrompts is a trap with Nano really. At low temperature with a JSON schema constraint, the model literally just copies the last assistant example. I had it answering "Type your search here" to questions about composing posts. Moving the examples out of initialPrompts and into the system prompt as plain guidance text, plus bumping temperature to 0.6, fixed it.
Caveats according to the documentation of Prompt API (https://developer.chrome.com/docs/ai/prompt-api) Chrome 138+ desktop, ~22 GB free disk, 4 GB VRAM, 16 GB RAM. English-strong (en/es/ja). First rewrite per session is slow because the model warms up while the subsequent are sub-second on an M1.