Yes this is true, you even got the model correct. I relied too much on Claude this time. I'm starting to work on the next AI agent lesson this week. I'll do better on the upcoming lesson. The next lesson is A2A focused
The writing is slop. It doesn't mention a detail I have seen people actually not get: that LLMs wrap tool calls in special tokens, so they are "out of band" and can't be mistaken with normal output.
It also spends an entire section trying to convey that large tools waste tokens:
A read_file that returns an 8k-token source file on turn 2 of a ten-turn agent gets resent on the eight requests that follow: 8 × 8k = +64k input tokens, $0.32, from one tool result.
but surely that's wrong - it's part of the same conversation, it only gets processed once and then cached. Otherwise doing long conversations would always cost an amount quadratic with length.
> I have seen people actually not get: that LLMs wrap tool calls in special tokens
This isn't necessarily true. I'm working on a local harness that doesn't do this and instead coerces everything to YAML (including tool calls) for better bucketing. Some models are indeed trained on the `<|tool_call>...<tool_call|>` token schema (or something similar—e.g. jinja), but it's vendor-specific and often times inconsistent (so you're constantly fixing calls or going back to the LLM).
nice! like HTML tags. I remember seeing this back with meta Llama 2. the <|start_header_id|>assistant<|end_header_id|> style. And [TOOL_CALLS] ... [/TOOL_CALLS] style with Mistral
You a are right as The article doesn't mention KV cache. Also, yes the writing is slop, and does not mention the LLM special tokens. Only discusses the use through the OpenAI-style JSON wrapper that allows you to define the schema of a tool call in JSON. For most of the audience, they are looking to build AI agents, and the high-level tool calling interface is what they would be using
Yes! You saw it immediately. Article was written by Claude. It has a distinct cadence that matches all the way through writing. It's the same in the CLI claude code. I'll make sure the upcoming lesson will be less ai generated text. I usually like to focus on writing the most read parts. Like the intro and core focus of the article
Thank you Claude, very cool!
Yes this is true, you even got the model correct. I relied too much on Claude this time. I'm starting to work on the next AI agent lesson this week. I'll do better on the upcoming lesson. The next lesson is A2A focused
The writing is slop. It doesn't mention a detail I have seen people actually not get: that LLMs wrap tool calls in special tokens, so they are "out of band" and can't be mistaken with normal output.
It also spends an entire section trying to convey that large tools waste tokens:
but surely that's wrong - it's part of the same conversation, it only gets processed once and then cached. Otherwise doing long conversations would always cost an amount quadratic with length.> I have seen people actually not get: that LLMs wrap tool calls in special tokens
This isn't necessarily true. I'm working on a local harness that doesn't do this and instead coerces everything to YAML (including tool calls) for better bucketing. Some models are indeed trained on the `<|tool_call>...<tool_call|>` token schema (or something similar—e.g. jinja), but it's vendor-specific and often times inconsistent (so you're constantly fixing calls or going back to the LLM).
nice! like HTML tags. I remember seeing this back with meta Llama 2. the <|start_header_id|>assistant<|end_header_id|> style. And [TOOL_CALLS] ... [/TOOL_CALLS] style with Mistral
You a are right as The article doesn't mention KV cache. Also, yes the writing is slop, and does not mention the LLM special tokens. Only discusses the use through the OpenAI-style JSON wrapper that allows you to define the schema of a tool call in JSON. For most of the audience, they are looking to build AI agents, and the high-level tool calling interface is what they would be using
AI slop warning
Yes! You saw it immediately. Article was written by Claude. It has a distinct cadence that matches all the way through writing. It's the same in the CLI claude code. I'll make sure the upcoming lesson will be less ai generated text. I usually like to focus on writing the most read parts. Like the intro and core focus of the article
[flagged]