A Quick Lesson Of AI Languages
What this covers
LLMs don’t get “smart” because you throw more markdown at a bigger context window. They get smart when you give them the right kind of context.
In this whiteboard, I break down:
- How context windows actually work and why quality falls off past a fraction of the max tokens
- Why markdown became the default for agents, and what really happens vs JSON, YAML, and custom formats
- The tradeoff between structure and token efficiency (and why “more compressed” rarely means “more accurate”)
- Why you – the human – are often the real bottleneck, and how poor intent and pointing kill agent performance
- The underrated power of the @ symbol as a way to reference files, objects, and relationships in your harness
If you’re building AI agents, RAG systems, or “agentic” products and your answer to everything is “use markdown and a bigger context window,” this video is for you.
I’ll share what I’ve learned building and researching agentic systems over the last year, and why we need to rethink how we frame problems, point at objects, and design context for long-horizon work.
Get in touch with me! twitter: x.com/shinyobjectz_
Source description (no synthesized summary yet).
The fundamental problem in AI engineering is not larger context windows or smarter models, but rather how we structure and communicate intent within existing contexts—particularly through better use of formatting conventions and disambiguation tools like the @ symbol.
- LLMs are most effective at only ~20% token consumption; larger context windows don't solve the efficiency problem
- Markdown outperforms all other formats (XML, JSON, YAML) across benchmarks for LLM context retrieval, yet the underlying issue is human communication of intent, not format choice
- The @ symbol and explicit intent-clarification mechanisms are underutilized tools for disambiguating context and improving agent performance
This asset isn't compiled yet
You're seeing its claims, ranked. Compile it to build the argument threads, weight them, and check each claim against your library — the full view.
Context is fundamentally one of the hardest problems that scientists, researchers, and developers want to solve today, and how we structure, use, and deliver context is core to how LLMs and agents appear intelligent.
“This is fundamentally one of the hardest problems that scientists, researchers, developers, all want to solve today. Context is how and why LLMs and agents are so intelligent, seemingly, is because of how we structure, use, and deliver, and train on this context.”
The real problem in AI context management is not harnessing the technology correctly—it is not solved by reducing tokens, making models smarter, or expanding context windows, but rather requires reconceptualizing how we interact with, point to, and communicate with AI systems.
“And I think this problem goes back to something even more intrinsically apparent before AI was a boom. And that is intent, purpose, and productivity. I want to discuss really complex topics like this. I want to break it down as simply as I can, and I want to kind of start proposing my theories, the things I'm seeing as a developer and researcher working on AI and AI primitives, on why we need to change the way we think about working with AI. The context window's bigger and bigger. Those are not efficient solutions. This is a problem I've seen over and over again. The way we interact with these objects, point to them, and communicate are really a fundamental issue that goes beyond just the context window.”
Recent claims about massive context windows (e.g., 1 million tokens for Claude, 2 million for Grok) are misleading because LLMs are most effective at only approximately 20% token consumption; at 60% consumption, performance already shows steep declines.
“Ooh, 1 million token context window. Ooh, Grok came out. Ooh, Grok, it has 2 million token window. Wow, 2 million, huge. You can fit anything in there. Well, turns out that doesn't really mean anything. LLMs are at their most effective, no matter how big this is, really only like 20% token consumption. Meaning, if you're only using 20% of the available token consumption is where they're the most effective. And then, as you start to use up more token usage, they get a little dumber and dumber to the point that not even 60% of the way, you have steep declines in performance.”
The @ symbol is fundamentally underused in AI harnesses and prompting, despite its established role in Slack and file references as a tool for pointing to and creating relations between intents and objects within context.
“Think about it. What was the last time you used the at symbol? You use it in something like Slack to mention someone. You use it to reference a file. You use it to point or create relations between what your intent is within this context. And I say, 'Hey, I'm talking about that file. I'm talking about that thing.' That at symbol is fundamentally probably one of the most underused tools in any of these harnesses.”
Humans are a liability for LLM performance because people are bad at prompting—both machines are bad at prompting themselves and for others, and performance improves when humans are better at clarifying their intent.
“Now, one of the other problems is you. Now, I mean that with all due respect, but you as the person are a liability for the performance of how this machine does. And we found in other studies that machines are actually terrible at prompting. They're very bad at prompting for themselves and prompting for others. And performance happens when lazybones is better about clarifying their intent.”
Long-horizon agentic systems (e.g., OpenClaw) require the ability to work continuously over hours while staying within the 20% effective token consumption range, which is a fundamental constraint that larger context windows do not solve.
“Because, as of today, agents and long horizon tasks, agents like OpenClaw, their entire goal and purpose is to work over a long period of time, scheduling tasks over a long period of time. Companies are investing in systems that can provide hours of continual work. How does that work when you can only consume 20% or so? You need it to work over hours within this 20% marker.”
Problems with how humans interact with AI systems and communicate intent have historical precedent in pre-AI era challenges and represent a recurring structural issue that transcends technological change.
“And it's become apparent to me that there's a crystal-clear problem that needs to be solved that goes beyond just solving for the context window, reducing tokens, making the models smarter and smarter. This is a problem I've seen over and over again. And to be honest, it seems very reminiscent of problems I've seen in my career over the years before AI was really even a boom.”
While Tune and token-optimization techniques can reduce token consumption, they do not improve accuracy beyond markdown—demonstrating that reducing token count does not necessarily translate to better LLM performance.
“But that said, you still run into a lot of the same problems where you're not necessarily becoming more accurate than markdown. You can be just as token efficient, but not more accurate. So, even though you can reduce token count, it doesn't necessarily improve accuracy.”
Across several different benchmarks, markdown has consistently ranked number one for retrieving and feeding context to agents, with XML as a distant second, though the reasons why remain unclear.
“When scientists tested a bunch of different file formats, this one always came out number one. Markdown has predominantly become the number one for retrieving and feeding context to the agents. For reasons why, I still think those are up in the air, but across several different benchmarks, I found that they all really do point that markdown is generally the best format. So, pretty much, markdown's number one. XML is kind of a distant second, and everything else kind of follows behind.”
JSON and other structured formats consume more tokens than plain text because they include repeated declarations of labels and schemas that serve as machine-parsing guidance but represent wasted token consumption for the LLM.
“Like, for instance, JSON has repeated declarations of certain labels or certain schemas. We often have these appendages that help guide a machine to parse it. But for the LLM, that's just another fraction of a cent. That's another token. That's another token that fills up this window.”
YAML and other formats offer token-efficient and composable structures, but these structural improvements do not change how the LLM thinks, interacts, or processes context, and therefore do not outperform markdown.
“Now, there's other things like YAML, where you have maybe a little bit more of this native Tune-type token-efficient language, but again, you have a lot of repeated characters, you have formats that are natively composable and flexible, but those structures don't change how the LLM thinks. It doesn't change how it interacts, and it still fills up the context window. We're not achieving anything better than markdown.”
Markdown is the de facto standard format for communicating with LLMs and delivering context because it is human-readable and supports rich text formatting (headings, bullets, callouts), and has achieved widespread internet adoption.
“One of the most common types of token arrangement, really just how we structure our inputs to the LLM, is called markdown. Markdown is known as a very human-readable and rich text format, meaning it's easy to put headings and bullets and callouts. And this format has, because of its adoption on the internet, has become the de facto way that we communicate.”
Tune is a repository that uses an encoder and decoder to convert JSON into a more token-friendly format, allowing structured data to be represented with less token overhead while maintaining parsability.
“When it comes to JSON, there's actually something called Tune, which is actually a really brilliant repository you should look up, where they use a encoder and a decoder to essentially turn JSON into a more token-friendly format.”
LLMs work as text prediction models, and the way tokens, words, and letters are structured across a context window in the right order affects how the model predicts the answer at the end.
“This is how LLMs work. They're basically just really fancy text prediction models. And the way that we structure this content does provide meaningful results.”
The speaker has been working on agentic systems for at least one year, collaborating with multiple companies and developing independent solutions, providing empirical grounding for their claims about context windows and agent design.
“I've been working on agentic systems now for at least a year, working with different companies, working on solutions myself, studying, learning.”
The speaker is planning to introduce a new agent harness framework that should fundamentally change how the field talks about and approaches AI and agent systems.
“I'm going to be talking about a new agent harness that I think is going to change the way, at least, we should start talking about AI and agent systems today.”