Long AI chats degrade in ways that look like bugs. Claude starts summarizing earlier messages to stay inside its window, which Anthropic’s help centre describes as Claude “organizing its thoughts” during long conversations. ChatGPT gets slower and starts losing the thread. Nothing in either interface tells you how full the conversation is, so users blame the model, the network, or whatever tool sits in the page.

I co-build a browser extension for ChatGPT, Claude, Gemini and Grok, and we shipped a context window meter: a small gauge in the corner of the page showing roughly how much of the window this conversation has used. The token math turned out to be the simple half. The hard parts were the denominator and deciding which copy of the conversation to trust.

The numerator: an estimate that admits it is one

We do not run a real tokenizer in the page. Each vendor uses a different one, they change, and shipping one per platform into a content script buys precision that the rest of the calculation cannot honour anyway. So the count is a character heuristic, the same one our word counter already used:

const CJK_RE = /[\u3000-\u9fff\uac00-\ud7af\uff00-\uffef]/g;

export function estimateTokens(text: string): number {
  if (!text || text.trim() === "") return 0;
  const cjkChars = (text.match(CJK_RE) || []).length;
  const otherChars = text.length - cjkChars;
  return Math.ceil(otherChars / 4 + cjkChars / 1.5);
}

Four characters per token for Latin text, 1.5 for CJK, which packs far more meaning per glyph. It ignores files, images and whatever system prompt the vendor injects, so it reads low on conversations with attachments.

That imprecision is fine, and saying so is part of the design. A meter that claims to be exact invites a user to check it, and the moment they find it off by 8% they stop believing the part that matters: that they are close to the ceiling. The number is labelled an estimate in the UI for the same reason.

The denominator is where a meter earns or loses its credibility, because being wrong about the ceiling is not imprecision, it is a different answer.

The denominator: context windows are a product decision, not a model fact

The context window of a consumer chat app is not the model’s API window. It is set by the vendor per plan, and it changes. This is the part most people get wrong, including us, twice.

For ChatGPT, the numbers come from OpenAI’s own plan comparison table, and they differ across three plans and two kinds of model:

export function chatgptWindowTokens(modelLabel: string, plan: ChatGPTPlan): number {
  const reasoning = isChatGPTReasoningModel(modelLabel);
  if (plan === "pro") return reasoning ? 400000 : 128000;
  // Go and Plus are the same two numbers on OpenAI's table.
  if (plan === "plus" || plan === "go") return reasoning ? 256000 : 54000;
  return 27000; // Free (its reasoning cell says "Varies", so Free cannot pin one)
}

A Plus user on an instant answer has 54K. The same person on a reasoning model has 256K. Same account, same page, nearly five times the room, and nothing in the interface says so.

Claude splits by model rather than by plan above the free tier, and Gemini by plan rather than by model. That last one caught us: we shipped “free users get the small models at about 128K”, and by August 2026 both halves of that sentence were wrong. Google’s help page now says free and paid users get the same three models, and the plans differ by window, 32K against 1M.

Detecting which case you are in has its own traps. Reading the model label from the DOM looks obvious until a user opens the model picker, at which point the page contains every model they could switch to, and a naive scan happily reports whichever one is first in the list. Our detector runs two passes and skips anything with a menu item role, so an open picker cannot vote. On ChatGPT we gave up on inference entirely and ask the app: /backend-api/accounts/check returns a plan_type, which beats guessing from a label that is often not on the page at all.

The bug: three machines, three answers, all correct

A user reported that one conversation read 2%, then 12% after a reload, then 64% on a second computer. Same account, same build.

Both numbers came from sources that are complete only by accident:

1. Our local cache. Background sync stores conversations in IndexedDB per device. A machine that synced yesterday holds a shorter copy than the one you are typing in now.
2. The rendered page. Long transcripts are virtualised. What is in the DOM is what you have scrolled through, not what exists.

Summing either one gives you a defensible number and the wrong one. Worse, the failure is silent: a partial cache looks exactly like a complete one.

The fix was to stop reconciling two partial sources and read the authoritative one. The platform’s own conversation endpoint returns the whole thread, identically on every device. The rules for when to call it live in a pure module so they can be tested rather than buried in a widget:

export const LIVE_REFRESH_MIN_MS = 20_000;

export function shouldReadLive(input: LiveReadInputs): boolean {
  if (!input.hasLiveReader) return false;
  if (!input.readThisConversation) return true;      // first read is unconditional
  if (input.msSinceLastRead < LIVE_REFRESH_MIN_MS) return false;
  return input.domTurns > input.domTurnsAtLastRead;  // only when turns were added
}

Three decisions in that function are worth more than the code:

• The first read per conversation is unconditional. There is no test that proves a cached copy is complete, so “refresh only when the cache looks short” cannot close the gap.
• Later reads compare on-screen turn counts to each other, never to the stored transcript length. The rendered count is not the conversation’s length, but a rise in it does mean something was added.
• The two numbers are reconciled by taking the larger, never the newer.

export function reconcileTokens(cacheTokens: number, liveTokens: number): number {
  return Math.max(cacheTokens || 0, liveTokens || 0);
}

That last one is the asymmetry that makes the feature honest. Both sources describe the same conversation, so the bigger number is the more complete view, and under-reporting is the failure that actually hurts. A meter that says 2% on a chat about to hit its ceiling is worse than no meter, because the user acted on it.

The judgment call we wrote down instead of guessing

When Anthropic published its chat window buckets, one model was missing from the chat page: Fable 5, named only in a sentence about Claude Code. A literal reading would drop it into the 200K remainder. We kept it at 500K and left the reasoning in the file:

// Fable 5 (not 5.1) is HELD at 500K by an explicit product decision, not by the
// page. Anthropic names it only in the Claude Code sentence, so a literal reading
// would drop it into the 200K remainder below, but "undocumented" is not the same
// as "confirmed small", and halving a flagship's denominator on an inference would
// make the gauge cry wolf on every long Fable chat. It stays at the previous value
// until Anthropic publishes a chat figure, at which point this branch is deleted.

Any team that mirrors someone else’s numbers ends up with a version of this: a value you cannot source, a default that has to be wrong in one direction, and a future reader who needs to know which way you chose and why. The comment is the deliverable, not the constant. A number with a deletion condition attached is maintainable; a magic number is a landmine for whoever inherits it.

What happens when the gauge fills

A meter that only reports is half a feature. The useful moment is the one just before compaction, when the conversation still has everything in it.

So the premium half of the feature carries a conversation forward: summarize the thread, open a fresh chat, and paste the summary into the composer as the first message. It works across platforms too, so a Claude thread can continue in a new ChatGPT chat. The seed is stored keyed to its target platform with a short expiry, and only a freshly booting content script claims it, so a later unrelated visit can never receive a stale paste.

The fallback matters more than the summarizer. When the backend is unreachable, we build the summary locally with no API call at all: short chats are carried verbatim, and long ones become an extractive excerpt, the opening request plus as many recent turns as fit a 7,000 character budget, with an explicit marker where the middle was dropped. The start says what the work is, the end holds the current state, and the middle is the safest thing to cut.

Key Takeaways

• Precision in the numerator is worth less than honesty about it. A character heuristic labelled as an estimate is more useful than a precise number the user cannot verify.
• Consumer context windows are per plan, per model and subject to change. Source them from vendor documentation and keep a dated comment saying when you checked.
• A cache and a rendered page are both partial. If correctness matters, read the authoritative endpoint once per conversation, then only when the page proves something changed.
• Pick the failure direction deliberately. Taking the larger of two token counts is not a rounding choice, it is a decision about which way the feature is allowed to be wrong.
• Write the reason next to the constant. A held value with a deletion condition survives the next maintainer; a bare number does not.

FAQ

How accurate is a character based token estimate? Close enough for a gauge, not for billing. Four characters per token is a good approximation for Latin script and 1.5 fits CJK better, but the estimate excludes files, images and vendor system prompts, so it reads low on conversations with attachments. Label it as an estimate in the UI.

Why not run a real tokenizer in the browser? Each vendor tokenizes differently and those tokenizers change. Shipping one per platform into a content script adds weight and maintenance for precision the rest of the calculation cannot match, because the true denominator and the hidden system tokens are unknown anyway.

Where do the context window numbers come from? Vendor documentation, not model cards. OpenAI publishes windows per plan in its plan comparison table, Anthropic lists chat windows per model on paid plans, and Google states free and paid windows for the Gemini app. We keep the plan by plan numbers current in our ChatGPT context window guide.

What is context compaction? When a conversation approaches the window limit, Claude summarizes earlier messages so the chat can continue, and the full history stays available to it. Anthropic says users may see Claude “organizing its thoughts” while this happens. It is not an error, but it is the moment older detail stops being quoted back verbatim.

If you work inside ChatGPT, Claude, Gemini or Grok all day, you can try AI Toolbox free on the Chrome Web Store. The context meter is free on all four platforms. Premium adds the carry forward handoff, full text search without a result cap, unlimited folders and bulk export, at $9.99 a month, $59 a year or $99 lifetime, with All Access at $199 for all four modules. These are browser extension features, not an AI subscription.

What does your team do when a conversation fills its window: start over, summarize, or split the work earlier?

About the author: Adi Leviim is the co-founder of AI Toolbox (formerly ChatGPT Toolbox), a Chrome extension with modules for ChatGPT, Claude, Gemini and Grok, used by 40,000+ people across 150+ countries to search, organize and export their AI conversations. He writes about the reality of building AI products with 7+ years of full-stack development experience. Follow him on Medium for honest takes on SaaS, AI tools, and shipping software that people actually use.

AI Toolbox | Medium | Twitter/X | LinkedIn

I Built a Context Meter for Four AI Chat Apps. Counting Tokens Was the Easy Part. was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.

The Problem: How to Analyze Massive Datasets

Imagine you have terabytes of website logs tracking every single visitor interaction, and from here, you want to filter out some information, like which pages are most popular or where visitors drop off in your purchase funnel, etc.

Press enter or click to view image in full size

Traditional tools and databases are simply not designed for datasets of this scale. That’s where MapReduce comes in.

What is MapReduce?

MapReduce is a programming model designed specifically to handle the challenges of processing enormous amounts of data that just won’t fit on a single computer. It was introduced by Google in 2004 to tackle exactly these kinds of scenarios. Let’s see how it works through our website log example…

How MapReduce Handles Big Data

MapReduce operates in two primary phases — The map phase and the reduce phase.

Map Phase

In the Map Phase, we first split these huge logs into smaller and manageable chunks. These chunks then get sent to different worker computers in a cluster.

Press enter or click to view image in full size

Think of each worker as a separate server that handles its assigned chunk. It has a Map Function that extracts the key information: in our case, it will map the keys, which are the specific webpage visited, to the values, which, if we are counting visits, can be the number of visits to that page (e.g., 1)

Reduce Phase

Then, we enter the reduce phase, where all the key-value pairs generated by the map phase are sorted and grouped by webpage (‘key’).

Press enter or click to view image in full size

We forward those to the Reduce Function. For each unique webpage, it adds up the ‘1’ values to find the total visits. It can also tackle more complex questions, such as average time spent, visitor demographics, etc.

And now, using this information, we can visualize it via charts and other visuals on the screen.

Press enter or click to view image in full size

Benefits of MapReduce for Log Analysis

We get a couple of benefits when processing data with MapReduce:

• Parallel Power: Distributing the work makes processing much faster than a single computer could manage.
• Scalability: Got even more log data? Just add more computers to the cluster and MapReduce can keep up.
• Fault Tolerance: If a computer fails during a job, MapReduce automatically reassigns its work to other computers in the network. This ensures that all the tasks are completed successfully without interruption.

Batch vs. Stream Processing

To understand why MapReduce is so unique, let’s quickly touch on batch versus stream processing:

Batch Processing

Batch Processing deals with data in large chunks that have already been collected. For example, if you search for a word in Google Docs or Microsoft Word in a large file, the data is available upfront, so it can be processed immediately.

Press enter or click to view image in full size

This is useful for large datasets where immediate results aren’t essential, such as when generating monthly sales reports, analyzing customer purchase history, or training machine learning models on data.

Streaming Processing

Stream Processing handles data as it arrives in a continuous flow. For example, when watching a YouTube video, you hit ‘play’, and it starts almost immediately. That’s because tiny pieces of video are sent to your computer in a continuous flow, letting you watch while the rest of the video is still being transmitted.

Press enter or click to view image in full size

Streaming is ideal for situations requiring immediate action on data streams, such as when you want to identify suspicious activity in financial transactions or when you need real-time analytics for social media feeds.

Micro-batch Processing

We also have micro-batch processing, which is a hybrid approach that bridges the gap between traditional batch processing and stream processing.

Instead of processing all of your data in one huge batch, micro-batch processing breaks data down into very small batches. These batches are processed at short, fixed intervals (often in seconds or minutes).

Press enter or click to view image in full size

Micro-batching is often the preferred method for scenarios demanding faster results than traditional batch, but where full-on streaming isn’t necessary.

What is the processing method used by MapReduce?

MapReduce is a batch-processing model because it operates on data that is already stored, not on a live continuous stream of incoming data. Input data needs to be divided and distributed before the Map phase of MapReduce even begins.

As you can imagine, Batch processing is slower than Streaming due to the accumulation of data before processing. But it’s generally simpler to set up and manage, while Stream Processing can be more complex due to the constant flow of data and the potential for errors or inconsistencies.

MapReduce Limitations and Modern Alternatives

While MapReduce was revolutionary, it has limitations in terms of speed and flexibility for iterative and complex data processing tasks. This is where tools like Apache Spark come in.

Apache Spark

Spark leverages in-memory processing, meaning it keeps data in RAM for very fast calculations compared to MapReduce’s reliance on disk storage. It handles a wider range of tasks, including SQL queries, machine learning, and real-time data processing (streaming).

Apache Flink

Apache Flink is another powerful framework used for real-time data processing (stream processing). It offers similar capabilities to Spark Streaming, allowing for immediate analysis of data as it arrives. This is a specialized tool for scenarios requiring real-time data analysis, often used alongside Spark for a complete big data processing toolkit.

Hadoop

Hadoop is a broader ecosystem that provides the foundation for tools like Spark and MapReduce to run. It includes a distributed file system (HDFS) for storing large datasets across multiple machines and a resource management system (YARN) that allocates resources (CPU, memory) to applications like Spark or MapReduce.

Think of it as the underlying infrastructure that Spark and other tools use to manage and store big data.

Cloud-based Services (AWS, Azure, GCP)

Cloud providers like AWS, Azure, and Google offer managed data processing solutions that often streamline the use of MapReduce frameworks. These include AWS EMR (which supports Hadoop), Azure HDInsight, and Google Cloud Dataflow (Google’s successor to classic MapReduce, built for both batch and streaming data).

In Conclusion

While MapReduce was a breakthrough, Spark has largely taken its place for most modern big data batch processing tasks. However, understanding MapReduce is still important because it gives you a solid foundation for understanding how these powerful tools work.

Map Reduce explained with example | System Design was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.