Earlier this year I started competing at WorldQuant BRAIN. My philosophy is to do it part-time and in an AI-native way: I design the system and decide what it is allowed to do, while the agents do the investment research. At the moment of writing, 10th October 2026, my public leaderboard position is 13 / 151816, approximately the top-0.01%.

This is a series for individuals and organizations doing quantitative finance research who want to adopt an AI-native approach:

1. Introduction
2. Alpha Knowledge Graph
3. Fighting Overfitting
4. Context Engineering
5. Harness Engineering (this article)
6. Loops Engineering
7. Open Source LLMs and Harness
8. Future of Quantitative Research

In the previous articles, I described the economic knowledge, statistical methods, and context behind my research system. Having those things available does not mean the agent uses them. It can read the knowledge graph, generate a familiar formula anyway, run a backtest, and tell me that it found something promising. The missing part is how the research actually gets done.

My starting point was Claude Code as the base harness. It gave me a working agent environment that could read files, run tools, and delegate tasks. I expanded it with my own research skills, specialist sub-agents, code, hooks, economic knowledge, and persistent research records.

I believe this surrounding system matters more than the choice of model once the model is capable enough to do the work. To explain why, I will first describe the architecture behind a harness, then show how I built on Claude Code for WorldQuant and what those choices meant for real submitted signals.

The architecture around the model

Claude Code could already read files, run commands and ask another agent for help. I could get a backtest started. But when was it actually finished? Did the signal add anything to my existing book? Which lesson from yesterday’s experiments should guide today’s work? Those decisions became the system I built around Claude Code.

By harness, I mean everything around the model that determines what it knows, what it can do, and what counts as a finished result. The basic action–observation loop is familiar from ReAct. CoALA adds a useful way to think about memory and decisions. In my research system, that comes down to six jobs:

• Skills give the agent a research method, including when to use it and what a finished result must contain.
• Sub-agents give design, execution and submission separate tasks and contexts.
• Memory and context bring earlier experiments and economic knowledge into the next decision.
• Tools and code run the repeatable work and return states the agent can actually interpret.
• Verification and feedback turn measurements into a reason to revise, stop or move on.
• Authority and budgets decide which actions the agent may take, especially when they change the submitted book.

These are not six neat boxes in a diagram. A skill may launch a specialist; a hook may call code. What I care about is simpler: can I follow a signal from the economic idea to the measured result and the decision to submit it?

Why I put the harness ahead of model choice

The State of AI Report 2026 has a line I keep coming back to: “Same model, better harness = stronger agent.” In the coding study behind its example, GLM-5.1 solved 52.5% of tasks with a minimal harness and 65.5% with a fuller one. Context, tools, verification and recovery all changed together. There is no honest way to credit that gain to one trick.

I have not run the equivalent controlled comparison on my WorldQuant system, so I won’t pretend the coding result measures my own uplift. But it captures a problem I see every day: a capable model can reason about a signal and still get nowhere if the workflow gives it half a result or forgets the last experiment. The report also warns that stronger models can outgrow their harnesses. More checks are not automatically a better harness; better decisions are.

1. Skills and sub-agents: a claim before a formula

What I added to Claude Code

I started with Claude Code’s agent loop and tools, then added my own skills, specialist roles, code and hooks using its extension points. Tell a general agent to “find a good alpha” and it can produce one quickly. The harder questions are left hanging: which part of the portfolio needs help, what economic claim are we testing, and what result would make us change our minds?

My research-cycle skill puts those questions into the procedure. It tells the supervisor what evidence to fetch and what each stage owes the next one. Then a specialist receives a narrower brief and its own context. The skill carries the method; the sub-agent does a particular job. Reusable skills and explicit agent conversations have precedents in Voyager and AutoGen. I had to make that arrangement work for financial research.

A claim before a formula

Here is how one cycle works. The analyst looks at the submitted book and at what we already tried. The hypothesis specialist states an economic mechanism and explains what a new signal might add. Modeling turns the claim into candidate constructions; experiment runs them and brings back measurements. Only then is there a submission decision. The diagram shows these handoffs.

The hypothesis handoff normally contains no formula. I want an explanation of who might be slow to react to information, and why, before the agent reaches for familiar operators. If a construction nearly works, experiment can try a bounded settings change. If the idea needs another construction, modeling gets the actual measurement back, not a vague instruction to “improve it.”

There is a slower loop too. As the book and experiment history change, the analyst refreshes the opportunity map. We can reuse a good idea; we cannot keep judging it against a portfolio that existed several submissions ago.

2. Memory and context: what should the next run remember?

“The agent has memory” sounds nice. I care more about whether yesterday’s result changes what it tries today. I keep the current task, measured experiments, economic knowledge and conditional lessons distinct. A backtest is an observation; a lesson is an interpretation. If I mix the two, one local result can quietly turn into a rule the evidence never justified.

MemGPT distinguishes working context from stored memory, and Reflexion explores carrying evaluation into later attempts. For me the practical question is: which record is the agent looking at? A batch export says what happened in an experiment; a submission record says what entered the book. A correlation number also needs its date and the book state against which it was measured.

I found automatic closures that said little more than “ran.” That tells me an action happened, not what we learned. I want the next agent to see the condition, the measurement and the limit of the inference. In the alpha example at the end, you can see an earlier recipe make it into a later modeling task.

3. Tools and code: an accepted request is not a finished backtest

Here is a problem that sounds boring until it ruins a research decision. WorldQuant can accept a simulation request while the calculation is still running. The backtest can arrive before its correlation check; the time series may need another retrieval attempt. Accepted is not finished. If my tool reports “success” at the first response, the agent starts judging a result it does not yet have.

Claude Code runs the command; my code handles the sequence. It checks inputs, starts and polls the calculation, resolves platform checks, retrieves available series and saves them alongside the design. Pending work stays pending; a missing result stays missing. SWE-agent studied how much an agent’s tool interface can affect its performance in software tasks. I see the same issue when the agent has to wait for a real backtest rather than a tidy one-line response.

The measurements still need interpretation. Passing the ordinary checks says something about the backtest. It does not establish the economic story or show that the signal improves the book. The tool’s job is to show what is ready and what we still do not know.

4. Hooks and feedback: what does the result change?

A check matters only if it changes the next action. I use Claude Code hooks to record specialist dispatches, inspect some actions before they run, and prompt reflection after experiments. Session start and compaction events help carry unfinished work into another context. None of this makes a bad idea good. It does give the supervisor a chance to notice a missing prerequisite or a promising near-miss before moving on.

If a result narrowly misses a settings threshold, the experiment stage can try a small rescue. If the construction itself is the problem, the supervisor sends the measured value, the required limit and the attempts already made back to modeling. Now the next specialist has a specific result to beat, not just “try again.”

I also keep the agent’s account of its progress separate from the measurements. Agentic Harness Engineering makes this distinction explicit for tracers and verifiers. In my system, I want to see the platform checks and experiment records even when the agent sounds very sure of itself.

5. Permission: when can a signal enter the book?

I am happy to let agents explore ideas and run authorized research cycles. Submission is a different decision. It changes an append-only book, and the next candidate will be judged against that new book. Claude Code’s tool permissions alone cannot decide whether a particular signal belongs there.

Sometimes I approve named candidates. At other times I authorize a standing routine with a daily allowance. Either way, the permission must point back to my actual decision. The book changes after every submission, so the platform checks have to be fresh. If two workers can submit, they must share the same remaining allowance. In the case below, the planner used one of two available slots.

6. One alpha, from idea to submission

Competitor-network peak freshness

Let me take one submitted signal through the whole system. The idea was that investors might be slow to price a change in a company’s influence within its competitor network. PageRank gives one measure of that influence, but we were not simply buying the most central firms. The signal asks how recently each firm’s centrality reached a rolling peak, compares that freshness with its own recent history, then ranks it against industry peers. Higher relative freshness points toward the long side under industry neutralization. There is a subtlety: the clock can reset when an old maximum leaves the window, even without a new high.

The previous day’s research had left us a peak-timing recipe and input constraints. I saved them, and the supervisor included them in the modeling specialist’s brief with the registered hypothesis. The specialist returned 28 initial designs. An experiment specialist ran them through the simulation code; hooks recorded the dispatches, and the measurements stayed linked to each design. The memory actually changed the next run.

Some variants passed ordinary quality checks but looked too much like what was already in the book. That finding sent the work back for two more rounds, changing the timing horizons and group adjustments. Across all three rounds, 55 variants completed. We had predicted self-correlation of 0.42 for this hypothesis; the selected signal measured 0.6919 on the platform, just inside the 0.70 limit. Its in-sample Sharpe was 1.93 and fitness 1.54. The different input field had not made the return stream as different as we expected.

I had given standing permission for at most two submissions that day. Three candidates looked individually eligible, but they overlapped heavily with one another. The planner chose one, not three. The platform accepted it on September 21, 2026, and the submission receipt was checked and saved.

I like this example because you can see the whole chain: an earlier idea reached the next specialist, experiments changed the construction, the platform measurement checked our prediction, and permission narrowed the final choice. But I would not call it proof of out-of-sample performance. The backtest covers 2019–2023 in sample; I have no measured 2024 result for this signal. Fifty-five completed variants also bring selection risk. And after the 0.42 versus 0.6919 miss, I would check diversification against the actual book again before trusting the story about a new data channel.

What’s next

What should the system carry into tomorrow? The correlation miss, the reusable construction and the 55 tried variants are three different kinds of information. I want the next research decision to reflect that difference. That’s the subject of the next article: the loops that turn recorded results into new work.

If you are working in a more traditional way and you think you need support in building and designing such AI-native investment research systems, let’s talk about how we can collaborate together. I am helping financial institutions personally and within a consulting company I co-founded to adopt AI for many years, and will be glad to help you do the same. You can reach out to me on Linkedin, X, and email as well.

My AI Agent is in Top-0.01% of WorldQuant Challenge #5: Harness Engineering was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

A couple of months ago I started competing at the WorldQuant BRAIN — the largest global quantitative finance competition. At the moment of writing, my rank is 24 / 207115 and growing (~top 0.01% globally, you can check my account here). My philosophy is doing it part-time and in 100% AI-native way — absolutely zero manual work. This is a series for individuals and organizations doing quantitative finance research who want to adopt an AI-native approach:

1. Introduction
2. Alpha Knowledge Graph
3. Fighting Overfitting
4. Context Engineering (this article)
5. Harness Engineering
6. Loops Engineering
7. Open Source LLMs and Harness
8. Future of Quantitative Research

Before I created specialist agents, lifecycle checks or an autonomous research loop, I had a more basic problem: what exactly should the system be allowed to believe? I had collected four very different kinds of material.

1. First, there were the official WorldQuant BRAIN and tournament documents: how the expression language works, which operators and datasets exist, valid simulation settings, and so on.
2. Second, I added theory and research methods. Three books by Marcos López de Prado provided tools for thinking about backtest statistics, strategy risk, multiple testing, portfolio structure and falsification. Finding Alphas, by Igor Tulchinsky, added practical material about idea construction, turnover, correlation, overfitting, biases and robustness.
3. Third, I collected papers and deep research about economic anomalies, market participants and information flows. As I explained in the knowledge graph article, this material became a graph of axioms, theories, anomalies, mechanisms and implementation shapes.
4. Finally, the project started producing its own evidence: simulation responses, daily profit-and-loss series, portfolio correlations, robustness statistics, platform checks, experiment histories, operational events, failed constructions and lessons derived from them.

All four are context. They are not all truth. And they do not fit naturally into one prompt, one document format or even one database. That is where context engineering started for me — not with deciding how many tokens fit into a model, but with designing levels of proof and the data systems that preserve them.

Designing levels of proof

I use “proof” here in an engineering sense: what evidence is sufficient to support a particular claim, who owns that evidence, and what is allowed to overturn it. The first three levels of my architecture answer those questions.

L1 is static truth:

what the platform says the game is. It contains official platform and tournament material. If I ask which settings exist, what an operator means, what checks a candidate must clear or how the tournament score is calculated, the agent should read this layer. The local copy can become stale, but a surprising experiment cannot rewrite a rule.

L2 is dynamic truth:

what the system actually observed. A simulation result belongs here. So do a daily PnL series, the current portfolio, a platform check, an operational event and the latest leaderboard state. Every measurement needs an observation time. Rerunning an experiment creates another observation; it does not edit the previous one until history looks cleaner.

L3 is inductive bias:

what we currently believe is useful. This is where the books, papers, economic theories, specifications, knowledge-graph interpretations, recorded lessons and scoped falsifiers belong. They guide the next action, but they can be overturned.

This is not a simple ranking where L1 is always “better” than L2 and L2 is always “better” than L3. The levels answer different questions. When two sources answer the same factual question, I use the strongest direct source. An official rule beats a restatement in an agent prompt. A measured platform response beats my expectation from a book. But when the question is what to test next, rules and measurements are insufficient: the system needs L3 to make a reasoned bet.

One database can hold both a measurement (L2) and my interpretation of it (L3), so the level belongs to the claim, not to where it is stored. The minimum context record is therefore more than text. It needs a claim, epistemic level, source, owner, scope, observation or review time, uncertainty, stable identifier and the evidence that could update it. For dynamic facts, I increasingly want two data streams: when the event was true and when my system learned it. That distinction matters when APIs arrive late or historical data is backfilled.

Context is a data platform, not a folder

Once I adopted the proof hierarchy, a second problem became obvious: context is not one corpus. It is a small data platform. The first and largest component is the research ledger. On the workstation I inspected for this article, it is a transactional SQLite database. This is not just an experiment table. It is an evidence ledger with several families of data:

• Research intent: mechanisms, hypotheses, predictions and implementation lineage.
• Measurement: platform responses, in-sample statistics, checks, daily PnL and robustness metrics.
• Interpretation: attributions, lessons and scoped falsifiers.
• Governance: candidate decisions, approvals and submission outcomes.
• Materialized analysis: portfolio deltas, correlation screens, pattern caches and robustness summaries that can be regenerated from canonical inputs.

Writing this article exposed a gap: my own design doc says daily PnL was moved out to Parquet files, but the database still holds all of it. The doc is a belief (L3), the database is an observation (L2), and the observation wins.

The second component is the operational event store. It is a separate SQLite database. An event can carry a cycle, batch, candidate, hypothesis, agent and process identifier plus a versionable JSON payload. Why not keep this in the research ledger? Because the retention semantics are different. Research evidence is canonical and should survive. Debug logs, dispatch events, queue transitions and temporary interventions can grow quickly and may be compacted or expired. Separating them means an observability flood cannot bloat or lock the scientific record.

The third component is the knowledge graph: a transactional property graph. It uses a simple but effective physical model: one node table, one edge table and JSON attributes for type-specific evidence, loaded into an in-memory graph for traversal. The transactional store prevents the corruption and concurrency problems I had with a large YAML file; the in-memory graph keeps reasoning convenient.

Finally, the filesystem is still a data system. The repository contains immutable platform documents, books, papers and derived reports. These files should not all be “ingested into memory.” Each class has a different lifecycle:

• Platform documents are immutable source material.
• Specifications are version-controlled design priors.
• Reports are derivations with inputs and a regeneration method.
• Memory files are compact human-readable routes to reusable lessons.
• Queue files, caches and logs are disposable operational state.
• Model binaries, PnL exports and large snapshots are artifacts that belong in object storage, not Git.

The result is a multi-type context system even though most of it still runs on one laptop. SQLite owns transactions, the graph owns relationships, Parquet is the intended shape for large time series, Markdown owns reviewable knowledge, and transient files coordinate local processes.

Stable IDs are here to stitch all the context together

Mechanisms are the bridge between the two stores: the same mechanism appears in the graph as an economic idea with its sources, and in the ledger as a label on every hypothesis and candidate that tested it. Empirical summaries can be reconciled back onto the graph for navigation, but those summaries are mirrors. The experiment database remains their canonical home.

Because both stores share this ID, the system can answer much better questions:Which theories support this mechanism? Which hypotheses implemented it? Which experiments tested those hypotheses? What happened in each test? Which failure was implementation-specific, and which repeated across distinct constructions? Which graph branch has strong theoretical grounding but little empirical coverage?

A useful context packet can therefore be assembled as a join across systems: a small graph neighborhood, source provenance, current empirical rollups, recent scoped failures and the relevant platform constraints. The model does not need the entire graph or every PnL row. It needs the shortest evidence path that supports the decision.

There are several data contracts I would make stricter next:

• First, every cross-store object should carry an explicit JSON schema version
• Second, every derived metric should identify the code version, source snapshot and method version that produced it.
• Third, every experiment should have a content-addressed reproducibility manifest: code commit, environment digest, platform settings, hypothesis and implementation IDs, etc
• Fourth, reconciliation should be tested as data quality, not trusted as plumbing

In other words, the knowledge graph is context. The research ledger is context. The link between them is also context — and deserves the same engineering discipline as model code.

The harness gives each agent only what it needs

This is the boundary between context engineering and harness engineering:

• Context management owns L1–L3. It preserves sources, measurements, priors, provenance, timestamps, scope and stable identifiers
• L4 is the agentic harness. It decides which evidence to retrieve, how much may cross a stage boundary, which specialist receives it and which conclusions are legal

Below it are L5 source code, L6 local infrastructure and L7 distributed infrastructure. They can execute a bad decision perfectly; they do not decide whether an input was a rule, an observation or a belief.

My first version ignored this split: the main instruction file grew into a 627-line wiki that mixed tournament rules, live status, methods and yesterday’s lessons. I cut it down to a small router, and context now arrives in four ways: a short always-on layer, a startup packet with recent lessons and warnings, rules that load only when a task touches their area, and specialist roles and memories retrieved on demand.

Consider the request: “What should I try next?”

The harness first composes a preflight packet:

1. From L1 it gets the relevant expression, setting and scoring constraints.
2. From L2 it gets the current portfolio, recent measurements, search coverage and unresolved outcomes.
3. From L3 it gets the most relevant graph branch, theoretical provenance, scoped falsifiers, research specifications and empirical lessons.

Then, subagents go into play:

1. The hypothesis specialist uses this packet to propose several economic angles without hiding weak reasoning behind formulas.
2. After one angle is selected, the modeling specialist receives the registered hypothesis and constructs genuinely different implementations.
3. The experiment specialist receives that implementation tree plus the relevant platform constraints and records measured outcomes.

Then, every stage writes to a different part of the data plane:

• Raw platform results and normalized measurements go to L2 in the research ledger.
• Run starts, tool calls, retries and coordination events go to the operational store.
• A lesson or scoped falsifier derived from the result goes to L3.
• Graph rollups are reconciled from the ledger rather than edited independently.
• Large PnL series, plots and fitted models become referenced artifacts.

How I would evolve this data architecture further

The current local architecture is more capable than it may look. SQLite is an excellent default for a single researcher: transactions are real, backups are simple, local queries are fast and there is almost no operational burden. But the limits are becoming visible. A multi-gigabyte research database is carrying tens of millions of time-series rows. Several workers can contend for local writes. Runtime events are growing. Artifacts and database snapshots are tied to one machine. Reproducing a result elsewhere requires more implicit state than I want. I would evolve it in five phases:

• Phase 0: reconcile design with reality. Before adding a cloud, I would finish or deliberately roll back the incomplete PnL migration, then make the state audit compare documentation with live storage automatically.
• Phase 1: externalize heavy artifacts. Daily PnL, plots, model files, large reports and database snapshots belong in S3-compatible object storage. PnL should be partitioned into compressed Parquet with immutable manifests, checksums and lifecycle rules.
• Phase 2: move metadata only when concurrency requires it. When remote workers, multiple machines or collaborators need to write concurrently, I would move the research ledger and operational store to managed PostgreSQL, probably as separate databases or at least separate schemas with different retention policies.
• Phase 3: add MLOps as a projection, not as the source of truth. Weights & Biases or MLflow would be useful, but I would not ask either product to replace the research ledger or knowledge graph.
• Phase 4: separate orchestration from state. Research workers should run in containers and receive immutable job manifests. A queue or workflow engine should assign idempotent jobs, leases, retries and timeouts.
• Phase 5: add richer analytical infrastructure only after the workload earns it. A lakehouse table format, a dedicated graph database or streaming platform may eventually help. I would adopt them only when concrete queries become painful: cross-experiment scans over billions of observations, many concurrent graph traversals, schema evolution across large Parquet collections or low-latency event consumers.

I can’t claim this architecture raised performance by a precise percentage, because I never isolated its effect. What it does is let the system disagree with itself honestly: a paper suggests, a simulation contradicts, a falsifier narrows, and the tournament rule stays the rule.

What’s next?

This article was about what the agent should know. The next one, Harness Engineering, is about what it is allowed to do. I’ll show how the rules from this article became code the agent can’t skip. When the agent kept ignoring an instruction, I replaced it with a check that blocks the action. And nothing permanent, like submitting an alpha, happens without my explicit yes.

If you are working in a more traditional way and you think you need support in building and designing such AI-native investment research systems, let’s talk about how we can collaborate together. I am helping financial institutions personally and within a consulting company I co-founded to adopt AI for many years, and will be glad to help you do the same. You can reach out to me on Linkedin, X, and email as well.

My AI Agent is in ~Top-0.01% of WorldQuant Challenge #4: Context Engineering was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

A couple of months ago I started competing at the WorldQuant BRAIN — the largest global quantitative finance competition. At the moment of writing, my rank is 78 / 223470 and growing (top ~0.04% globally, you can check my account here). My philosophy is doing it part-time and in 100% AI-native way — absolutely zero manual work. This is a new series of articles for individuals and organizations doing quantitative finance research who want to adopt AI-native approach:

1. Introduction
2. Alpha Knowledge Graph
3. Fighting Overfitting (this article)
4. Context Engineering
5. Harness Engineering
6. Loops Engineering
7. Open Source LLMs and Harness
8. Future of Quantitative Research

Previous 2 articles were introduction to the challenge and defining graph-based reasoning structure for our AI agent. Now we can proceed to the practical part of any investment strategy development — ensuring that it will perform out of sample as good as in sample, taking into account WorldQuant BRAIN platform constraints: no access to the raw data to do any kind of cross-validation and results validation limited only to the 5 years in-sample period. Of course, in the real world where you have access to raw data, I recommend you to design a solid cross-validation and simulation pipeline, as I described previously in this blog: “How to finally believe your backtests” series (part 1, part 2, part 3), “feature importance as a backtest tool” (link) and some other earlier work.

Defence 1: economic grounding as an out-of-sample prior

Even before WorldQuant, I learned it the hard way: a pattern found by random search has no reason to persist. A pattern that exists because a specific participant is structurally unable to arbitrage it away has a mechanism for persisting and surviving out of sample. So before any simulation runs, every hypothesis must register an economic claim with four key elements: the economic anomaly, the information channel, why it’s mispriced, and why nobody else has taken it. This is the hard gate for my agents to proceed or not with the hypothesis.

Does it work? The honest test isn’t performance — it’s whether the work survives at all. Of 20,542 alphas built on a written economic claim, 209 made it into the live book. Of 5,199 built on a mechanism the system clustered automatically with no claim recorded, 28 did. That’s 1.02% against 0.54% — grounded ideas reach the book at roughly twice the rate.

The figure above shows three of each. The rejections are instructive: two died on LOW_SUB_UNIVERSE_SHARPE — the edge existed only in the liquid core — and one on LOW_FITNESS, carrying Sharpe 2.19 but fitness 1.08 at 38% turnover. The return didn't survive its own trading. Grounding doesn’t make an alpha good. It changes the odds that the work survives.

Defence 2: refusing to search hyperparameters

The most seductive kind of overfitting isn’t just a bad idea. It’s a bad idea, tried 200 ways with different lookbacks and fields, until one of them happens to look great. So the 2nd rule is a strict one too: hypotheses have to come from first principles, not from noticing a pattern in past results. At most two decay variants per hypothesis.

Let’s look at a real example. Take one alpha shape — the same formula skeleton, with different fields and lookback windows plugged in. That’s what a sweep is: one idea, tried 20–30 ways. Now ask the obvious question: which variant is “the best”? The natural move is to sort by Sharpe and take the top one. I checked whether that move is safe, on two real sweeps:

• In the first, it was. The variant with the best Sharpe was also the variant with the best Fitness — BRAIN’s other main quality measure, roughly “is this return earned cleanly, or just by trading constantly.” Same alpha, either way you sort. No ambiguity.
• In the second, it wasn’t. The best-Sharpe variant had a Sharpe of 2.46 — and a fitness of 0.93. BRAIN’s fitness floor is 1.0. Sort by Sharpe, ship the “winner,” and BRAIN rejects it. The variant that was actually best — fitness 2.01, at a quarter of the turnover — was a different alpha, sitting further down the Sharpe ranking where nobody would have looked.

I ran this check across 25 sweep families. The Sharpe-winner and the fitness-winner were the same alpha in only 7 of them. Eighteen times out of twenty-five, “the best variant in the sweep” wasn’t a fact about the alphas — it was a fact about which column you happened to sort by.

And you can’t tell in advance which kind of family you’re in. Every extra variant you generate costs you something for certain — it inflates the trial count that the Deflated Sharpe Ratio (see below) then penalizes you for — in exchange for a benefit you can’t verify until it’s too late to matter.

Defence 3: De Prado statistics as stability gates

The machinery, straight out of López de Prado’s Advances in Financial Machine Learning:

• Probabilistic Sharpe Ratio computes P(true SR > SR*) from the Sharpe ratio’s asymptotic distribution, corrected for the return series’ actual skewness and kurtosis rather than assuming Gaussian returns. A backtest with fat tails or negative skew needs a much higher raw Sharpe to hit the same PSR as a well-behaved one.
• Deflated Sharpe Ratio — the same PSR calculation, but the benchmark SR* is replaced with the expected maximum Sharpe ratio you’d get from N independent trials under the null of zero skill, a function of N and the variance of Sharpe ratios across those trials. The more variants tested, the higher the bar a real signal has to clear to be distinguished from the best of pure noise.
• Probability of Backtest Overfitting — Combinatorially Symmetric Cross-Validation splits the PnL history into S groups, forms every train/test combination, and for each one checks whether the in-sample-best configuration still ranks above median out-of-sample. PBO is the fraction of splits where it doesn’t — an overfit strategy ranks well in-sample and effectively at random out-of-sample.
• P[failure] — derived from the strategy’s hit rate and its average-win-to-average-loss ratio, this is the probability that a betting process with those parameters produces a below-target Sharpe over a finite run. It’s a risk-of-ruin estimate, independent of whichever specific path the backtest happened to realize.
• Šidák-corrected dominance threshold, P5 — draws many synthetic return paths for the alpha and takes the 5th-percentile Sharpe across them, then compares it against a threshold inflated by the Šidák correction, 1 − (1−α)^(1/m), which controls the family-wise error rate across m simultaneous alpha comparisons instead of testing each one at the nominal α.

Three are hard gates — PSR ≥ 0.90, P5 ≥ 1.25, P_fail < 0.20 — and the submission gate fails closed: a candidate with no computed de Prado cannot be cleared. Missing evidence counts as failing evidence.

Of 4,778 alphas with these statistics computed, 801 clear all three. 16.8% — which I consider as not bad rate at all, very likely thanks to strong economic grounding of every idea. Five of every six things that looked good enough to measure were not stable enough to keep.

The illustration above clearly illustrates the difference between traditional in-sample BRAIN metrics and De Prado metrics: all six alphas sit at Sharpe 2.05–2.30 and fitness 1.45–2.22 — very similar to each other. But De Prado statistics disagree about three of them. Note the blocked alpha in the first row: fitness 2.22, the highest of all six. The platform likes it more than anything in the passing column, and its P5 is 1.17.

Defence 4: path-shape checks

De Prado’s statistics all treat returns as a distribution . But a stream that earned steadily and one that earned everything in year one and nothing since can have the identical distribution and pass the identical gates from Defence 3. One more check looks at the path instead of the distribution, because that’s the only way to catch an edge that already died before the window ended:

• Persistence. Instead of one Sharpe averaged over five years, compute the Sharpe on just the recent stretch and compare it to how the alpha performed earlier. Still earning at the same rate → ALIVE. Fading → SOFTENING. Recent stretch flat or negative while the five-year average still looks fine → DECAYING: an edge that's already dead, wearing a healthy headline number.

That recent-stretch number is what the figure above shows, labeled “Sharpe, last stretch,” sitting right beside the five-year Sharpe in the metrics next to it. All six alphas clear Sharpe 3.0 on the five-year number and clear every gate from Defence 3 — the only thing separating them is that one recency figure.

The metric that would normally make it my best alpha is the one hiding that it stopped working. And it isn’t one unlucky alpha — it’s structural. Below Sharpe 2.5, under 1% of my alphas are decaying. Above Sharpe 3.5, 44% are. Of the 16 alphas in my corpus with in-sample Sharpe ≥ 3.5, not one is still rated ALIVE. A five-year Sharpe is an average — by construction it cannot tell you the edge died in year four. Only a check that looks at the path can.

What’s next?

We now have a way to generate economically-grounded ideas using knowledge graphs and a 4 defence mechanisms (for WorldQuant BRAIN, in real investment world we have many more!) to judge whether they’ll survive. This gives us strong and economically grounded set of inputs for our agents and mathematically rigorous mechanism to judge the outputs. Next articles will be about designing the agent in between them: how it consumes context, how it uses tools as a harness, how to make it work in the loops and many more modern agentic engineering items.

If you are working in a more traditional way and you think you need support in building and designing such AI-native investment research systems, let’s talk about how we can collaborate together. I am helping financial institutions personally and within a consulting company I co-founded to adopt AI for many years, and will be glad to help you get it done with you too. You can reach out to me on Linkedin, X, and email as well.

My AI Agent is in Top-0.04% of WorldQuant Challenge #3: Fighting Overfitting was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

A couple of months ago I started competing at the WorldQuant BRAIN — the largest global quantitative finance competition. At the moment of writing, my rank is 97 / 246073 and growing (top ~0.04% globally, you can check my account here). My philosophy is doing it part-time and in 100% AI-native way — absolutely zero manual work. This is a new series of articles for individuals and organizations doing quantitative finance research who want to adopt AI-native approach:

1. Introduction
2. Alpha Knowledge Graph (this article)
3. Fighting Overfitting
4. Context Engineering
5. Harness Engineering
6. Loops Engineering
7. Open Source LLMs and Harness
8. Future of Quantitative Research

In this article we will focus on one of the central ideas of my approach — using knowledge graphs to make agents actually reason in economic terms and not just hallucinate things.

Reasoning “in economic terms”

Naive “vibe research” approach

What happens if you just open Claude Code with the state-of-the-art model like Fable 5 and just ask it to generate an “alpha”? This was of course my first approach, and this immediately led to multiple issues:

• “monoculture trap” and fixation on alphas that overfit — even LLMs like Claude Opus will struggle to generate truly different alphas and highly likely the ideas will not perform well out of sample
• very naive fixes and suggestions — highly likely LLM won’t go beyond ideas like “revert”, and linear additions / removals of new fields from new datasets without deep reasoning
• even harder will be to push raw LLM to generate deeply decorrelated ideas from your existing portfolio unless you will add harness elements for your book analysis

First principles of alpha generation

So to fix these and many other issues, I had to rethink the whole concept of “alpha generation” from the first principles. In general, we can “beat the market” (i.e. get money from other market participants) in 3 ways:

• be faster (not available to us in this competition, we’re not going to do any HFT play)
• have different / better information (for example most of people analyse only stock prices, but we can also look into fundamentals, sentiment data, etc — available to us thanks to WorldQuant datasets)
• model / process information better (i.e. use non-linear modeling instead of linear models — partially available to us thanks to WorldQuant operators)

So what we need to do, is to encode who are the other market participants, what data they might have and how they might process it and build hypotheses how we can use more / better data and model it better to extract money from other data participants. So, essentially, alpha appears when we:

1. know other participants (I looked into 26 types of them ~ nodes in the graph) like HFT firms, sell side analysts, corporate insiders, options market makers, volatility sellers, etc and how they act
2. understand how we can act against them with new data and new modelling, for example understand how information diffuses differently, or what heterogeneous beliefs, biases are present, etc (~ edges in the graph)

This thinking map should be encoded as a knowledge graph as a structured map of who believes what and why they’d be wrong.

Knowledge graph architecture

To facilitate reasoning from the first principles, I introduced a hierarchy of concepts in a knowledge graph:

1. Axioms level (7 axioms): the assumptions the textbook market is supposed to run on — rationality of investors, no arbitrage, risk aversion, market clearing, etc. What makes them useful isn’t the list, it’s that each axiom carries two kinds of edges: grounds, pointing at the theories that assume it, and violated_by, pointing at the places it demonstrably breaks. Alpha lives in that second set of edges.
2. Grand theories level (7 theories): APT, CAPM, EMH, behavioral finance and some others that arise from the axioms. The graph doesn’t store these as labels but as assumption-sets — EMH stands on all five of rationality, rational expectations, no arbitrage, market clearing and homogeneous expectations, while behavioral finance is defined by the three it violates. That makes “what breaks if investors aren’t rational?” a graph traversal instead of an essay.
3. Theory extensions level (39 extensions): information diffusion, limits to arbitrage, market infrastructure, factor crowding theory, etc. Each extension hangs off the specific axiom it breaks — limits to arbitrage approximately violates no-arbitrage, microstructure violates market clearing, etc. This level is the actual inventory of tradeable market failures: not “markets are inefficient”, but which assumption fails, and in which documented way.
4. Anomalies level (94 families): theories and their extensions represent 26 market participants using 20 WorldQuant datasets that are connected with different relational edges. This is the last purely-literature level in the graph: everything above it is what academia believes, everything below it is what I actually built and measured.
5. Mechanisms level (3,432 mechanisms): an anomaly family tells you that something is mispriced — post-earnings drift exists, revisions underreact. It doesn’t tell you how to take money out of it. A mechanism is one specific, falsifiable claim about harvesting one anomaly with the fields actually available on the platform
6. Operator skeletons level (5,323 skeletons): a skeleton is an alpha with all of its content removed — the abstract syntax tree normalized down to a shape, with placeholders where the data goes: <f> for a field, <i> for a lookback, <neut> for a grouping. So group_rank(ts_delta(<f>, <i>), <neut>) is a shape realized by 599 of my alphas at a best Sharpe of 2.10.

To build the graph, I ran a deep research pass to find what the literature already had, posing the same question in parallel to Claude, Gemini and Parallel and reconciling the three answers against each other. Two findings shaped everything:

1. No published theory-level knowledge graph of asset pricing exists. The finance ontologies that do exist (FIBO, OntoFINE) model instruments, risk and value — entities, not theories — and the LLM-finance graph papers use graphs as feature extractors rather than as representations of what the field believes. So the top four levels were greenfield.
2. But, Jensen–Kelly–Pedersen’s Is There a Replication Crisis in Finance? (2023) clusters 153 firm characteristics into 13 themes, replicated in-sample and validated out-of-sample across 93 countries, with a live data portal — and it maps almost directly onto the anomaly layer. That became the backbone for my graph, supplemented by some other papers.

Knowledge graph use and updates

Inference via graph

The graph isn’t a reference the agent consults if it feels like it — it’s a mandatory rule for every reasoning pass. A hypothesis that can’t land on a node is rejected before we even start any WorldQuant simulation. Every proposal has to arrive carrying:

• the mechanism node it extends, and that node’s current empirical status
• which axiom is violated, through which extension — not “markets are inefficient” but market clearing fails in this specific documented way
• the counterparty — who’s on the other side, and why they structurally can’t stop

The traversal is the same every time: an axiom is violated by an extension, the extension manifests as an anomaly family, the family is implemented as a mechanism, the mechanism is realized as a shape. For example:

• Short-term reversal: market clearing is approximately violated by microstructure → whoever absorbed last week’s forced selling provided liquidity and deserves paying for it → counterparty, market makers.
• Variance risk premium: options pricing is extended by the Bollerslev–Carr–Wu branch → implied vol exceeds realized because someone’s paying for crash insurance → counterparty, volatility sellers.

The graph updates itself

Mechanism nodes carry their verdict in the same row as their claim — status, graveyard flag, and a re-open condition written in advance. An attribution engine walks results back up, so status is derived from the results of simulation on WorldQuant, not asserted by whoever wrote the hypothesis. Two real updates:

• Promoted. Betting-against-beta entered as a proposed claim from the literature. 435 alphas eventually attached to it, enough cleared the bar, status changed to confirmed.
• Buried. The net-operating-assets over-investment anomaly failed across three genuinely distinct constructions and went to the graveyard, with its re-open condition recorded at burial: a new construction, or corroboration from two independent factor models.

Alongside sit 425 falsifiers — this was tried, here’s how it failed — each scoped, and the scope guarded: the broader the claim, the more independent failures it needs.

Technical implementation

Simple SQLite with two tables: nodes(id, kind, label, attrs) and edges(src, dst, etype, attrs), with the edge primary key being the whole triple — so one pair can be joined by several relation types at once, and re-asserting a known fact doesn't duplicate it. Filtered columns are indexed columns; everything else lives in a JSON blob so a new field isn't a migration. V1 was YAML files, until one interrupted truncate-and-rewrite left me holding half an ontology.

Traversal materialises into a NetworkX MultiDiGraph on demand — SQL is the truth, the graph is a disposable read model. Three things come free: nx.ancestors turns "is this grounded?" into a set-membership test; 1 − Jaccard overlap of two mechanisms' ancestors gives a decorrelation prior before simulating; and anomaly families with no mechanism child are the frontier, found by topology rather than by asking a model what's missing — ask a model that, and it points at wherever it just was.

A note: this is one ontology, not the ontology

I organised the graph around economic theory because my binding constraint was out-of-sample performance on WorldQuant, and theory is the only thing that tells you why an edge should still exist next year. But that choice is not obvious, and it’s not the only one. Four genuinely different architectures I could have built instead:

• Participant-first (a flow-of-funds graph). Nodes are who trades and what forces them to —index reconstitutions, redemption cycles, margin rules, tax-loss windows, rating triggers. Edges are forced to sell by, must buy at, cannot hold. Alpha is anticipating a trade that has to happen regardless of price.
• Information-supply-chain. Nodes are facts about the world and everything that stands between a fact and its price: who records it, with what lag, who revises it, who proxies it. Edges are is measured by, lags, is revised by. Alpha is the gap between when something becomes true and when it becomes priced.
• The economy itself. Nodes are firms; edges are supply relationships, shared customers, competitors, common ownership. Alpha is information propagating along that network faster than the market propagates it.
• Pure geometry, no economics. Nodes are functional forms and transformations; edges are is a special case of, spans the same subspace as. Alpha is an unoccupied direction in a well-defined function space. This is the factor-zoo view, and note that it’s my bottom layer promoted to be the backbone of a new architecture.

The thing to understand is that whatever you choose as the spine, you get cheap reasoning along it and expensive reasoning across it. My theory-first graph makes “why should this survive OOS?” a one-call traversal — and makes “who is on the other side of this trade today?” an expensive lookup bolted on the side. A flow-first graph would be exactly the opposite. The ontology isn’t a description of the domain; it’s a decision about which questions you want to be able to ask cheaply.

What’s next?

In this article we focused on making an LLM thinking structurally to avoid so-called “AI slop” generation which in the investing world is very dangerous. In the next article, we will review another side of overfitting, which is related more to time series analysis of generated alphas. After that, having a strong reasoning framework to generate ideas amd strong analysis framework to judge them, we will proceed to agent design articles.

If you are working in a more traditional way and you think you need support in building and designing such AI-native investment research systems, let’s talk about how we can collaborate together. I am helping financial institutions personally and within a consulting company I co-founded to adopt AI for many years, and will be glad to help you get it done with you too. You can reach out to me on Linkedin, X, and email as well.

My AI Agent is in Top-0.04% of WorldQuant Challenge #2: Alpha Knowledge Graph was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

A couple of months ago I started competing at the WorldQuant BRAIN — the largest global quantitative finance competition. At the moment of writing, my rank is 103 / 258835 and growing (top ~0.04% globally, you can check my account here). My philosophy is doing it part-time and in 100% AI-native way — absolutely zero manual work. This is a new series of articles for individuals and organizations doing quantitative finance research who want to adopt AI-native approach:

1. Introduction (this article)
2. Alpha Knowledge Graph
3. Fighting Overfitting
4. Context Engineering
5. Harness Engineering
6. Loops Engineering
7. Open Source LLMs and Harness
8. Future of Quantitative Research

What is a WorldQuant BRAIN platform?

WorldQuant BRAIN is a platform that allows participants to create investment strategies (what they call alphas) across multiple universes, multiple data sets, data fields, and possible combinations on the platform. As a participant, you need to think of an idea, check the datasets, pick the right fields and combine them with the right mathematical operators.

It is later judged by the platform 1) individually by Sharpe ratio, returns, turnover, drawdowns, and 2) how it impacts your existing set of alphas that you already submitted, so you have to think in terms of the portfolio, correlation between the alphas you create and the residual impact is also very important, and of course everything is against other participants who are doing it in their own way.

What makes work quite easy is that they provide platform datasets and an FEL (fast expression language) to model interactions between different fields in the datasets. You do need to deal with data procurement and cleaning. You can participate even without any technical knowledge.

What is hard, especially if you think about the real-world trading, is that the out-of-sample is not visible and is judged only on the additional competitions like the IQC separately. Also, BRAIN doesn’t allow access to the underlying raw data, so you cannot do traditional data analysis. You should think more in terms of economic theories, delegating data analysis to the WorldQuant platform, which is a trade off to accept.

I am not diving into the details of WorldQuant BRAIN platform on purpose here, because they have amazing tutorials on the platform when you register and excellent in-depth YouTube series as well. In this series we will focus more on the agentic systems design, assuming data and backtesting routines already being developed and deployed in your team.

My AI-native research philosophy

WorldQuant’s interest in making this platform accessible to the public is in scouting talented individuals and researchers. That’s why the consultant program exists, and people get rewarded for creating good investment ideas. The current educational system at the platform is focused on manual competition. Of course, some people have started using large language models (LLMs) in the recent years. A couple of interesting open-source projects and research articles demonstrate how large language models can develop algebraic alphas.

The AI-native and AI agent approach is rarer, but based on my conversations with some of the people at the IQC, it’s becoming more and more popular. I chose the AI agent path because I am building a generalizable financial research machine, not just a WorldQuant bot. I find their algebraic alpha system extremely interesting, but my goal is to build a research agent system that will work with other data and on other platforms as well.

It’s still a debatable question: where is human judgment placed in AI agent system design? Many people find human judgment crucial, but I’m taking a bit of a maximalist stance. I have to confess that I personally didn’t dive into the BRAIN system at all. I delegated to my AI agent(s) absolutely everything about learning the platform, finding strategies, and some elements of self-improvement as well. As you’ll see later, I gave the system maximum autonomy for alpha generation and execution, running multi-hour sessions without my involvement at all. Where I see human involvement is:

• Irreversible decision-making, which in the brain case is the alpha submission. In the real-world case, it is the investment decision and execution.
• System design. A lot of my judgment and knowledge went into the system design, but not into individual investment strategy generation.

High-level architecture review

The system designed this way allowed me to test ~2000 hypotheses with ~30'000 simulations and learn ~12'000 lessons about the USA market reflected in the knowledge graph. 200+ alphas are submitted on the platform, with average Sharpe ratio ~1.8, ~10.5% turnover and ~5.5% max drawdown. My best alphas have 3+ Sharpe ratio. Full portfolio performance (at the IQC competition): 4.91 Sharpe, 8.94% annualized returns, 2.5% max drawdown, 9.1% turnover (5 years of in-sample data). All of this just over a couple of months of part-time work. Here are the core elements of the agentic system I’ve built:

1. Static context

First of all, we need to ground an agent in the tournament rules, which are saved as .md files in the platform documentation: data set documentation, operator documentation, FAQs, the forum data, etc. This is the immutable layer that is never edited.

2. Dynamic context

There is a context that is changing depending on my behavior and other participants’ behavior. There are databases for: all the simulations and experiments that I do, submissions that they do, the pipeline state, memory, constantly updated information about the tournament and its participants so I understand my position and the stage of the competition.

3. Knowledge graph

The know-how I am bringing to this competition is the construction of a knowledge graph. I will dive into this in the next articles. In a nutshell, the idea is that they need to guide agents to reason in economic terms and search for sources of alphas. No single prompt skill or even a multi-agent architecture can embed this structure into reasoning. I have collected multiple academic papers describing many different economic theories and constructed a knowledge graph that my agent uses every time it creates a new idea, all the time, it is economically grounded in it.

4. AI agent harness

The raw LLM is almost destined to fail and not deliver the results expected for such a complex job as investment research. That’s why, around the Claude Code, I have built a rich ecosystem of agents with multiple skills, added many hooks for protecting key workflow decisions, and wrapped it all in self-improving loops that don’t stop after finding one idea that doesn’t work. Instead, they push until success is achieved. Typically, the definition of success is a new alpha that is better than my previous alphas, de-correlated, genuinely improves my portfolio, and works well against my competitors.

5. Code

100% of the code was designed, written and tested by Claude Code. I used openspec to define the functionalities I needed, yet quite honestly, I even didn’t do manual code review. Code included but wasn’t limited to: portfolio analysis, knowledge graph routines, CLI around WorldQuant API, mathematical methods for alpha quality assessment and overfitting analysis.

6, 7. Local and cloud infrastructure

This is the part we will completely ignore in this series. Too much just depends on WorldQuant and how they organize their data and backtesting. And it’s just enough to know that some things I run locally, and some in the cloud :)

How you can use this series of articles

If you read my previous blog articles on Medium, you could have noticed that I’m always trying to accompany my work with open-source code. In recent years, I started to find it really unnecessary because AI is getting much better at generating code, and what you really need is a good specification (i.e. context) and good harness for your agentic coding system.

In this series of articles, I want to share with you exactly that. If you are an expert already on the frontier of quantitative finance research, it is very likely that it will not be hard for you to take these articles and work with the AI system of your choice to use it as a specification and upgrade your system. And I am absolutely happy with that. I would love to hear examples of people and teams using my work in such an AI-native way to improve their investment research.

If you are working in a more traditional way and you think you need support in building and designing such AI-native investment research systems, let’s talk about how we can collaborate together. I am helping financial institutions personally and within a consulting company I co-founded to adopt AI for many years, and will be glad to help you get it done with you too. You can reach out to me on Linkedin, X, and email as well.

My AI Agent is in Top-0.04% of WorldQuant Challenge #1: Introduction was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Last week I was running an executive workshop for CTOs of companies in a single PE group. One of the questions that I’ve been asked there and at other events is “what is the key critical reading list” for my technical staff team? The speed of changes doesn’t allow me to give you a single evergreen list, but here is my answer for May 2026.

Chief Technology Officers / VPs of Engineering

CTOs need to understand the brutal truth about ROI of AI integration. Most of the studies show that it’s not about augmenting engineers with copilots, but building secure autonomous workflows or agents and remove humans from the loop and your work as CTOs is to build governance around it.

• Read three things together that triangulate the autonomy thesis: (a) Anthropic 2026 Agentic Coding Trends Report, (b) McKinsey “AI revolution in software development”, DX Q4 2025 AI-Assisted Engineering Impact Report.
• Adopt the DX AI Measurement Framework — but measure the autonomy metrics (% of merged PRs agent-authored, hours of human review eliminated, end-to-end task cycle time), not just utilization.
• Pick one vendor governance posture for autonomous fleet operations: GitHub Agent HQ + AGENTS.md, AWS Kiro spec-driven, or Anthropic Claude Code + custom MCP. Running multiple agent platforms in parallel beyond a 6-month pilot is expensive and dilutes the autonomy curve.
• Threshold to escalate / re-evaluate: if after 6 months <5% of merged PRs are agent-authored, the bottleneck isn’t the tool — it’s the absence of specs, harness, or review structure that lets agents work autonomously. Fix those before buying more capacity. Read DX — AI productivity gains: More modest than expected (read as a diagnostic — modest gains = stuck in assistant mode).

Tech Leads / Architects

Tech Leads need to design company harness for agents orchestrator. I often hear opinions from experiences SWEs that “OpenAI Codex is better for X, Cursor is better for Y and Anthropic Claude Code is better for Z”. It seems like this to you only because you haven’t built a proprietary harness around it. You want to have a set of skills, specs, subagents etc that ANY tool will give you more or less the same result, ideally an equal result.

The job is no longer reviewing AI-generated code one PR at a time — it’s designing the system that lets many agents run safely in parallel. Master these five primitives:

1. Spec-driven development — the human’s highest-leverage activity becomes writing precise specs agents execute against. I personally recommend starting with OpenSpec by Fission-AI.
2. Memory management — agents without persistent task memory forget across sessions and across the team; this is the biggest unlock once you trust the model. Claude Code has solid native memory system, but Beads is worth giving a try too.
3. AGENTS.md — the de-facto open standard for repo-level agent context (adopted by GitHub, OpenAI, Anthropic, Cursor, Sourcegraph, Jules, Factory).
4. Background agents running asynchronously — the dominant 2026 workflow; you delegate, they ship PRs. Check out Cursor’s Background Agents docs and Anthropic’s Claude Code on the web.
5. Context engineering — the engineering discipline of feeding agents the right code, docs, static-analysis signals, and tests at the right time. I recommend reading Thoughtworks — Context engineering: Tackling legacy systems with generative AI.

Software Engineers

Engineers have to master foundations and one single tool very well. I believe first principles thinking is even more important than before and learning new frameworks or new tools is a futile idea for most of the people doing hands-on work. Master one agentic tool. Master CS foundations. Master your domain.

Pick one platform and master it. Anthropic has the strongest ecosystem of free courses to understand what’s actually inside AI agents — useful both as a coding engineer and as a stepping stone into AI engineering. I personally focus on Claude Code as a main tool and I adopted it at Neurons Lab as a CTO, so I recommend completing these four Anthropic Skilljar courses in order:

1. Introduction to Agent Skills — https://anthropic.skilljar.com/introduction-to-agent-skills
2. Building with the Claude API — https://anthropic.skilljar.com/claude-with-the-anthropic-api
3. Introduction to Model Context Protocol — https://anthropic.skilljar.com/introduction-to-model-context-protocol
4. Claude Code in Action — https://anthropic.skilljar.com/claude-code-in-action

Of course, of all the elements I describe here, they work only together. If your technical team is measured in dozens of people, all the elements have to be very well aligned. During the workshop with the private equity group, I could see the symptoms of misalignment on what to measure, zoo of different tools and no strategy on how to educate individual contributors to see real gains for your company. All can be fixed, but it starts with education and alignment.

Let’s chat if you’re working on building Agentic SDLC at your company as well — will be happy to help!

Agentic SDLC: Strategic Reading List for CTOs, Tech Leads and Engineers in H1 2026 was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

A quick announcement and I hope you’ll enjoy the materials!

This is a very brief announcement for people who follow me for a while.

You know that I started this blog about AI a long time ago when LLMs and LLM agents simply didn’t exist.

You know that I was among the first practitioners sharing how to build RAG systems and vibe code startups. Then, I was the first one writing about building LLM agents and why they’re more powerful than the RAG systems. Then I shared my philosophy for building personalized AI assistants and shared a couple of specific implementations for health and personal productivity.

The technology is changing so fast, so the only way you can keep up is grounding yourself with first principles thinking and always practicing to stay on the top of the things.

Having this in mind, together with Igor and Ligency we decided to create a practical Udemy course for building AI Agents with Claude Cowork and Openwork as a free, open-source alternative. Not chasing the most hyped tool or buzzword. But building strong foundations so you can master any tool and any hype wave.

You can find the course by the link below with 40% discount promo code active for ~30 days after publishing this article:

Practical Intro to AI Agents & Agentic AI with Claude Cowork

Enjoy and see you soon!

Hello everyone! This is a continuation of my new series related to a broad topic of agentizing personal life. First article was related to management of my health and wellness. Before that I wrote one article on the overall philosophy and architecture of modern AI agents and a bit more technical deep dive there as well that provides useful background if you want to dive deeper into modern AI agents.

Today, I want to share with you a more work-related setup of how I have built a personal AI Chief of Staff that helps me dozens and hundreds inputs daily and frees at least a couple of hours every day. I chose Claude Code to be my main workhorse platform, but the very same setup can be easily replicated in user-friendly Claude Cowork or more sophisticated personal assistant setup like OpenClaw. As always, I open-source the skills in my repository.

My regular review flow

When I do this work myself, I typically do it on the daily, weekly, and monthly basis before I do any focused work or have any meetings. You can find my freely available Notion templates for weekly and monthly reviews here.

1. Housekeeping: read and clean up my emails, my social media and messengers, my Slack and, finally, my Apple Notes. This prepares to do any daily work without thinking that I forgot anything. GTD in action.
2. Briefing and planning: analyze my Calendar events and time spent in Rize, my Clay (Mesh) events, HubSpot deals and my Notion tasks backlog. This is about setting up the stage for my work with needed context.
3. Drafting and acting: plan the tasks in my Notion, setting up priorities, answering and sending key messages, and, finally, act on my tasks.

Data sources, MCPs and Claude skills

Work environments

My main environments where a lot of work that I want to automate happens:

1. Communications — Gmail, Slack, Socials and Messengers (from Linkedin to WhatsApp)
2. People and Scheduling — Calendar, Rize, Clay (recently became Mesh), HubSpot
3. Information — Notion, Obsidian, Apple Notes, Google Drive, Remarkable

Of course, there is a lot of other software specific for focus tasks, i.e. Github for software engineering, Ashby for hiring and others. However, I want my Chief of Staff to help me navigate faster the most mundane pieces of work related to just getting through all the inputs I receive continuously.

MCPs and connectors that I need

• Layer 1: claude.ai Built-in Connectors. Enable at claude.ai/settings/integrations. No additional setup is needed: Gmail, Google Calendar, Slack, Notion, Clay, HubSpot, Canva and other standard connectors in Claude.
• Layer 2: Native MCP Servers

Google MCP. One server, all Google products, all accounts. Gmail, Calendar, Drive, Docs, Sheets, Slides.

brew tap ngs/tap
brew install google-mcp-server

claude mcp add-json google --scope user '{
  "command": "/opt/homebrew/bin/google-mcp-server",
  "env": {
    "GOOGLE_CLIENT_ID": "YOUR_CLIENT_ID.apps.googleusercontent.com",
    "GOOGLE_CLIENT_SECRET": "YOUR_CLIENT_SECRET"
  }
}'

Slack (per workspace). Token from api.slack.com/apps → OAuth & Permissions. You added all User Token Scopes for full access. Team ID from the workspace URL.

claude mcp add slack-neurons-lab -e SLACK_BOT_TOKEN=xoxb-your-token -e SLACK_TEAM_ID=TKB19144W -- npx -y @modelcontextprotocol/server-slack

Notion (per workspace). Token from notion.so/profile/integrations → New internal integration. Critical step: share each top-level page with the integration via ... → Connections.

claude mcp add notion-personal -e NOTION_TOKEN=ntn_your_token -- npx -y @notionhq/notion-mcp-server

Apple Notes. Required granting macOS Accessibility and Full Disk Access permissions to node.

claude mcp add apple-notes -- npx -y @ailenshen/apple-notes-mcp@latest

reMarkable. Alternative with full-text search index: uvx rm-mcp@latest with REMARKABLE_INDEX_PATH and REMARKABLE_INDEX_REBUILD=1.

# Cloud mode (no SSH needed):
uvx rm-mcp@latest --setup   # opens browser for auth, prints token
claude mcp add remarkable -e REMARKABLE_TOKEN='your-token' -e REMARKABLE_OCR_BACKEND=sampling -- uvx remarkable-mcp

Rize. No pre-built MCP existed. I had to create and open-source one.

claude mcp add-json rize-tracker '{"type":"stdio","command":"uv","args":["run","--directory","/path/to/rize-mcp","server.py"],"env":{"RIZE_API_KEY":"your-key-here"}}'

Obsidian. No MCP needed. Claude Code reads the vault directly from the filesystem when you cd into it.

• Layer 3: Chrome Extension. Covers everything without an MCP: LinkedIn, WhatsApp, X, Instagram, Telegram. Pair in Claude Desktop → Settings → Chrome Extension.

Claude skills architecture

Continuing the approach from my previous article, skills for my Claude Chief of Staff will be part of a “work” plugin, where work is a separate broad life area I maintain with AI. As always, you can find the skills described in this repository.

Examples of AI Chief of Staff work

Complete inbox triage

Just after asking Triage my emails, slack messages and unread meassages in messengers, waiting for some time (15 minutes, but it could take me an hour easily without AI) and asking for a anonymized version for my blog article, I will get the following result:

Daily / weekly brief

Just after asking Can you give me a weekly brief for my upcoming meetings with all the needed context from Slack, Notion, Clay and other systems, waiting for some time (~5 minutes for AI, something that I could do for 20–40 minutes) and asking for a anonymized version for my blog article, I will get the following result:

Backlog planning

Just after asking Take my tasks in personal Notion, and assign to them a project, priority and task type if not assigned yet, waiting for some time (~4 minutes for AI, I personally could spend maybe 5–10 minutes on it too, but I don’t have to anymore) and asking for a anonymized version for my blog article, I will get the following result:

Of course, the main changes I can see in my Notion, where all the tasks I created quickly without setting up project, priority and type is now done and pretty well done.

Cross-environment meetings preparation

In this example, I have asked my assistant to act on a task in Notion. It was about a review of a WhatsApp thread that contains Loom videos, documents, and links. I asked it to summarise it for me so I do not necessarily have to watch it all if I do not have enough time.

Next steps

First of all, I definitely want my AI Chief of Staff agent to be more proactive and run in background. I already run the inbox triage with a /loop Claude Code command on schedule so I know it’s dealing with my emails and other messages proactively and manages my inbox but I definitely want to extend this system to other actions.

Also, with so many moving parts the system is a little bit shaky. Ideally it would be good to reduce the number of environments and variables for example by moving everything I have from Notion to Obsidian and getting rid of Apple Notes to make the work for the agent easier. Last but not least, the token usage is still sub-optimal so I’ll have to work on optimization as well.

I will write about it in the next sessions while working on making Claude Code dealing with my other life areas or work activities. Knowledge management, finances, investments, relationships and many others — I still have to make AI help me with them :) You can text me on X or LinkedIn if you’re interested in building similar systems for you personally or in your business and need support. Let’s stay in touch!

Claude Code for Life #2: A Personal AI Chief of Staff for Daily Work was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Hello everyone! I want to start the new series related to a broad topic of agentizing personal life. I already wrote one article on the overall philosophy and architecture of modern AI agents and how to use different currently available commercial tools in practise. Also, a couple of years ago I was among the first practitioners building custom AI agents for our clients, so there is a bit more technical deep dive there as well.

Today, with the increased popularity of Claude, which is really well deserved, I want to show you how I am building a Personal Life Management agent following the architecture and understanding described in previous articles, but with very specific down-to-earth personal life applications. I will open source all the work I am doing on the way so you can follow my guide and use it as well. Let’s start with a cornerstone of a good life, which is physical health and wellness :)

Obsidian and Claude Code

I believe that Claude Code requires no introduction :) I’m a long-time Obsidian user as my personal knowledge management system, mainly because I like to have full control and all the data locally. Also, I have a mix of different files (PDFs, text files, Excels, EPUBs, and other formats) to deal with.

With the Claude philosophy to have everything locally and dealing with the .md files, it became a very natural choice to combine both tools together to allow Claude to use my local data. Of course, I also need to add all the external data sources to be queried on demand without cluttering my actual knowledge base.

The specific setup is inspired first of all by Thiago Forte’s PARA method and a bit more recent and AI-native Chase Adams’ setup. By the way, to run the terminal as an Obsidian tab, you can use this Obsidian community plugin.

Model Context Protocols (MCPs)

First, I need to ensure that my agent has access to all the external data sources and can query them on demand. In my case, I track my body composition with Withings scales, my sleep and recovery with Oura Ring, all my exercises with Garmin Watch and my food intake with Yazio app.

Withings MCP Server (body composition)

Package:gchallen/withings-mcp.First, you need to get API credentials. Go to Withings Developer Portal and create an application, note your Client ID and Client Secret and you’ll need to complete an OAuth flow to get your access token. Then, register your MCP in Claude with the following command:

bashclaude mcp add withings — npx -y @gchallen/withings-mcp \
 -e WITHINGS_CLIENT_ID=’your_client_id’ \
 -e WITHINGS_CLIENT_SECRET=’your_client_secret’ \
 -e WITHINGS_ACCESS_TOKEN=’your_access_token’

Oura Ring MCP Server (sleep and recovery)

Package: elizabethtrykin/oura-mcp. This one requires cloning and building from source — it does NOT work as a direct npx install. Go to Oura Developer Portal and create a Personal Access Token.

# Clone the repository
cd ~/mcp-servers # or wherever you keep your MCP servers
git clone https://github.com/elizabethtrykin/oura-mcp.git
cd oura-mcp
# Install dependencies and build
npm install
npm run build
# Add to Claude Code - point to the built server
claude mcp add oura - node /full/path/to/oura-mcp/dist/index.js \
 -e OURA_ACCESS_TOKEN='your_personal_access_token'

Garmin Connect MCP Server (exercises)

Package: garmin-connect-mcp. You use your regular Garmin Connect email and password (the same ones you use to log into connect.garmin.com).

claude mcp add garmin -- npx -y garmin-connect-mcp \
  -e GARMIN_EMAIL='your_email@example.com' \
  -e GARMIN_PASSWORD='your_garmin_password'

Yazio MCP Server (diet and nutrition)

Package: yazio-mcp. This uses an unofficial reverse-engineered API — Yazio does not provide an official API. The MCP server could break if Yazio changes their internal endpoints. Fine for personal use, just know the risk.

claude mcp add -s user yazio \
  -e YAZIO_USERNAME='your_email@example.com' \
  -e YAZIO_PASSWORD='your_yazio_password' \
  -- npx -y yazio-mcp

Claude marketplace, plugins and skills

In order to ensure that I can work on my agentic reasoning system separately from my files and the actual applications, I decided to create a personal marketplace and make it available to the public, in which I will create plugins with different life areas, which will contain different skills. You can find an example of a single skill below:

---
name: body-exercise
description: Analyze exercise consistency, training progression, and workload management from Garmin. Trigger on: exercise, workout plan, gym consistency, training volume, training frequency, cardio, strength progress, mobility, "am I training enough", or any question about training habits and execution.
---

# Body — Exercise

Analyze exercise behavior and training consistency with a focus on execution
against defined habits and systems.

## MCP Server

garmin-mcp — pull workout history, training load, training status, steps,
activity sessions, and related performance signals.

## Goals

Read goals from 300 Areas/Body/CLAUDE.md before every analysis.
Prioritize alignment with:
- gym 4x+ per week habit
- strength/cardio/mobility balance
- sustainable progression over intensity spikes

## Analysis

- Pull current week and last 4 weeks from garmin-mcp
- Evaluate session frequency against weekly habit target
- Compare recent load vs historical baseline to flag over/under-loading
- Assess pattern consistency (missed streaks, irregular session timing)
- Distinguish volume progress from random single hard sessions
- Correlate training pattern with sleep/recovery context when relevant
- For planning questions, recommend next 7 days intensity mix:
  - hard / moderate / light / rest distribution

## Resources

Before giving recommendations, search 400 Resources/ for relevant material:
1. Use find and grep to locate files related to the current query
2. Read .md and .txt files directly
3. For PDFs, extract text and scan for relevant sections
4. For Excel files, read and parse relevant sheets
5. Prioritize newer files over older ones
6. Cite which resource informed your recommendation

Search broadly — file names may not be descriptive. Look at actual content.

## Tone

Quant analyst reviewing a dashboard. Numbers first, brief context, no fluff.

## Schema

Reference ../../schemas/exercise.json for field definitions.

With this architecture, I can be sure that I can work separately on the Marketplace plugin and skills, and it is decoupled from my agent, which actually does the work. You can find git with this marketplace here with all the skills described included, and this is where you add it to your Claude:

Use case examples

Simple data queries

As expected, Claude is able to access my local data very easily and provide the answers around them. The same works with data sources from the MCP servers.

Deep analysis and comparisons

This is a more interesting case that requires Claude’s reasoning — knowing which data sources to pull, what to prioritize, and how to guide the comparisons and analysis. Thanks to MCPs and skills it was done perfectly!

References grounding and strategy

In many cases I want to push Claude to think within some boundaries, i.e. my saved resources in Obsidian (think of my saved notes, articles, courses etc) instead of its own reasoning. This can be done very good as well.

Next steps

Of course, the very obvious next step is the creation of new life areas with corresponding skills and adding new MCPs to work with my created engine. A couple of other subtle details that became apparent and are important to fix to make everything work even better:

1. All the data has to be in a Claude-friendly data format. As you could see, scanned local PDFs are not so easy to recognise natively with Claude, so I either need to turn them into .md files or to find some other tool or MCP that is better in OCR.
2. For many things, I need a better interaction on the web, and I will need to use our MCP or a tool that allows a direct browser interaction and use.

I will write about it in the next sessions. You can text me on X or LinkedIn if you’re interested in building similar systems for you personally or in your business and need support. Let’s stay in touch!

Claude Code for Life #1: Managing my Health and Wellness was originally published in Data Science Collective on Medium, where people are continuing the conversation by highlighting and responding to this story.

Hello everyone! In this article I want to share what I’ve been researching lately: how to evaluate AI agents for portfolio construction and investment decision support.

I’ve worked in AI + finance for years, long before LLMs made it fashionable. And one thing that’s been constant in this space is that evaluation is the real battleground. In the classic machine learning era, the workflow was familiar: collect a dataset, split into train/validation/test, run simulations, optimize clear numerical metrics.

LLMs changed that. The output stopped being purely numerical and became language, reasoning, and judgment. Suddenly “accuracy” wasn’t a single number anymore. Over the last year, one practical answer has emerged: LLM-as-a-judge. We’ve used it at Neurons Lab across multiple client projects, and it works, but it comes with a catch: you still need high-quality evaluation data, and in finance, that’s often the hardest thing to get.

That’s where AgentBeats became interesting to me. I discovered it through Berkeley’s AI agents course, and the premise is almost a meta shift: instead of building a dataset and hand-crafting metrics first, you build an evaluator agent — an agent whose job is to test, challenge, and score other agents.

1. The setup: AI agent to build AI agent to evaluate AI agent

Agentic AI coding at its fullest

For this project I added one rule for myself: I wouldn’t write any code by hand. I only gave instructions, and an AI coding agent (Claude Code) built everything for me. That sounds unreliable, but it did all the work perfectly. Of course, I still had to do the hard part: be clear about what I want, collect the right context, and define the system in a way an “assistant engineer” can execute. So before the build, I gathered a bunch of inputs and dropped them into the repo:

• Discord chats from the competition (where the real problems show up)
• Workshop videos + transcripts
• The official docs and Github repositories
• Working examples from people who submitted before me
• My own rough description of what the agents should do

This was enough to build the system practically end-to-end.

One agent builds, one agent judges

At a high level, the system is simple: Purple agent = does the job, Green agent = judges the job. In my case, the “job” is goal-based portfolio recommendations. The purple agent gets a short story like “I want to buy a house in 10 years”, “I’m saving for retirement”, “I’m planning for my child’s education” with some numerical inputs like savings. It then returns which assets/tickers to use, the allocation weights and a short explanation. You can find purple’s agent code here.

For the first version, I kept it purple agent intentionally basic: LLM generates the portfolio without tools. In real portfolio management you’d add lots of logic like constraints, risk models, backtests, taxes, and so on. But that’s not the point yet. First I wanted the full pipeline running end-to-end. Then the green agent takes the goal, the portfolio, the reasoning and gives an estimate like: how likely is this plan to reach the goal?

That’s the core idea I’m exploring: instead of needing a perfect dataset upfront, you can build an evaluator agent that applies consistent rules and judgment and keep improving it over time.

2. The build: evaluation logic for investment management

Next I focused on building the Green Evaluator agent. At the beginning, the first baseline was also purely LLM-based. I did that on purpose, not because it’s the best evaluator, but because I needed to first polish the infrastructure: GitHub Actions, Docker, and making sure everything actually runs end-to-end on the AgentBeats platform. And honestly, the only way to make that smooth was to study multiple working examples from other participants and copy the patterns that already worked.

Once the whole workflow was stable, I added the quantitative part. I used Yahoo Finance to download historical data for the tickers, estimated basic performance parameters from history, and ran forward simulations to approximate how likely the portfolio is to hit the goal. Then, as a sanity check, I sometimes double-checked specific things using Google Search as a tool.

So at this point the evaluator stopped being “just text.” It became a more agentic setup that uses multiple tools: market data, Python code execution, and web search. In the end, I could set up my own leaderboard around this idea: people can build their own purple agents that design portfolios better than a raw LLM, submit them, and my green agent scores them using simulations — producing a single score that reflects the probability of achieving the goal. You can find the code for the green agent here.

Next steps and improvements

Of course, this submission is just a proof of concept. It’s not how real production agents in investment management should look.

In the real world, “here are some tickers and weights” is not enough. A portfolio has to fit the investment policy of the firm, and the policy isn’t just a PDF — it’s the operating system of the business. Things like:

• rebalancing rules
• tax strategy
• additional asset classes (beyond a simple ETF mix)
• the client’s liabilities and cashflows
• constraints you can’t break (exposure limits, banned assets, concentration rules)

The evaluator also has to go much deeper than my first version. If you want something closer to professional investment research, you’d at least move toward approaches like Black–Litterman (or more modern): combining historical data with market “views”. And those views can come from internal research, external signals, or even the internet — but they need to be explicit and testable.

Same for simulations: forward-simulating with basic assumptions is a start, but a serious evaluator would model a wider range of regimes and shocks. That might mean more advanced stochastic models, scenario generation, or even generative ML models that can produce richer “possible futures” than simple historical replay.

And finally: risk preferences are not always a single number. Many investors care about specific exposures — geographies, sectors, asset classes, or event risks — and those constraints need to be reflected both in portfolio construction and in evaluation.

I’ll keep working on this. And I’m very likely to bring this “agentic evaluation” approach into real client work at Neurons Lab, because it solves a problem we see all the time: teams need to iterate fast, but the data needed for perfect evaluation is often messy, incomplete, or simply unavailable.

Thanks for reading!