AI投资系统：运行正常不等于决策正确

Why automating analysis didn't automate conviction

One verification record showed 1,532 tests passed with zero failures. Another showed 16 out of 16 checks green.

I still would not place a trade because the system told me to.

Later, a broader validation found that 48 tests had sat outside the default regression. When they were included, 15 failed and one would not even collect. The original numbers were real inside their boundary. The boundary was incomplete.

Those tests measured whether predefined software paths behaved as expected. They did not measure whether the path asked the right question about a company.

A missing test can eventually be added. A broken data source can be replaced. The more dangerous failure was a system that ran cleanly, produced a precise valuation, and answered the wrong investment question.

I built two versions of an AI-driven investment system. Version one became an unmaintainable pile of features. Version two had a much better product architecture, technical architecture, and iteration process. It kept improving as software, yet I still did not trust it to make the decisions that mattered.

The Agent made plenty of judgments. I still couldn't place the bet.

This is what I misunderstood about automating investment: analysis can be delegated. Conviction does not arrive automatically with the output.

This article was co-created with an AI Agent. I supplied the investment experience, evidence, judgment, and boundaries; the Agent helped with structure, drafting, evidence organization, review, and platform adaptation. I reviewed the factual claims and remain responsible for the final argument and sign-off. The Production Log records material corrections and the limits of this workflow.

What I Was Trying to Automate

Before building either system, I already used AI in my investment research.

The workflow was more structured than asking a chatbot for stock tips. I would put earnings reports into NotebookLM and use it to extract the relevant facts. A prompt would turn those facts into an initial research report. Then I would use chat to challenge the report: check a number, request another source, question an assumption, add missing context, and update the conclusion.

The important part was not the chat interface. It was that the conclusion remained provisional.

A follow-up question could change the research path. A contradiction could force a different model. If the answer felt wrong, I could ask why, inspect the evidence, and keep pulling on the thread until I understood what the real uncertainty was.

I wanted to turn this workflow into a system.

My ambition was straightforward: an Agent would monitor markets, identify promising companies, collect evidence, value them, and recommend investment decisions. I would still execute the trades, but the system would make investing simpler and, I hoped, improve my odds.

There was a hidden assumption inside that ambition: if I automated enough analytical steps, conviction would appear at the end of the pipeline.

Version One: Feature Accretion Disguised as Progress

The first version was built feature by feature.

I would describe a capability. The Agent would implement it. Then I would ask for another capability, and the Agent would add that too. Market analysis. Company research. Valuation. Portfolio views. Risk rules. More data sources. More reports.

At first, the speed felt extraordinary. Every conversation seemed to produce visible progress.

Then I realized I no longer knew what the system actually contained.

I could not clearly explain which capabilities were complete, which were partial, which data path was authoritative, or what a new change might break. I asked the Agent to write blueprints so I could recover a map of the system. But documenting the pile did not restore control over it. Bugs appeared across modules, fixes created new inconsistencies, and the codebase became difficult to maintain.

Humans are often accused of building code piles. AI may be even better at it. It can produce locally reasonable code faster than the owner can maintain a coherent model of the whole.

The failure was not simply "bad code." It was a delegation failure.

I had delegated implementation faster than I had built the ability to evaluate what was being implemented. Output increased while my understanding decreased. By the time I felt the loss of control, the codebase itself had become the only complete description of the system—and neither I nor the Agent could reason reliably about all of it at once.

So I abandoned the feature-by-feature approach and tried again.

Version Two: Better Engineering, Same Investment Problem

For version two, I did what version one should have done from the beginning.

The AI and I co-designed the product architecture. We defined the technical architecture. We created iteration rules, acceptance criteria, validation steps, and clearer boundaries between modules. Instead of asking for isolated features, I tried to build a governed system.

This genuinely improved the engineering.

The modules became easier to name. Changes became more auditable. Tests and checks caught real failures. The system became more deterministic, and some parts became useful: reading positions and prices, attaching source and date evidence, recalculating formulas, checking concentration, and enforcing risk limits.

But the system kept iterating without becoming a tool I could rely on for the central investment question.

Part of the bottleneck was data. I wanted to solve the problem with free sources, but availability and coverage varied widely. A source might work for prices but not financial history, work for one market but not another, or return stale or degraded data exactly when a deeper comparison required it. Deterministic logic cannot recover information that was never acquired reliably.

Yet data was only part of the problem.

The bigger issue was that the system had become more consistent by becoming less adaptive. It could reproduce a method. It was much worse at recognizing when the method did not fit the company.

I had improved technical determinism while removing part of the intelligence I valued in the original research loop.

When the Pipeline Completes but the Analysis Fails

Tencent made the problem visible.

My fixed system ran a generic valuation path. It used a single forward earnings multiple to represent a diversified platform. Other inputs, including the earnings estimate and industry data, were also weak or degraded. The pipeline completed. It produced a valuation and an action signal.

The result later had to be formally rejected.

The software had done what it was designed to do. That was precisely the problem. It had no reliable way to ask whether the design represented Tencent's actual business mix before turning the inputs into an answer.

In the iterative research workflow, I could interrupt much earlier. Does one multiple make sense across social networks, gaming, advertising, financial technology, cloud, and investments? Which earnings figure is being used? Are the growth and discount assumptions consistent with each other? What important business segment disappears when everything is compressed into one number?

Those questions do not merely improve the input. They can change the model.

Micron exposed the same problem from the opposite direction.

A memory manufacturer can look optically cheap near peak earnings. If the system treats the latest profit as a stable base, a low multiple can become a false signal. The research question is not simply "What multiple should this company trade at?" It is also: How much of current profit comes from pricing? Where are we in the supply cycle? What happens when new capacity arrives? Which margins are sustainable through a downturn?

When the automatic research route had gaps, the more useful workflow switched to auditable filings and an explicit cycle thesis. It separated current strength from normalized earnings and asked what evidence would show that the economics had structurally changed.

Tencent and Micron did not need two different numbers from the same machine. They needed different questions.

This is the failure mode I now fear most:

A failed system throws an error. A more dangerous one runs cleanly and returns an answer that makes no business sense.

The Experiment That Changed My Mind

I compared the fixed investment system with the iterative research loop on the same kind of task: valuing a company and deciding what the evidence implied.

This was a diagnostic comparison, not a controlled benchmark. The two paths did not necessarily have identical tools, search access, or data. I used the comparison to locate a workflow failure: which path helped me detect and correct a bad premise before it became a recommendation?

The fixed system had real advantages. Its calculations were reproducible. Its output format was stable. It followed the path it had been given.

But that path was also its limit.

The iterative workflow started with extracted earnings evidence and an initial report, then used follow-up questions, search, cross-checking, and memory to revise the conclusion. It could notice a category error. It could replace a source. It could challenge the peer group. It could decide that a different valuation method was needed. Most importantly, it could turn "this answer looks wrong" into a new research question instead of forcing the answer through the existing pipeline.

The adaptive workflow produced better analysis in these cases. This does not show that chat is inherently superior to software, or that an LLM should be trusted without verification. It showed that open-ended research and deterministic validation are different jobs.

I had tried to make one system do both.

By encoding more of the work as fixed logic, I gained consistency but lost the ability to change the research route. By leaving everything in chat, I retained adaptability but risked inconsistent calculations, forgotten evidence, and conclusions that were difficult to reproduce.

The answer was not to choose chat or code. It was to stop asking either one to own the whole investment decision.

Analysis Can Be Delegated. Conviction Still Has to Be Owned.

An Agent can collect facts, compare companies, build scenarios, calculate valuations, and even recommend an action. Those are analytical outputs.

In my own process, conviction was different.

I do not mean certainty. Investment decisions are made under uncertainty, and confidence theater is not conviction. I mean an owned explanation of why the expected upside justifies the downside, which evidence matters, what could falsify the thesis, and why this opportunity deserves scarce capital now.

That explanation has to be owned by the person who accepts the consequence.

If an Agent tells me to buy but I cannot explain why its model fits the business, I have not received conviction. I have received an answer. If the system produces a target price but I cannot identify the assumptions that would reverse the conclusion, I do not own the decision. I am borrowing the system's confidence.

That is why I would not place the bet even after the Agent had made many judgments.

I originally hoped automation would make investing simpler and improve my odds. Instead, it automated many tasks around the difficult part while leaving the difficult part intact: choosing what matters and accepting uncertainty.

I no longer expect investment automation to provide a silver bullet. It can reduce effort, improve evidence quality, and enforce discipline. In my own system, trying to remove myself from the judgment also removed my ability to trust the result.

My Current Architecture: Code, Agent, Human

I abandoned the fantasy of fully automated investing, but I did not abandon Agents or the useful parts of the second system.

I changed the division of labor.

These are responsibilities, not three isolated products. The Agent can call deterministic tools, and software can orchestrate an adaptive loop. The boundary is about which kind of claim each layer is allowed to settle.

Keeping a human in the loop is not enough if the human only clicks approve. The accountable person must be able to reconstruct the thesis, challenge its assumptions, and name the evidence that would reverse the decision. Otherwise approval is a ceremony, not ownership.

I now prefer to develop a method inside the adaptive Agent-human loop first. I want to see which questions recur, which evidence changes the conclusion, and which calculations become stable. Only then do I extract the mature, repeatable parts into deterministic software.

This reverses my original approach. I used to automate early and try to recover intelligence later. Now I keep the research path flexible until I understand what deserves to become deterministic.

Code should verify an investment process, not impersonate an investor.

A 30-Minute Decision Ownership Audit

You can apply this beyond investing. Choose one consequential output from an Agent system—a hiring recommendation, legal review, medical research summary, strategic plan, or investment report—and ask five questions.

1. What decision is this output actually supporting?

If the answer is "produce a report," the business question is still missing. Name the real decision and who owns its consequences.

2. Which parts are deterministic?

Mark the facts, calculations, freshness checks, and hard constraints that code can verify. These are good automation candidates.

3. Where might the method need to change?

Identify the moments when new evidence should change the question, source, comparison set, or model. Keep those moments inside an adaptive Agent-human loop.

4. Can the accountable human explain the bet?

Use three sentences: Why act now? What is the most important way this could be wrong? What evidence would reverse the decision?

5. Would that person act?

If the answer is no, do not add another feature yet. Record the missing evidence or unresolved judgment. The gap may not be automation. It may be ownership.

Then make two changes: move one mature verification step into deterministic software, and move one ambiguous judgment back into the Agent-human loop.

Version one taught me that feature velocity can destroy control.

Version two taught me that better architecture can solve engineering chaos without solving the business judgment at the center of the system.

My current approach is less ambitious and more useful. I use adaptive Agent-led research to explore questions, challenge models, and develop methods. I reuse deterministic software for evidence checks, calculations, risk barriers, and workflows that have become stable enough to deserve determinism. I make the final judgment and execute the trade.

This is an N=1 result, not proof that AI cannot invest. Another builder may find a better boundary, and future systems may earn a different level of authority. It is also not investment advice.

It is simply the lesson I needed two systems to learn:

Don't automate the bet. Automate the evidence around it.

---

This section is generated from the actual production record for this article. It is included because the system that helped write the argument also had to be corrected by the human who owns the publication decision.

The last corrections mirrored the article's argument. A checklist can confirm that a section exists or two files match. It cannot decide whether the contract itself asks the right question. File presence was not publication readiness, just as a completed valuation was not investment conviction.