On this page
Post 02 ended with a green npm test command.
That statement was true, and it was easy to give it more weight than it deserved.
The two frontend tests called a small query-construction helper. They established that the default request kept limit: 8, and that selecting a category added the category parameter. They did not create the Angular component, render its template, change the category selector, observe an HTTP request, or check what appeared on screen.
The application worked in the browser. The automated evidence still stopped before the interaction began.
For Post 03, I did not ask Codex to add another feature. I asked it to improve the verification of the feature we already had.
By a verification loop, I mean the feedback cycle around a software change: make the change, check it against the expected behavior, use any failure to guide a correction, then run the checks again. In shorthand: change -> verify -> detect failure -> correct -> verify again.
This experiment establishes the automated foundation for that loop around the existing category filter. A separate AI review session later shows that the tests detect two important regressions, but Post 03 does not show Codex implementing a new feature, finding and correcting a regression, and repeating the complete cycle. That remains work for a future experiment.
The repository checkpoints are available here:
- Starting point:
post-02, commit7741daff30c0acb96ffa3a4c4e64051b2d523c18 - Final implementation:
post-03, commit95a8e78767b5ed2a7cd16ed8f4fe30b8f3693240 - Repository diff:
post-02...post-03
The central question was not whether Codex could increase the test count. It was whether Codex could establish a verification loop that would fail when important application behavior regressed.
The green command with a narrow boundary
The Post 02 helper was small and useful:
export function recentTransactionQuery(category) {
return category
? { limit: 8, category }
: { limit: 8 };
}
Its tests proved that function returned the expected objects. That was the full automated frontend boundary.
If the template lost its (change) binding, those tests would still pass. If category selection stopped triggering an HTTP request, they would still pass. If the component received the filtered response but kept showing the previous transaction, they would still pass.
This did not make the helper tests wrong. It made their evidence narrower than the feature.
The task
I gave Codex a detailed engineering task because I wanted the experiment to stay focused on verification rather than drift into frontend redesign:
Improve the automated frontend verification for the existing transaction category-filtering behavior.
Starting point: use the accepted `post-02` state of this repository. The Angular frontend is under `frontend/`. It currently has a passing `npm test` command, but the existing tests exercise only the transaction query-construction helper rather than the Angular component and user interaction.
Please inspect the frontend source, Angular workspace configuration, package scripts, dependency state, and current tests before editing. Then briefly explain:
- which user-visible behaviors are most important to protect;
- the smallest suitable testing approach for this repository;
- any configuration or dependency changes you expect to need.
Implement focused automated verification of the existing category-filter behavior. Prioritize observable application behavior over test count. The resulting checks should exercise the rendered Angular component and, as appropriate for the approach you choose, establish important behavior such as:
- initial loading of accounts, categories, and recent transactions;
- selecting or switching a category and displaying the returned transactions;
- preserving the account summary while transactions are filtered;
- meaningful loading, empty-result, and transaction-error states;
- asynchronous request completion behavior where inspection shows it is relevant.
You do not need a separate test for every bullet. Choose a proportionate minimum set that would fail for meaningful regressions in the interaction.
Use controlled and deterministic dependencies or HTTP responses. Avoid tests that pass only because they call an extracted helper while bypassing the Angular component, template, or user interaction.
Constraints:
- Do not add new product functionality.
- Do not prescribe or adopt a broad frontend architecture refactor as part of this task.
- Do not change production code unless a failing test demonstrates a defect or a small change is necessary for proportionate testability.
- If a test reproduces a product defect, preserve and report the failing evidence before fixing it, explain the defect, and make the smallest justified correction with a regression test.
- If a suspected issue cannot be reproduced, report it as unconfirmed rather than claiming it is fixed.
- Do not add arbitrary sleeps or timing-dependent tests.
- Do not add CI/CD, AGENTS.md, Codex-specific repository instructions, skills, MCP configuration, specialized agents, or delegated workflow setup.
- Keep README verification instructions accurate if commands change.
Run the narrowest new checks while developing, then run the complete frontend test command and production build. Run other repository checks only if your changes make them relevant. At the end, report the behaviors now verified, commands and results, production files changed and why, and remaining limitations.
The prompt defines the testing level and the scope boundaries. It does not choose a runner, prescribe a test file, specify how to control HTTP, decide whether the helper tests should remain, or tell Codex how to handle Angular’s asynchronous state.
Those decisions were left to the implementation session.
Codex chose the component boundary
Codex inspected the Angular workspace, TypeScript configuration, package scripts, dependency state, component, existing tests, and installed Angular builder before editing.
It proposed Angular’s own component TestBed, the HTTP testing backend, the Angular 21 unit-test builder, Vitest, and jsdom. That was a proportionate choice for this repository: enough Angular runtime to render the real component and template, but no browser automation or end-to-end stack.
The boundary changed in an important way:
The final diff adds one app.component.spec.ts, a test-specific TypeScript configuration, the Angular test target, and direct Vitest/jsdom development dependencies. It removes the old helper-only test because the same query contract is now exercised through the component’s real HTTP requests.
No production application file changed.
What the tests actually do
The new suite contains three tests.
The first creates the real AppComponent, observes the three startup requests, supplies accounts, categories, and transactions, then checks the rendered account summary, selector options, and transaction content.
The second drives the category interaction through the DOM. A selected value is assigned to the real <select>, and a change event travels through the production template binding:
chooseCategory('Groceries');
const groceriesRequest = http.expectOne(
request => request.url === '/api/transactions/recent' &&
request.params.get('limit') === '8' &&
request.params.get('category') === 'Groceries'
);
fixture.detectChanges();
expect(text()).toContain('Loading transactions...');
expect(text()).not.toContain('Corner market');
expect(text()).toContain('1,200.25');
expect(categorySelect().disabled).toBe(true);
groceriesRequest.flush([
{ ...recentTransactions[0], id: 11, description: 'Weekly shop', amount: -61.2 }
]);
await renderCompletedRequests();
expect(text()).toContain('Weekly shop');
expect(text()).not.toContain('Corner market');
The test then switches to another category, returns an empty response, and checks the empty-result message while the account summary remains visible.
The third test returns an HTTP 503 for a filtered transaction request. It checks the transaction-specific error message, confirms that the account summary remains present, and verifies that the selector becomes available again.
This is still mocked HTTP. The tests do not start Spring Boot or PostgreSQL. But they now cover the collaboration that was missing before: template event, component state, request parameters, controlled response, and rendered result.
The setup did not work on the first attempt
The final test file is compact. Getting there was not a straight line.
The first narrow run exposed a TypeScript compilation-boundary problem. The minimal Angular workspace’s application configuration included main.ts, not discovered spec files. Codex classified that as test configuration rather than a product defect, added a separate tsconfig.spec.json, and pointed the test target at it.
The next problem was asynchronous completion. Codex initially used fixture.whenStable(), but the check could complete before the component’s native Promise.all(...).finally(...) chain had cleared the loading state.
It then tried fakeAsync and a microtask flush. That approach expected legacy ProxyZone support that was not available in this Vitest runner.
The final solution was a bounded microtask drain tied to the component’s loading signals. The snippet below is reformatted for the article width; its behavior and technical meaning are unchanged:
async function renderCompletedRequests(): Promise<void> {
for (
let remainingTurns = 10;
fixture.componentInstance.loading() ||
fixture.componentInstance.transactionsLoading();
remainingTurns -= 1
) {
if (remainingTurns === 0) {
throw new Error(
'The component did not finish loading ' +
'after its pending request completed.'
);
}
await Promise.resolve();
}
fixture.detectChanges();
http.verify();
}
It is not the only possible synchronization helper. It does have two useful properties for this experiment: it does not wait on wall-clock time, and it fails with a clear bound if the loading state never settles.
It also couples the test to the component’s public loading signals. A separate AI review session identified that coupling as a minor maintainability limitation.
The preserved conversation records Codex’s diagnosis and course changes, but some tool operations are summarized without complete terminal output. I can establish the sequence and the final code; I cannot reconstruct every intermediate error line from the transcript.
Verification results
Codex reported the focused suite passing, followed by the complete frontend test command, production build, and diff check.
In a separate AI review session, the final checkpoint was rerun:
npm test — 3 tests passed
npm run build — production build succeeded
git diff --check — clean
The tests failed when the behavior broke
A passing suite established that the final checkpoint was stable. The stronger evidence came from deliberately breaking the behavior in a disposable copy.
The separate AI review session first removed the selector’s (change) binding. Two of the three tests failed because no filtered request appeared.
The same session then restored the binding and changed recentTransactionQuery so it no longer propagated the selected category. Again, two tests failed, this time showing the unfiltered request that had been issued.
Those two mutations do not prove comprehensive coverage. They do show that the tests can detect the regressions most central to their claim: disconnected user interaction and lost category propagation. That matters more than the number three.
The autonomy was real, but bounded
No follow-up human correction appears in the implementation conversation. Codex selected the exact tooling, grouped the scenarios, decided to remove the redundant helper test, diagnosed the TypeScript boundary, abandoned two unsuitable async strategies, and arrived at the final synchronization helper without being steered through those choices.
That is high implementation autonomy, but not unconstrained autonomy. The task already said that helper-only coverage was insufficient, named the important user-visible behaviors, required controlled responses, and ruled out sleeps and broad refactoring. I defined the engineering target and its boundaries. Codex independently chose how to reach it in this repository, then recovered from the failures it encountered.
What is still outside the loop
Three passing component tests are not comprehensive frontend verification.
The most important remaining gap is the partial-startup ordering risk carried forward from Post 02. If one startup request fails early while the original transaction request remains in flight, the selector may become available and a filtered request may overlap the original one. These tests complete the startup requests together. They do not reproduce or rule out that sequence.
The suite switches from Groceries to Transport, but never returns to “All categories.” It checks representative transaction content rather than every rendered field, and it does not cover account or category startup failures. Because it uses jsdom with mocked HTTP, it also stops before the deployed path from browser to Spring Boot to PostgreSQL.
Codex’s own final report mentioned the browser/backend boundary and the missing account/category failure paths. It did not identify the partial-startup ordering risk or the return-to-all gap. Its report was accurate, but not exhaustive.
What changed
Post 02 left the repository with a passing command whose evidence stopped at a helper function. Post 03 left it with a small component suite that renders the application, drives the interaction, controls the HTTP boundary, and checks visible outcomes. It required no production refactor, and the mutation audit demonstrated that the tests fail for the central regressions they claim to protect.
The result is narrow, but useful:
detailed verification task
-> autonomous test-harness choices
-> real setup failures and corrections
-> component-level regression protection
-> independent attempts to break the tests
-> demonstrated regression sensitivity with explicit limits
This gives the next experiments a stronger foundation. New implementation work can be judged against behavior whose failure is now observable, while the unresolved asynchronous risk remains recorded rather than silently treated as solved.
A green command is not a verification strategy by itself. Its value depends on what can make it fail.