MARCH–NOVEMBER 2026 · CONVERSATION META-SUMMARY

Trinzo AI output quality

A meta-summary of our conversations

Looking across our discussions from March to November 2026, there's a remarkably consistent theme in how we've approached Trinzo's AI Meeting Minutes & Project Updates application.

The underlying question has shifted from “Can AI produce good meeting minutes?” to “Can we demonstrate that the AI preserves what matters, avoids misleading the reader, and reduces the work required to produce an accurate, usable record?”

That distinction has influenced the architecture, testing methodology, scoring systems, user interface and even how we've defined whether the application is ready.

Five guiding principles

There are five major principles running through our conversations:

• Accuracy means preserving meaning, rather than reproducing reference minutes word for word.
• Omissions can be more consequential than obvious hallucinations, because a missing commitment may go unnoticed.
• Actions, decisions, discussions and existing arrangements need different treatment, particularly when interpreting conversational language.
• Output quality must be measured alongside human review effort, because a comprehensive but exhausting document defeats much of the application's purpose.
• Improvements must be demonstrated through controlled comparisons, rather than assumed from more sophisticated prompts, models or architecture.

1. How our thinking developed

Mar–Jun · Reliability and architecture

We explored output consistency, hallucinations, assumptions, evidence, structured data and the difference between generating text and controlling AI behaviour.

Aug · Meeting understanding and accuracy

We identified action extraction, contextual interpretation, omissions and false commitments as major weaknesses. The ambition was approximately 97% accuracy with minimal editing.

Sep · Evaluation methodology

We challenged strict reference matching, measured unsupported owners and missing actions, and began treating reviewer effort as a quality dimension.

Oct · Practical readiness and comparative testing

The focus became whether the application was usable in a limited internal pilot, what could mislead users, and which defects genuinely warranted intervention.

Nov · Semantic preservation and targeted refinement

Beta demonstrated substantially faster processing and better meaning preservation in the recorded benchmark. The next challenge was protecting compatible arrangements and eliminating remaining omissions without destabilising what already worked.

2. The quality dimensions we've repeatedly returned to

An important distinction emerged: these dimensions should not all be collapsed into one accuracy percentage.

For example, a system might achieve excellent factual precision because it includes very little. Another might capture almost everything but introduce unsupported owners.

Neither should receive an excellent overall assessment without those limitations being visible.

3. What our actual testing revealed

This is where the conversations become particularly useful, because we have quantitative evidence rather than relying entirely on theoretical quality criteria.

September: the early accuracy problem

In our 1 September discussions, the earlier meeting-minutes evaluation produced these figures:

One of your most important observations was that the reference minutes were sometimes extremely granular, almost word-for-word.

That raised a methodological issue: a difference from the reference document is not automatically an AI error.

Suppose the reference contains three separate points about the same project risk, while the AI combines them into one accurate sentence. A literal comparison may penalise the AI, even though the combined output is arguably better.

We therefore began distinguishing between:

• A genuinely missing meaning.
• A meaning preserved in different wording.
• Several points legitimately consolidated.
• An unsupported addition.
• A factual contradiction.
• A reference detail that might not be necessary in the final minutes.

This is one of the strongest principles we've developed: the transcript is the evidence source; the reference minutes are a comparison artefact, not unquestionable ground truth.

October: Original versus Beta

The 30 October benchmark was a much more encouraging result.

Approximately 8.5× faster across 10 runs involving five transcripts.

An improvement of 19 preserved meanings, or 27.5 percentage points.

The implications were quite significant.

Beta was faster, preserved more of the evaluated source meaning and avoided the duplication problem observed in the comparison.

However, 55/69 still means 14 evaluated meanings were not counted as preserved. That is why our recommendation was not to declare the application finished.

Instead, the direction was:

Keep the Beta baseline unchanged, investigate the remaining missing meanings, and test targeted changes against the existing benchmark.

That is also why we've been cautious about introducing new processing stages simply because they sound sophisticated.

4. The most important conceptual problem: what actually counts as an action?

This has probably been the most persistent source of complexity.

In August, the webinar rehearsal examples showed that the AI could turn conversational language into false actions, while simultaneously overlooking genuine commitments.

We kept coming back to the fact that these statements are fundamentally different:

The subtle issue is that the AI can produce a perfectly grammatical action that is semantically wrong.

For example:

Source

“We already have a meeting arranged with the regulatory team next week.”

Incorrect output

“Arrange a meeting with the regulatory team.”

Correct output

“A meeting with the regulatory team is already scheduled for next week.”

The incorrect output invents work and changes the project's status.

This is the reason our November discussions specifically identified compatible arrangements as an area for further testing.

An existing arrangement can be important enough to include in the minutes without being a new action. It must also be preserved accurately when generating a project update.

The deeper principle

We've effectively been working towards a system that distinguishes:

Discussion

What was considered

Decision

What was agreed

Action

What someone committed to doing

Arrangement

What is already planned

Completed

What has happened

Unresolved

What still needs confirmation

This distinction matters more than formatting, because it determines what the final document communicates about the actual state of a project.

5. Why we explored structured meeting understanding

In early August, we discussed the architecture:

The rationale was that minutes and project updates should be sibling outputs from shared meeting understanding.

Otherwise, if the project update is generated from imperfect minutes, the second generation stage can amplify the first stage's omissions or mistakes.

We discussed shared processing for transcript cleaning, names, participants, topic classification, commitments, dates, evidence and confidence.

But our thinking subsequently became more pragmatic.

A more elaborate architecture is only worthwhile if it demonstrably improves output quality, reliability or reviewer effort.

That distinction matters because we've also explored specialist classifiers and structured extraction.

August's classification experiments

The recorded initial validation results included:

The important finding was that action state, evidence type and minority signals remained weak, even when other classification tasks looked excellent.

That aligned with our practical observations: understanding whether something was discussed was easier than reliably deciding whether a genuine action had been created.

We also discussed class imbalance, per-class performance, confusion matrices and threshold calibration, rather than endlessly tuning an aggregate score.

6. The distinction between an AI error and an editorial choice

One of our recurring questions has been whether the AI has genuinely failed or whether it's produced a different but acceptable representation.

I would summarise our evolving position as follows.

This means we've gradually moved away from treating output quality as a conventional text-similarity problem.

Instead, we're measuring whether the AI has accurately represented the meeting's information and implications.

That is a much more defensible standard for Trinzo.

7. Human review is part of output quality

Our September discussions about Jacqui Fox brought another dimension into focus.

We were considering someone who may be time-poor, relatively unfamiliar with the technical implementation and understandably sceptical about AI-generated records.

An output that requires 20 minutes of detective work to verify may have little practical advantage over manually writing the minutes.

We developed five usability considerations:

• Orientation: Can the reviewer understand what was generated within five seconds?
• Attention: Can they immediately identify what needs checking?
• Evidence: Can they inspect the transcript support for questionable statements?
• Correction: Can they fix an error without fighting the interface?
• Completion: Can they confidently finish, save and approve the record?

We also discussed a simple 0–2 scoring rubric for each, producing a score out of 10.

The most important measure was review effort per meeting.

This introduced a useful distinction between:

• Model quality: How accurately does the AI represent the source?
• Product quality: How reliably and efficiently can a person turn the output into an approved record?

These are connected, but neither substitutes for the other.

8. Our approach to evidence, uncertainty and hallucinations

Another consistent theme has been avoiding false certainty.

We've discussed confidence and evidence traceability, but the goal has never been to cover every sentence with warnings.

The useful principle is that uncertainty should be specific and actionable.

For example, an action with an unconfirmed owner should retain the action and indicate that ownership requires confirmation.

It should not silently