Four Production AI Systems for a Japanese Contact Centre Operator

#The starting position

The operator runs contact centres for public sector clients in Japan. One of its services is an emergency consultation line: people call when they are unsure whether a medical situation needs an ambulance, and trained operators assess the situation and advise them. Another is multilingual information support for large events, where operators answer visitors' questions in Japanese and English.

Both services depend on operators doing several things at once. On a consultation call, the operator listens, asks the right questions, decides on advice and records the call in a structured form with sixteen fields, all while a worried person is on the line. On the event line, the operator listens to a question, searches reference material and answers, often switching between languages. Typing and searching compete with listening, and every second spent on them is a second taken from the caller.

The operator wanted AI that would take on the typing and the searching without taking away the operator's judgment, in Japanese, at production quality, inside real working centres.

#What we built

01 System A: transcription and extraction for recorded calls

The first production system handles recorded audio and text. Operators or supervisors upload a recording or a transcript, the system produces a Japanese transcription, extracts structured information from it, and exports the result to Excel for the client's existing reporting. This phase proved the transcription and extraction quality on real calls before anything ran live.

02 System B: live calls and the 16-field emergency form

The second system works during the call. It captures live audio, separates the speakers so the operator's questions and the caller's answers are distinguished, and fills the sixteen fields of the emergency consultation form as the conversation happens. The operator reviews the drafted form, corrects anything the AI has wrong and saves it, and the system produces an end-of-call summary. The operator spends the call listening instead of typing.

03 System C: a bilingual knowledge base for a major international event

For the event support service, we built a knowledge base that answers questions in English and Japanese. As a call progresses, it suggests relevant answers from approved content, and operators can also search it directly. Translated embeddings let a question asked in one language find content written in the other. Operators and content managers maintain the content themselves, and questions the knowledge base could not answer are tracked, so the content grows where callers actually need it.

04 System D: realtime call-centre assistance

The fourth system provides realtime transcription and supervised AI support for day-to-day operations in the contact centre, with a person supervising its output.

05 Recognition engineering after deployment

Once the live system was in use, the hardest remaining problem was local vocabulary: place names, landmarks and regional terms that general speech models often mishear. We added an administrator-managed vocabulary for each region, deterministic corrections that apply only when the context makes them safe, an audit of recognition results, and automated regression tests. The latest of these enhancements has been verified in our environment and is being validated on the client's servers before it is treated as live.

#How it works on a live call

A caller rings the consultation line. As the operator answers, the system begins transcribing in Japanese, labeling who is speaking. When the caller describes symptoms, location and timing, those details appear in the draft form in the right fields. The operator keeps their attention on the caller, asking follow-up questions and giving advice.

When the call ends, the operator checks the draft. A misheard place name is corrected in a few seconds. The form is saved, and an end-of-call summary is attached. Typing that used to happen while trying to listen, or after the caller had hung up, has become a review.

On the event line, a visitor asks in English where to find a venue service. Before the operator has finished listening, the knowledge base has suggested an approved answer, drawn from content that may have been written in Japanese. If no answer exists, the question is logged for the content team to address.

#Why production AI in a contact centre is different

A prototype can transcribe a clear recording in a quiet room. A contact centre has accents, background noise, stressed callers, overlapping speech and a stream of local names, and it runs every day. Getting from the first to the second is most of the work.

Three principles carried the project. First, the human stays in charge of anything that matters: the AI drafts and suggests, and the operator decides. Second, quality is engineered and tested continuously, especially for vocabulary, with regression tests so that each improvement is protected. Third, each system ships in a form that is useful on its own, so the operator gains value at every phase and the next phase builds on proven ground.

#The architecture behind the four systems

All four systems share a common foundation, which is part of why each new one could be delivered on proven ground.

Each is a web application that operators use in the browser, backed by cloud infrastructure on AWS and a database that stores transcripts, extracted forms, summaries, review decisions and audit records. Speech recognition models handle Japanese transcription and speaker separation, and language models handle extraction, summarization and question answering, each constrained to the task in front of it.

The knowledge base adds vector search. Approved content is converted into embeddings, including translated embeddings, so a question asked in English can retrieve an answer written in Japanese and the other way round. Answers come only from content the operator's team has approved, which keeps suggestions grounded in material the client stands behind.

Around the models sit the parts that make the systems operational: review screens for operators, content management for the knowledge base, administrator tools for regional vocabulary, logs that show what the AI produced and what the operator changed, and automated tests that run whenever recognition rules change. Those parts are less visible than the models, and they are what make the systems dependable in daily use.

#How we delivered it, end to end

Recorded before live. System A established transcription and extraction quality on recorded calls before System B ran on live ones.

Human review designed in. The review step on every emergency form was part of the specification, not a safeguard added later.

Content ownership with the operator. The knowledge base gives operators and content managers the tools to maintain answers, so the client controls what the AI suggests.

Quality work after go-live. Recognition engineering continued after deployment, driven by real errors seen in production.

Honest status for each release. Enhancements are verified in our environment and validated on the client's servers before they are called live.

#The decisions that mattered

Assist, never replace. Keeping the operator in control is what allowed AI into an emergency service at all.

Engineer the vocabulary. General models were not enough for local names; managed vocabulary and context-gated corrections were.

Track the gaps. Logging unanswered questions turned the knowledge base into something that improves with use.

#What changed

#What we would tell a contact centre starting this

Choose the task your operators spend the most time on that does not need their judgment, usually typing and searching, and let AI take that first. Keep a person in charge of every record that matters. Then invest in the unglamorous quality work, local vocabulary and regression tests above all, because that is what turns an impressive demonstration into a system operators rely on every day.