# Four Production AI Systems for a Japanese Contact Centre Operator

> Operators were transcribing emergency consultation calls by hand and searching documents while callers waited. We built four production AI systems: recorded and live Japanese transcription, a 16-field emergency form filled during the call with human review, a bilingual knowledge base, and realtime operator support.

- Source: https://synconai.com/case-studies/contact-centre-production-ai-in-japanese
- Publisher: SynconAI (https://synconai.com)
- Type: Case study
- Evidence: Delivered client engagement. The client is anonymised; the industry, products and outcomes are theirs.
- Organisation: A Japanese information and contact centre operator running emergency consultation lines and multilingual event support services for public sector clients
- Industry: Communications, Media & Technology
- Products: Custom AI, Speech-to-text, Large language models, Vector search, AWS, Web application
- Service: Custom AI Development (https://synconai.com/custom-ai-development)
- Reading time: 6 minutes

## In short

We built four production AI systems for a Japanese contact centre operator. They transcribe recorded and live Japanese calls, fill a 16-field emergency consultation form during the call for an operator to review, answer questions from a bilingual English and Japanese knowledge base, and support operators in real time, with a person approving every record.

## Key points

- In a contact centre, the AI drafts and the operator decides. Human review on every emergency form is what made production use possible.
- Local vocabulary is where speech recognition succeeds or fails. Place names and regional terms need their own managed vocabulary and their own tests.
- Track the questions the knowledge base cannot answer. Missing questions are the best content backlog an operations team can have.
- Ship in phases that each stand alone. Recorded transcription came before live calls, and each system was useful in production before the next began.

## Live call to reviewed record

A call arrives and its audio is captured live. Speech recognition tuned for Japanese, with speaker separation, produces a running transcript. An extraction step reads the transcript and fills the 16 fields of the emergency consultation form as the conversation happens. The operator reviews and corrects the form before it is saved, and an end-of-call summary is produced. Alongside the call, a bilingual knowledge base suggests answers from approved content, and operators can search it directly. Questions the knowledge base could not answer are tracked so content managers can fill the gaps.

### 1. Capture

- Live call audio
- Recorded audio upload
- Text upload

The same pipeline handles live and recorded calls.

### 2. Transcribe

- Japanese speech recognition
- Speaker separation
- Regional vocabulary

Accuracy on local place names decides whether operators trust it.

### 3. Extract

- 16-field emergency form
- Structured fields
- Excel export

The AI drafts the record while the call is still going.

### 4. Review

- Operator review
- Corrections
- End-of-call summary

A person approves every record.

### 5. Assist

- Bilingual knowledge base
- Live suggestions
- Missing-question tracking

Approved answers, in English or Japanese, when they are needed.

## The starting position

The operator runs contact centres for public sector clients in Japan. One of its services is an emergency consultation line: people call when they are unsure whether a medical situation needs an ambulance, and trained operators assess the situation and advise them. Another is multilingual information support for large events, where operators answer visitors' questions in Japanese and English.

Both services depend on operators doing several things at once. On a consultation call, the operator listens, asks the right questions, decides on advice and records the call in a structured form with sixteen fields, all while a worried person is on the line. On the event line, the operator listens to a question, searches reference material and answers, often switching between languages. Typing and searching compete with listening, and every second spent on them is a second taken from the caller.

The operator wanted AI that would take on the typing and the searching without taking away the operator's judgment, in Japanese, at production quality, inside real working centres.

## What we built

### 01 System A: transcription and extraction for recorded calls

The first production system handles recorded audio and text. Operators or supervisors upload a recording or a transcript, the system produces a Japanese transcription, extracts structured information from it, and exports the result to Excel for the client's existing reporting. This phase proved the transcription and extraction quality on real calls before anything ran live.

### 02 System B: live calls and the 16-field emergency form

The second system works during the call. It captures live audio, separates the speakers so the operator's questions and the caller's answers are distinguished, and fills the sixteen fields of the emergency consultation form as the conversation happens. The operator reviews the drafted form, corrects anything the AI has wrong and saves it, and the system produces an end-of-call summary. The operator spends the call listening instead of typing.

### 03 System C: a bilingual knowledge base for a major international event

For the event support service, we built a knowledge base that answers questions in English and Japanese. As a call progresses, it suggests relevant answers from approved content, and operators can also search it directly. Translated embeddings let a question asked in one language find content written in the other. Operators and content managers maintain the content themselves, and questions the knowledge base could not answer are tracked, so the content grows where callers actually need it.

### 04 System D: realtime call-centre assistance

The fourth system provides realtime transcription and supervised AI support for day-to-day operations in the contact centre, with a person supervising its output.

### 05 Recognition engineering after deployment

Once the live system was in use, the hardest remaining problem was local vocabulary: place names, landmarks and regional terms that general speech models often mishear. We added an administrator-managed vocabulary for each region, deterministic corrections that apply only when the context makes them safe, an audit of recognition results, and automated regression tests. The latest of these enhancements has been verified in our environment and is being validated on the client's servers before it is treated as live.

## How it works on a live call

A caller rings the consultation line. As the operator answers, the system begins transcribing in Japanese, labeling who is speaking. When the caller describes symptoms, location and timing, those details appear in the draft form in the right fields. The operator keeps their attention on the caller, asking follow-up questions and giving advice.

When the call ends, the operator checks the draft. A misheard place name is corrected in a few seconds. The form is saved, and an end-of-call summary is attached. Typing that used to happen while trying to listen, or after the caller had hung up, has become a review.

On the event line, a visitor asks in English where to find a venue service. Before the operator has finished listening, the knowledge base has suggested an approved answer, drawn from content that may have been written in Japanese. If no answer exists, the question is logged for the content team to address.

## Why production AI in a contact centre is different

A prototype can transcribe a clear recording in a quiet room. A contact centre has accents, background noise, stressed callers, overlapping speech and a stream of local names, and it runs every day. Getting from the first to the second is most of the work.

Three principles carried the project. First, the human stays in charge of anything that matters: the AI drafts and suggests, and the operator decides. Second, quality is engineered and tested continuously, especially for vocabulary, with regression tests so that each improvement is protected. Third, each system ships in a form that is useful on its own, so the operator gains value at every phase and the next phase builds on proven ground.

## The architecture behind the four systems

All four systems share a common foundation, which is part of why each new one could be delivered on proven ground.

Each is a web application that operators use in the browser, backed by cloud infrastructure on AWS and a database that stores transcripts, extracted forms, summaries, review decisions and audit records. Speech recognition models handle Japanese transcription and speaker separation, and language models handle extraction, summarization and question answering, each constrained to the task in front of it.

The knowledge base adds vector search. Approved content is converted into embeddings, including translated embeddings, so a question asked in English can retrieve an answer written in Japanese and the other way round. Answers come only from content the operator's team has approved, which keeps suggestions grounded in material the client stands behind.

Around the models sit the parts that make the systems operational: review screens for operators, content management for the knowledge base, administrator tools for regional vocabulary, logs that show what the AI produced and what the operator changed, and automated tests that run whenever recognition rules change. Those parts are less visible than the models, and they are what make the systems dependable in daily use.

## How we delivered it, end to end

**Recorded before live.** System A established transcription and extraction quality on recorded calls before System B ran on live ones.

**Human review designed in.** The review step on every emergency form was part of the specification, not a safeguard added later.

**Content ownership with the operator.** The knowledge base gives operators and content managers the tools to maintain answers, so the client controls what the AI suggests.

**Quality work after go-live.** Recognition engineering continued after deployment, driven by real errors seen in production.

**Honest status for each release.** Enhancements are verified in our environment and validated on the client's servers before they are called live.

## The decisions that mattered

**Assist, never replace.** Keeping the operator in control is what allowed AI into an emergency service at all.

**Engineer the vocabulary.** General models were not enough for local names; managed vocabulary and context-gated corrections were.

**Track the gaps.** Logging unanswered questions turned the knowledge base into something that improves with use.

## What changed

| Area | Before | After |
| --- | --- | --- |
| Recorded calls | Transcribed by hand | Japanese transcription, extraction and Excel export |
| Emergency forms | Typed during or after the call | 16 fields drafted live, reviewed by the operator |
| Call records | Written from memory | End-of-call summaries attached |
| Event questions | Manual document search | Bilingual knowledge base with live suggestions |
| Local names | Frequently misheard | Managed regional vocabulary and regression tests |
| Content gaps | Unknown | Missing questions tracked for content managers |

## What we would tell a contact centre starting this

Choose the task your operators spend the most time on that does not need their judgment, usually typing and searching, and let AI take that first. Keep a person in charge of every record that matters. Then invest in the unglamorous quality work, local vocabulary and regression tests above all, because that is what turns an impressive demonstration into a system operators rely on every day.

## Questions

### Does the AI make decisions on emergency calls?

No. The system transcribes the call and drafts the 16-field form while the conversation happens, and it produces an end-of-call summary. The operator reviews and corrects the form before it is saved. The AI removes typing and searching; judgment stays with the trained operator.

### How do you handle Japanese place names and local terms?

Through recognition engineering after deployment: an administrator-managed vocabulary for each region, deterministic corrections that only apply in the right context, an audit of recognition errors, and automated regression tests so a fix for one term does not break another. The latest enhancement has been verified in our environment and is being validated on the client’s servers.

### Is this built on Salesforce?

No. These are custom AI systems built as web applications with cloud infrastructure, a database and vector search. The same architecture principles apply when the record of truth is Salesforce, and we use them in Service Cloud and Agentforce work.

---

SynconAI. https://synconai.com/case-studies/contact-centre-production-ai-in-japanese