Peter Yang Builds a Voice-Based Japanese Tutor in Six Steps

I build a phone-friendly AI language coach for a trip to Japan. It teaches ten phrases per lesson and then runs a spoken role-play in which the learner practices them. The video is sponsored by Google and uses its live voice and image-generation tools.

The demo shows a learning path with ten lessons and illustrated scenes. In a call with the coach, the learner hears phrases, repeats them and then practices in imagined situations. I say the same approach could be adapted for other languages.

First I explore the idea with an AI assistant. My prompt specifies 100 useful travel phrases in ten lessons, a progress screen, a lesson view and live voice. I provide an existing app as a visual reference and ask the assistant to research and ask three questions without building yet.

The questions clarify the experience: teach the phrases first, then role-play; show Japanese, romanization and English; and initially build for me and a few friends because API calls have a cost. That discussion settles the basic product shape.

Next I ask for a specification with requirements, technology choices and a few representative screens. Seeing mobile and desktop mockups lets me react to the design, not just read a document. I request an onboarding screen and desktop versions.

I use a review interface to edit text and leave feedback directly on the mockups. My own specification tool is paid, while the review tool is open source; I suggest that a creator can also ask an assistant for screens and requirements without that specific tool.

With a reviewed specification, I ask an agent to build the first version. The voice API requires a key. I enter it into a local environment file rather than pasting it into chat, and delete the demonstration key after showing the setup.

Testing reveals that the app listens to the wrong microphone. The agent changes the input, and I continue testing the actual interaction. I also find that all phrase bubbles appear at once instead of as the lesson progresses, and that responses feel slow.

I ask for generated diorama-style art for the lessons. After reviewing a first image, I request a consistent set for all ten scenes. I also revise the coach’s avatar after the first version depicts a rice ball rather than the character I intended.

I say the interface iteration took around thirty minutes, but the lessons themselves matter just as much. I discuss ten travel topics with the assistant, combine or reorder lessons and avoid ending on an emergency-themed lesson.

In a separate content thread I review the coach script and the phrases for each lesson. I notice one document still describes an older avatar, showing that written material can lag behind the design. The most useful test is to take the lessons myself and invite others to try them.

After the app mostly works, I deploy it using my existing hosting account. The build encounters outdated libraries and other issues, which are addressed during deployment. The resulting demo has a passcode because the voice API incurs usage costs.

I recap the six stages: explore, specify and design, build, improve the app, improve its content and launch. I estimate about three hours in total, with roughly two spent iterating and testing. I see that care for both behavior and lessons as the difference between a quick prototype and a useful learning app.