Speech synthesis and voices

Voice input uses NLS sentence recognition separately: open the unified settings with /settings and select the "Voice input" tab. It needs its own AppKey and Alibaba Cloud AccessKey ID / Secret, and the NLS token is issued automatically. See speech recognition for details.

Using it in the mod

In the full interface, click "Settings", or type /settings in the assistant input box, then select the "Speech narration" tab. The settings command has only this one entry point; it is a client interface command, not a server command in the vanilla chat bar. All tabs keep using the single assistant.css, and IME handling is shared with the assistant input box. Changes to controls are saved automatically right away, so switching tabs or closing the page needs no extra save, and leaving the narration tab stops any preview.

1. Enter the Alibaba Cloud Bailian API key and select the region that matches the key (Beijing by default). You can also leave the key empty and set DASHSCOPE_API_KEY before starting the client. Voice settings become available only when a key is present.
2. The default model is qwen3-tts-flash and the default voice is Cherry. You can also enter system voices such as Serena and Ethan. Leave the Workspace ID empty to use the compatibility domain; once it is filled in, the dedicated domain of that workspace is used.
3. Set the language and volume, then click "Preview". Parameters are already saved automatically, and previewing does not require enabling auto narration first. Incomplete credentials may be kept while you are still filling them in; the full parameters are validated on preview or actual synthesis.
4. Enabling "Auto narration" saves it automatically; "Stop narration" cancels the current synthesis and playback.

The "Notification sound" option on the same page is on by default, independent of "Auto narration", and needs no API key. Turning it off only removes the local cue when a notification expands; the body still plays according to the auto narration setting and no longer waits for the cue. It maps to notificationSoundEnabled in the client config/system/system-speech.json; existing files without the field default to on, and a toggle is saved immediately.

Speech is off by default. config/system/system-speech.json is a client-local configuration, separate from the server's system-agent.json and the interface mode configuration. A key entered and saved in the interface is stored in plain text in that local file; a key from the environment variable fallback is never written to the file or filled into the page. Keys are not sent to the MC server and do not enter conversations, attachments or world saves. Once enabled, the body of newly arriving System messages is sent to the selected speech service for synthesis; the chat bar, other mods and player chat are not in scope.

"Follow game language" maps the language from the client's current language (accepting forms such as en_us and en-US), and falls back to Auto for unsupported languages. You can also choose automatic detection or specify a language explicitly. Qwen3 uses input.language_type; Qwen-Audio-TTS uses the official input.language_hints, for example ["en"] for English and ["zh"] for Chinese, and the field is omitted for automatic detection. Streaming and non-streaming share the same mapping. Fixed language keys are first translated to text through the client's Component.translatable and then synthesized; the Chinese and English configuration page, preview texts and error messages all have language keys.

For mixed Chinese and English, "Automatic detection" is recommended; for a whole passage of English, choose "English". "Follow game language" still provides a Chinese hint on a Chinese client and is not the same as detecting the body. According to the Qwen-Audio-TTS HTTP API, the target language hint improves how numbers, abbreviations and symbols are read, and it is independent of the source language of the cloned sample. The old Qwen-Audio adapter sent no language hint at all, so Chinese in an old configuration had no actual effect on that protocol; it alone cannot be used to conclude why Dev is read letter by letter. English is currently not split apart, and names or abbreviations are not forcibly rewritten; the exact pronunciation still has to be confirmed by previewing the actual voice.

Speech holds a separate session per interface or current notification, and the full interface, simple mode and the HUD all produce narration through speech/tts/SpeechText. Ordinary results read the CardView's SpeechContent and fall back to summary when it is not set; nothing is inferred from messageKey or Java type, and template fields or buttons are not concatenated. Quest completion, unlocking and details add quest content according to the settings; the first overview reads only the quest count, and the starter gift reads only the generic claimable hint, not the source quest name or individual items. Claiming a gift refreshes silently; a quest completed after an explicit submission also does not read a submission receipt separately, and is announced uniformly by the completion event. Binding narration keeps the "ding!", the actual system name and the introduction, and the fixed "System" in the top-left corner is not narrated. Help only says the manual has been expanded, history and state sync stay silent, and no inference is made that rewards have arrived.

The settings page offers three toggles, "Narrate quest details", "Narrate quest rewards" and "Narrate quest background", mapping to questRequirements, questRewards and questDescription in the client system-speech.json; the first two are on by default and the background is off. With details off, only the quest name and the rewards allowed to be narrated are read; when it is on, requirements are appended, and the background must additionally be enabled on its own and the source text must actually contain narrative. Empty descriptions, crafting or submission steps, key instructions, mechanic explanations and undeterminable content are never read as background; from mixed content only the existing narrative is extracted, and no story is written in. Ordinary item objectives, count ranges, experience, choices and random rewards are formatted directly, and optional objectives are marked explicitly; item names keep the vanilla Component and are expanded in the client locale. Descriptions always respect FTB's hide-before-completion setting.

Quest narration material is delivered directly with CardData; there is no longer a client packet for querying quest narration. The server provides structured requirements and rewards when preparing business cards, and organizes unfamiliar quests/rewards and background descriptions in the background; common types do not call the model. Source reads re-check the module, visibility and detail permission on the main thread, and the client cannot supply raw quest text or model prompts. Cards and speech do not wait for organization to finish; on a miss they first use the structured/native text, the background cache serves later cards, and the current narration does not append late paragraphs. Model organization may happen during card preparation, independent of whether narration is enabled on the client; it still follows the external AI preference, player quota, concurrency limit and unified usage accounting, and after external AI is disabled, existing cache can be used while uncached parts keep the original text.

Organized fragments for special objectives and rewards are cached by source text, language, model address/name and the rendered prompt, still using an in-memory LRU of at most 512 entries per server instance; concurrent requests in the same batch are merged, and a restart clears it. Background filtering is managed separately by QuestBackgrounds, saved to the system:quest_backgrounds table (GZIP-compressed NBT) in the world save system/database/shared.dat, keyed by quest unique ID and language, and checked against a content fingerprint of the quest title, original description, prompt, model and endpoint. A successful "no background" result is also persisted, while a failure only suppresses retries for 30 seconds; concurrent generation for the same quest is shared. On a hit, the quest is re-read and its visibility and source content are checked, so hidden conditions cannot be bypassed. The prompts are data/system/system/prompts/quest_narration.txt and quest_background.txt, and support data pack overrides and /reload. Neither cache stores player progress, reward claim state or executable operations.

Queries, narration and text organization never submit quests. Manual objectives are handled only after the player explicitly sends /submit 1, /submit <quest name> or the vanilla chat command /system submit <number|name>; the number refers to the most recent quest list. The submit tool supports native item consumption, checkboxes, experience and manual custom button objectives, while other specialized inputs are still performed in FTB; pausing, prerequisites, ordering and FTB automatic rewards are preserved, and one command does not enter the next round after an automatic reset. Gifts still have to be claimed explicitly. Card display does not wait for background organization; switching or closing a notification cancels local audio, and a background completion does not replay it. Read-only organization already started on the server may finish the cache and actual metering, and recalling cloud requests already sent is not guaranteed.

When the interface receives a new reply it replaces the old narration, and a new request, closing the interface, switching modes or entering another page cancels the original session. HUD messages such as binding prompts and FTB completion/unlocking are still shown in order by a queue of at most 32 notifications, but no longer pile up speech independently: when the current notification's turn comes, one narration is submitted, and confirming, performing an action, or hiding it by another interface or F1 immediately cancels its request and playback. Notifications have no auto-close timer; after narration ends they stay displayed and wait for an explicit action, and manual switching does not wait for speech to finish. Restoring a hidden notification restores only the text, without replaying speech; history, state sync and document rebuilding also do not replay. Preview replaces the current narration; "Stop narration" stops the current audio and clears pending speech without emptying the text notification queue.

When the HUD notification box expands from a hidden state for the first time, it plays the registered system:notification cue once and then plays the body narration. Consecutive messages keep it expanded, only switching the body narration, without repeating the cue; a message received while it is collapsing and re-expanding in the opposite direction also does not replay it, and only the next round of notifications prompts again after it is fully collapsed. The sound source is assets/system/sounds/notification.ogg, converted from the supplied MP3 to mono Ogg Vorbis, about 1.78 seconds and 16 KiB; assets/system/sounds.json defines and preloads the sound, with Chinese and English subtitles. The cue uses the Voice category, with a gain of 75% of the page narration volume, multiplied by the game master volume and the Voice volume. The local cue still plays when auto narration is off, no key is present, the body is muted or the cloud service is cooling down; when the narration volume is zero, or the master or Voice volume is muted, neither plays.

The cue and the body share the session of the current notification. While the cue plays, streaming synthesis or cache reads are prepared in parallel, but the body waits for the vanilla sound instance to actually finish, judged by its real active state, before starting, rather than after a fixed delay, without extra cloud requests or usage. If the cue is missing or cannot start, the normal narration flow continues; confirming early, closing, switching messages, hiding the interface, disconnecting and a sound reload all stop the cue, in-flight synthesis and the body together, and a delayed decode callback cannot keep the old cue playing. Notification restoration and document rebuilding do not replay the cue; ordinary replies in the assistant interface and previews on the settings page keep the original direct playback.

Narration is converted to plain text in the client language, at most 500 UTF-16 units (protecting surrogate pairs), optionally with an appended ellipsis. The HUD's Message.text keeps the full visible body, the long body and the gift item area scroll separately, and it is no longer truncated at 360 code points / 6 lines; Message.speechText stores the narration separately. The original four-argument constructor reads the custom notification body by default, and the five-argument constructor can specify the narration or leave it empty for silence. Audio goes through MC's Voice volume category first and is also controlled by the master volume and the page volume; when muted, no synthesis is requested, and muting during playback ends the current audio without triggering fallback playback. A failure of synthesis or of either playback path does not affect the text, quests or rewards, and the same message is not retried; the speech queue is cleared and auto narration is paused for 60 seconds, after which later new messages may try again, and manual preview can still retry immediately.

Optional full-sentence audio cache

In the speech settings, "Reuse synthesized speech" is off by default, matching cacheEnabled: false in config/system/system-speech.json; an old configuration without the field is also off. When enabled, identical synthesis input checks the local audio first and calls the provider only on a miss, keeping the original SSE first-packet playback. The cache is written only when the provider finishes successfully, the PCM is complete and the session was not cancelled, so a failure or a mid-way close leaves no half-playable cache behind. Previews also follow the current form's toggle.

The cache directory is config/system/tts-cache/, using files that carry a version, the PCM format and a SHA-256 checksum; at most 64 MiB / 256 entries, evicted by least recent use when written. The cache key covers the provider, model, voice, region, workspace, actual language, actual narrated text and credentials, while the file name stores only a digest, not plain-text keys or text. Changing the credential uses a different key set, and the page volume does not affect the cache. The audio itself is stored on the local disk and may contain player names; turning the cache off does not delete old audio, and deleting the whole cache directory clears it.

Reads, writes and checks all happen on the original synthesis worker thread; a corrupt or unreadable cache entry counts as a miss, and a write failure does not interrupt narration. On a hit, the same session pipeline for vanilla / OpenAL is used directly, it can still be cut off immediately, and it does not report cloud usage or increase the TTS request count. The cache reuses only audio and does not cache quests, rewards or business results.

SSE streaming synthesis and playback

Alibaba Cloud now uses SSE streaming output by default. Both qwen3-tts-flash and Qwen-Audio-TTS send X-DashScope-SSE: enable, and the latter also specifies format=pcm and sample_rate=24000; both protocols consume the Base64 PCM in output.audio.data (24 kHz, 16-bit, little-endian, mono). The full text is still submitted only once, without splitting sentences into multiple calls; received audio is appended chunk by chunk instead of waiting for the whole synthesis or downloading the final OSS URL. See the Qwen3-TTS API and the Qwen-Audio-TTS HTTP API.

SpeechAudio stores PCM in length-limited segmented memory, and playback can start once about 200 ms of audio has accumulated; this is the length of audio waiting to be played, not a fixed 200 ms wait, so a short sentence starts immediately even below the threshold. Cloud first-packet and network latency still exist, so no fixed delay from text display to sound can be guaranteed. The vanilla path uses SoundManager.play and NeoForge's SoundInstance.getStream, reading at most about 50 ms of available PCM at a time; when the network temporarily has no data it keeps the stream alive with a short silence, and returns EOF only after an end marker or a cancellation. Minecraft's current Channel.pumpBuffers explicitly skips null, so null is a stream-end signal, not audio data passed to OpenAL; all end decisions are handled by SpeechAudio.Cursor, and the sound thread does not wait for the network. At most 8 MiB / 180 seconds of PCM are stored, a half frame spanning SSE fragments is kept for the next fragment to complete, and the stream must be frame-aligned at the end.

TtsProvider.supportsStreaming() defaults to false. A new provider can implement synthesizeStreaming(Request, AudioSink, Consumer<Usage>), providing the sample rate, channels and 16-bit little-endian PCM through the AudioSink and reporting accumulated usage. Old providers without streaming keep the full WAV synthesis, which is decoded on the worker thread and enters the same playback pipeline; a failed Alibaba Cloud streaming request is not automatically resent as non-streaming, to avoid being charged twice.

HttpTransport.stream applies a 90-second deadline and a 16 MiB total response limit to the whole stream; ServerSentEvents frames UTF-8 incrementally, supporting CR/LF, comments, multi-line data, a BOM and characters split across network chunks, with a 2 MiB limit per event. It validates the HTTP status, the SSE Content-Type, JSON errors and the final finish_reason=stop, and playback never exposes upstream body text or signed URLs to players. The end of a session, stopping narration, a disconnect, a revoke or a resource reload cancels in-flight requests and releases the PCM, and stale callbacks are guarded by session and generation. Usage already received still counts toward the local account that made the request; cancelling the local connection does not guarantee that the cloud side stops generating or billing immediately.

Vanilla playback and the OpenAL fallback

PlayStreamingSourceEvent checks on the sound thread whether the actual sound source has entered playback; when vanilla returns NOT_STARTED, starting throws, the source is not playing or it has not started within 3 seconds, OpenAL is switched in exactly once. After vanilla starts successfully the sound is treated as ending naturally, and it is not replayed when it finishes normally.

The fallback path follows E:/Java/BGMPlayer/common/src/main/java/com/iafenvoy/bgm/player/music/AudioPlayer.java and creates a source inside MC's current OpenAL context, uploading, playing and refilling the same PCM stream with 4 loopable buffers, bypassing the MC sound source pool. When there is no data it waits for further ticks, and after the buffers run dry it resumes from the original position once new data arrives, without requesting again or replaying the beginning; it ends only after the producer ended and the buffers are drained. Without an available context or hardware source it reports playback failure. The status bar distinguishes vanilla playback from "Narrating (OpenAL)".

Before entering the fallback, the vanilla stream and the delayed start callback are cancelled to avoid double playback; the fallback reads from a separate PCM cursor, so vanilla pre-reading does not swallow the beginning. The fallback volume is synchronized every tick as the page volume × master volume × Voice volume, and it pauses when a single-player world is paused and continues after resuming. The source and buffers are created, queried and destroyed on the same render thread, and the source/buffers are released when playback ends or is cancelled. The client-only SpeechSoundEngineMixin cleans up speech before vanilla stopAll / emergencyShutdown, so leaving a world, revoking a binding, turning off auto narration, globally stopping sounds, switching devices, reloading resources and quitting the client all leave no fallback source or delayed playback behind. A sound system reload discards the old speech queue without replaying it.

Matching designed voices to models

According to the current official documentation, the non-realtime HTTP interface of qwen3-tts-flash uses system voices; voice-enrollment in voice design creates Qwen-Audio-TTS / CosyVoice voices, and the returned voice_id is bound to the target_model it was created with. An arbitrary designed voice cannot be entered directly for the default model.

This update additionally integrates the non-realtime HTTP interface of Qwen-Audio-TTS, supporting qwen-audio-3.1-tts-flash, qwen-audio-3.1-tts-plus, qwen-audio-3.0-tts-flash and qwen-audio-3.0-tts-plus. The dedicated workspace domain can be used, or the compatibility domain when the Workspace ID is empty. The default qwen3-tts-flash is unchanged. CosyVoice synthesis is not part of this implementation.

The system-voice.json exported by the tool contains provider, region, workspace, model, voice and language, and no key, for the tool to keep synthesizing or for manual reference. In the mod, fill in the region, Workspace ID, model, voice ID and language by hand in the "Speech narration" tab of /settings; changes are saved automatically, then click preview. The mod offers no JSON import or in-page reload button; regular persistence still uses config/system/system-speech.json. A newly created voice may be under review; check its status with the tool's query, and it can be used normally only once it is OK.

CosyVoice and current model price comparison

Beijing region official public unit prices checked on 2026-10-25, excluding free quota and discounts:

Model	Synthesis price
qwen-audio-3.1-tts-flash, currently used for the cloned voice	CNY 1.5 per million input tokens, CNY 12 per million output tokens
cosyvoice-v3.5-flash	CNY 0.8 per 10,000 characters
cosyvoice-v3.5-plus	CNY 1.5 per 10,000 characters
cosyvoice-v3-flash	CNY 1 per 10,000 characters
cosyvoice-v3-plus / cosyvoice-v2	CNY 2 per 10,000 characters

CosyVoice bills only by input text: one Chinese character counts as 2 billable characters, while other letters, punctuation and spaces count as 1. In Singapore, cosyvoice-v3-plus / cosyvoice-v3-flash cost CNY 1.9082 / 0.9541 per 10,000 characters respectively; the table above lists no Singapore prices for 3.5 and v2, so the Beijing prices cannot be applied directly.

According to the voice cloning guide, creating a cloned voice for Qwen-Audio-TTS and CosyVoice is free, and synthesis with that voice is billed at the target model's price. Do not confuse this with the per-voice billing of the separate qwen-voice-enrollment. The current system-test1 is bound to an Audio model; switching to CosyVoice requires creating the voice again for the target model.

An earlier test actually returned 37 input tokens, 81 output tokens, 118 tokens in total and 69 characters; at the current Audio model price that is about CNY 0.0010275. CosyVoice 3.5 Flash with the same number of billable characters would be about CNY 0.00552, which only describes the price difference of this sample and does not mean all text is billed at the same ratio. The Java mod does not integrate CosyVoice synthesis yet, and this table is only a selection reference.

Python helper tool

tools/voice_design.py uses the Python 3.10+ standard library and needs no pip package. --lang zh/en switches the language of the help and tool tips. The API key is read from DASHSCOPE_API_KEY, and hidden input is used when it is missing; placing the key directly in a command-line argument is not supported. The tool does not automatically retry billable requests.

First set the Bailian key in the current terminal, replacing YOUR_WORKSPACE_ID below with the actual workspace ID; you can also omit --workspace to use the compatibility domain. --workspace can also be provided by DASHSCOPE_WORKSPACE_ID; common options go before the subcommand:

python tools/voice_design.py --lang zh --help
python tools/voice_design.py --lang zh --workspace YOUR_WORKSPACE_ID design --prompt "A calm, restrained young female electronic assistant, clear articulation, steady tone, with a slight mechanical feel" --prefix system

The default target is qwen-audio-3.1-tts-flash, with a Chinese preview by default. The voice description is 1–500 characters, the preview text 15–200 characters, and the prefix 1–10 English letters/digits. An English design also needs an English preview:

python tools/voice_design.py --lang en --workspace YOUR_WORKSPACE_ID design --prompt "A calm, clear young female virtual assistant" --voice-language en --preview-text "Host, the system link is established. A new quest is ready."
python tools/voice_design.py --lang zh --workspace YOUR_WORKSPACE_ID list
python tools/voice_design.py --lang zh --workspace YOUR_WORKSPACE_ID query VOICE_ID
python tools/voice_design.py --lang zh --workspace YOUR_WORKSPACE_ID delete VOICE_ID

The models listed in the current official voice design documentation use the Beijing region, and the tool's voice operations are also restricted to Beijing. --region singapore is used for temporary uploads; keys or voices from different regions cannot be mixed. delete requires entering the full voice ID to confirm.

design writes preview.wav and system-voice.json under voice-output/<time>-<random id>/; --output sets the output root directory. The tool creates the output location before calling the API, so that an unwritable path does not create a voice that cannot be saved. Save the voice ID and configuration before decoding the preview; if the preview is abnormal, do not create a billable voice again, but check it with query first. The preview WAV can simply be opened in a local player.

Cloning a voice from a recording

clone accepts 16-bit PCM WAV (at least 5 seconds, no more than 60 seconds, sample rate at least 16 kHz, mono or stereo, no more than 10 MiB). The official recommendation is 10–20 seconds of clear continuous speech. Floating-point WAV must be converted to 16-bit PCM first; the original audio is not modified. Cloning is also voice-enrollment/create_voice, passing the reference audio as audio, not a text voice design.

python tools/voice_design.py --lang zh --config run-test-client/config/system/system-speech.json clone voice-input/system-test1.wav --id system-test1

--config only reads the key, region and Workspace ID from the specified client configuration and does not print the key; the environment variables still work when there is no configuration file. The default target model is qwen-audio-3.1-tts-flash.

Alibaba Cloud generates the voice ID automatically; it cannot be customized as system-test1. --id keeps a local alias and writes voice-output/system-test1/system-voice.json, where voice is the real cloud ID. A cloud prefix supports at most 10 English letters and digits; when --prefix is not given explicitly it is generated from the local ID, and system-test1 maps to systemtes1. Enter the exported voice parameters into the mod's speech narration settings; changes are saved automatically.

Before submitting, submission.json is written to disk, keeping the local ID, prefix, target model, audio digest and submission state, without keys or Base64 audio. If a network failure leaves the result unknown, the record is kept and reusing that ID refuses another submission; run list by prefix first to avoid being charged twice. After a voice ID is returned, use query to confirm the status is OK. No extra billable speech synthesis is requested automatically for verification.

When you need to preview the cloned result, run synthesize explicitly; it calls synthesis once for the target model and downloads the WAV, also saving the text, request ID and usage metadata:

python tools/voice_design.py --lang zh synthesize --profile voice-output/system-test1/system-voice.json --output voice-output/system-test1/test.wav

An existing output file is never overwritten, to avoid accidentally synthesizing twice; a failure is not retried automatically. Audio downloads carry no Bearer token, and HTTP OSS URLs are upgraded to HTTPS. After generation, the metadata drops the download signature URL and the inline audio.

The WAV returned by some services keeps a streaming placeholder length, so players may report several hours. The tool rebuilds a standard WAV header from the actual PCM bytes without changing the sample data or requesting synthesis again.

Temporary upload

Text voice design and ordinary text synthesis both need no file upload. The tool also offers the official temporary upload flow for later reference-audio / cloning integration:

python tools/voice_design.py --lang zh upload reference.wav --model voice-enrollment

It first obtains the upload policy for the specified model, then uploads to OSS with a signed form and returns oss://... valid for 48 hours; the policy itself is short-lived, so use it immediately. Later model requests that reference this URL must set X-DashScope-OssResourceResolve: enable. Uploading does not create or clone a voice; whether a model accepts the corresponding audio format and duration is still up to that model's API.

The upload tool limits reference audio to 20 MiB and honors any smaller limit in the policy; file names are randomized and the file form field is sent last. The Bearer key is sent only to Bailian, and OSS uploads use only the signed policy. The signing policy, audio and key are not written to logs; the temporary URL is printed according to the tool's purpose.

Interfaces and boundaries

• com.iafenvoy.system.speech.tts.TtsProvider is a synthesis interface without MC dependencies; it keeps the WAV byte interface and provides an optional streaming PCM callback. Providers are managed by an ordinary factory Map in speech.TtsProviders, with the built-in key aliyun. Extensions call TtsProviders.register(name, factory) during client initialization and do not take part in the MC registry lifecycle; TtsProviders.create(provider) looks up and creates an exclusive instance by exact configuration string and does not add a namespace automatically. The billing table separately prefixes short names with system: before matching; new providers validate protocol parameters themselves. See Registry quick reference for a complete example.
• AliyunTtsProvider implements both Alibaba Cloud HTTP protocols through the shared HttpTransport, and MC uses the SSE path above by default. The retained non-streaming interface has a total response timeout of 45 seconds and an audio download timeout of 25 seconds; JSON is at most 12 MiB and audio at most 8 MiB. No redirects and no automatic retries. Non-streaming downloads accept only Alibaba Cloud OSS audio domains, old HTTP URLs are upgraded to HTTPS, and downloads carry no Bearer token.
• AssistantSpeech.Session is held by the interface and the current notification; globally only the current narration and one pending item are kept, a new narration replaces the old item, and the text queue is managed by AssistantMessages. Closing a session also cancels the in-flight Future, PCM and playback, and a closed session cannot submit again. A single synthesis worker thread and a single work wait slot stay bounded; requests, SSE parsing and WAV decoding for old providers do not occupy the MC thread, and results from an old generation must not start playback again.
• The local usage ledger does not read the file synchronously when narration starts; loading, cost calculation and saving are handled by a separate system-speech-usage serial executor. The usage interface reads only the latest snapshot and does not wait for a background lock or the disk. Network requests, SSE parsing, WAV decoding and the audio cache still run on the synthesis background thread; the main thread manages only sessions and playback state.
• SpeechAudio keeps bounded 16-bit little-endian, mono or stereo PCM; SpeechSound provides vanilla non-blocking reads, SpeechPlayback manages vanilla-first / single fallback, and OpenAlSpeechPlayer refills a circular buffer queue from a cursor. Only a client sound lifecycle mixin is needed; no resource pack, external player or audio library is required.
• AssistantSpeech.status(), SpeechPlayback and the settings page share the SpeechState enum, unifying synthesis, playback and error states; provider error codes are converted at the entry point, and the interface reads the existing bilingual texts through translationKey().
• No new Java dependencies, model files or native runtime libraries. TTS is invoked independently by the client, and usage is written only to the local config/system/speech-usage/<client account UUID>.json; usage, keys, body text and voice IDs are never uploaded to the MC server. /system usage shows the server AI usage and the player's own local TTS / ASR on the same page, usage accumulates across servers and is not cleared by revoke. For the accounting basis and official prices, see Usage and cost.
• Billing rates use the client config/system/system-speech-pricing.json; when missing, defaults/system-speech-pricing.json from inside the JAR is copied and then loaded, keeping any existing configuration. Changes take effect through client F3+T or a restart, and historical accumulated cost is not recalculated; see Pricing configuration for the detailed fields.

Verification scope

The first integration ran Java compilation, testMod source set compilation and JAR packaging, a Python AST syntax check and CLI help, and static checks of the bilingual key sets and template bindings; no billable API was called at that time.

On 2026-10-25, following the user's instruction, one Qwen-Audio-TTS audio clone was actually completed: local ID system-test1, target model qwen-audio-3.1-tts-flash, cloud status OK; query, clone and test speech synthesis all succeeded under the compatibility domain. The reference audio was converted locally to 24 kHz, mono, 16-bit PCM WAV, and the original file was kept. The test output was 6.48 seconds long, and the WAV header was corrected to the actual PCM length. The result is in the ignored voice-output/system-test1/, and no key was written into the result. This acceptance covers online cloning and downloading, and does not mean that audio playback in MC or temporary upload has been accepted.

For the narration queue and the OpenAL fallback, this round only ran Java compilation, testMod source set compilation, JAR packaging and static checks, confirming signatures and the sound lifecycle against the current Minecraft / NeoForge / LWJGL dependency sources. No tests were run, no game was started and no billable API was called again; whether vanilla actually produces sound, the failover and in-game resource cleanup have not been accepted at runtime.

The SSE streaming enhancement passed Java / testMod source set compilation and JAR packaging; chunked playback is implemented according to the official streaming protocol and the source of Channel.attachBufferStream, pumpBuffers and AudioStream.read. No tests were run, no game was started and no billable interface was called; first-packet latency and continuous playback have not been measured, so no measured latency improvement figure is given.

The speech session and notification switching changes passed compileJava testModClasses jar compilation and packaging; interface closing, notification confirmation/timeout/hiding, generation cancellation and delayed callback paths were statically checked. No tests were run, no game was started and no billable interface was called; mid-way cutoff and rapid notification switching have still not been accepted in game.

On 2026-10-26, the full-sentence audio cache and the Qwen-Audio language hint passed compileJava testModClasses jar --console=plain compilation and packaging; static checks covered being off by default, the old-configuration missing-field fallback, the form and voice import keeping the toggle, a hit not reporting cloud usage, complete PCM writes and cancellation protection, and the Chinese and English key sets both contain 377 keys and match. input.language_hints was confirmed against the official HTTP documentation; no game was started, no tests were run and no billable TTS API was called, and the in-game cache hit effect and the actual pronunciation of Dev have not been previewed and accepted.