When transcribing with params.vad = true, whisper already computes the speech
segments and keeps them in the state. Expose them so callers can reuse those
boundaries (for example to align or clip subtitles to speech) instead of running
a second, separate VAD pass.
Times are on the original audio timeline in centiseconds; the count is 0 when VAD
was not used. test-vad-full.cpp checks the segments are ordered and non-empty.
whisper_full_get_token_data().t0/t1 are in the VAD-processed timeline (silences
removed), so they don't line up with the original audio. Add
whisper_full_get_token_t0/t1 that map them back.
A token inside a speech segment is interpolated within it; a token that falls in a
removed inter-segment silence snaps to the nearest boundary, so it never lands in
the middle of a cut-out gap. Without VAD the raw times are returned unchanged.
* whisper : make voice_length() utf-8 aware for CJK
voice_length() weights each token by how long its text takes to say, which drives
how a segment's time is shared between its tokens. It looped over raw bytes, so
every CJK character (3 bytes) was counted ~3x and full-width punctuation never
matched, skewing token timestamps for Chinese/Japanese.
Decode one utf-8 code point at a time and give full-width ,。!? etc. the same
weights as their ASCII counterparts. Pure-ASCII text is unaffected.
* whisper : one statement per line in voice_length()
---------
Co-authored-by: linxiaodong <calm.lin@wukongsch.com>