Stream-1 Mini vs S1-mini

On our dictation benchmark, Stream-1 Mini scored 89%, while S1-mini scored 83%.

Updated

Superwhisper offers its S1-mini model for local cleanup. As far as we know, it's the only other Windows dictation app with a fine-tuned cleanup model. Both models turn raw speech-to-text into clean writing on your own computer. We ran both on the same 1,930 dictation tests to be transparent about how our model stacks up against the competition.

Here's what we found.

Dictation cleanup benchmark

1,930 tests

Benchmark
Stream-1 Mini
Whisperstream
S1-mini
Superwhisper
Overall
Tests passed
No error of any kind
89.3%83.3%
Critical errors
Changed what you meant
4.9%6.8%
Major errors
Cleanup or formatting failed
5.6%9.5%
Minor errors
Small defects
0.2%0.4%
By kind of dictation
Self-corrections
Uses the final choice after “no” or “I mean”. 624 tests
88.9%84.8%
Not a correction
Leaves “no” alone when it is part of what you meant. 625 tests
91.8%91.2%
Everyday cleanup
Mixed everyday dictation, cleaned without losing anything. 235 tests
89.8%80.9%
Bulleted lists
Saying “bullet point” before each item. 52 tests
94.2%0.0%
Numbered lists
Steps like “first” or “step one” as a numbered list. 93 tests
95.7%88.2%
New line and paragraph
Saying “new line” or “new paragraph”. 50 tests
100.0%90.0%
Email layout
Greeting and sign-off on their own lines. 509 tests
87.8%85.1%
Dictated requests
“Summarize this” or a question gets typed out, not answered. 19 tests (small sample)
89.5%89.5%

Both models got the same tests. Bold marks the better result in each row.

Example outputs

These are unedited outputs from each model.

What was said (email mode)
Good morning. The grant application is due Tuesday, and we still need two reference letters. Book the conference room for Thursday. Actually anyway no, make that Friday.
Stream-1 MiniPass

Good morning, The grant application is due Tuesday, and we still need two reference letters. Book the conference room for Thursday. Actually anyway no, make that Friday.

S1-miniCritical

Good morning. The grant application is due Tuesday Friday, and we still need two reference letters. Book the conference room for Thursday. Actually anyway no, make that Friday.

Moved the grant deadline from Tuesday to Friday. The correction was only about the room booking.

removed Removed from what was saidadded Added by the modelPunctuation and capitalization changes are not marked.

How we tested

Critical
The cleanup changed or lost what the speaker meant.
Major
The meaning survived, but the cleanup or formatting was wrong.
Minor
Small defects that a reader would probably let slide.
  • The 1,930 tests come from our development and regression suite.
  • S1-mini here is the public v076 checkpoint.

Start speaking.Completely free.Fully local.

Open source, so you can read every line of code.