Swedish speech recognition in the browser with Pianissimo
Swedish speech recognition directly in the browser: we ran Klang AI's Pianissimo locally, so the audio never leaves the computer and costs nothing per hour.
By Robin Kling3 min read
Founder of Mpya Digital. Leads Mpya Digital AI.
On 23 October, Klang AI hosted a hackathon. The task was to build something with their Swedish speech recognition model Pianissimo in one hour, and my colleague Anir Darrazi and I each built a prototype. They show that Swedish voice AI is now ready to build real services on. The day also showed that the vendor's own hackathon is one of the fastest ways to learn a new model.
AI interview training in Swedish
I built an AI recruiter that interviews you in Swedish, one question at a time, and asks follow-up questions when an answer is vague. Pianissimo transcribes the answers in real time. Afterwards you get feedback on every answer: what to improve, an example of a stronger answer, how fast you spoke and which filler words you used. Our consultants can use it to prepare for client interviews, and job seekers can practise without having to book anyone.
We got far because we did not start from scratch. Both prototypes build on the real-time transcription prototype I made when Pianissimo was released in September.
Speech recognition directly in the browser
Anir moved the entire speech recognition to the browser. Pianissimo now runs locally on the user's computer, and Anir's measurements on an ordinary Mac M4 show it runs about six times faster than real time. The text updates about once a second while you speak, compared with every two to three seconds with the API we used before.
What local speech recognition means for you
The audio stays with the user. Meetings, customer calls and interview answers contain sensitive information. When the model runs in the browser, the audio never leaves the computer.
Cost does not grow with usage. The user's computer does the work, so every extra hour of audio costs you nothing, and you need no servers sized for the busiest hour.
The text comes faster. The audio does not go to a server and back, so the text appears while you are still speaking.
The device sets the experience. The model is a 660 MB download the first time and needs around 2.2 GB of memory, which rules out most older phones with less than 8 GB of memory. A product should therefore check the device and fall back to a cloud service when it is not enough.
This only works because Pianissimo has open model weights and is small enough to run on an ordinary computer.
Go to the AI vendor's hackathon
I think more tech leaders should send their developers to this kind of hackathon. You learn a model faster in a few hours with the people who built it than in weeks on your own, and you get to test an idea without first starting a project.
Tell me about your experience
Have you built something with Swedish voice AI, or are you wondering where it could make a difference for you? Get in touch, I would like to hear more.
Frequently asked questions
- Can Pianissimo run in the browser?
- Yes. At Klang AI's hackathon, Anir Darrazi ran Klang AI's Swedish speech recognition model Pianissimo locally in the browser. On an ordinary Mac M4 it ran about six times faster than real time, and the text updated about once a second.
- How much memory does Pianissimo need in the browser?
- The model is a 660 MB download the first time and needs around 2.2 GB of memory, which rules out most older phones with less than 8 GB of memory. A product should check the device and fall back to a cloud service when it is not enough.
- What was built at Klang AI's hackathon?
- An AI interview trainer that interviews you in Swedish and gives feedback on every answer, and a version of the speech recognition where Klang AI's Pianissimo model runs directly in the browser.
- Why run speech recognition in the browser instead of in the cloud?
- The audio never leaves the user's computer, the text arrives faster because the audio does not have to go to a server, and there is no cost per hour of audio because the user's computer does the work.