I wanted a completely offline, real-time AI translator for my Mac. So, I built one using WebRTC VAD, whisper.cpp (for audio), and llama.cpp (1.8B model for translation). It translates system audio or mic input and saves timestamped logs.
The good news: When running from the terminal, it actually works. The latency is mostly sentence-by-sentence.
The bad news (why I need help):
To be completely honest, the environment setup was a nightmare. I hit over 14 compilation errors just trying to install whisper-cpp-python with Metal support. I eventually used an AI agent to help me brute-force the CMake flags, and it worked... but I still don't fully understand how it was fixed.
There are also some weird bugs I couldn't iron out:
• There seems to be a slight memory leak over time.
• Audio/text sync is a bit off right at startup.
• While the latency is usually good, sometimes it buffers and spits out several translated sentences all at once.
I also used Claude to package the whole thing into a macOS .dmg. It runs, but the packaged app feels noticeably less smooth/responsive than running the raw python script in the terminal. I have no idea why the packaged version is lagging.
The code is open-sourced on my GitHub. If there are any C++ or Python optimization wizards here, I would love some pointers on how to fix my plumbing!
I wanted a completely offline, real-time AI translator for my Mac. So, I built one using WebRTC VAD, whisper.cpp (for audio), and llama.cpp (1.8B model for translation). It translates system audio or mic input and saves timestamped logs. The good news: When running from the terminal, it actually works. The latency is mostly sentence-by-sentence. The bad news (why I need help): To be completely honest, the environment setup was a nightmare. I hit over 14 compilation errors just trying to install whisper-cpp-python with Metal support. I eventually used an AI agent to help me brute-force the CMake flags, and it worked... but I still don't fully understand how it was fixed. There are also some weird bugs I couldn't iron out: • There seems to be a slight memory leak over time. • Audio/text sync is a bit off right at startup. • While the latency is usually good, sometimes it buffers and spits out several translated sentences all at once. I also used Claude to package the whole thing into a macOS .dmg. It runs, but the packaged app feels noticeably less smooth/responsive than running the raw python script in the terminal. I have no idea why the packaged version is lagging. The code is open-sourced on my GitHub. If there are any C++ or Python optimization wizards here, I would love some pointers on how to fix my plumbing!