I've been thinking of and briefly working on a similar project.
One idea is to analyze timing as well, and "trigger" things after certain sequences (so play 1-3-5 as say eighth notes and then get an in-rhythm arpeggio one octave higher) or detect the beat and play on the upbeat.
I haven't done any Rust, but this might give me a good reason to give a try.
Have you considered making it a plugin? (makes replay easier in my opinion, but brings other pain like relaunching the DAW between builds...)
I had added a metronome and a note generator to this earlier which aren't working well exactly right now. But this would definitely increase playability. I agree that there should be a VST plugin version of this as well. It can also live as a standalone app and a plugin. Could you elaborate on the analyze timing? If you had to play using this, what would you like to hear ?
I've got a few thoughts for features, if you're open to them:
1. Ability to specify where your "played" voice resides in the voicing: As the bass note, as an inner voice, or as the top line.
2. Options for first species, second species, third, florid, etc counterpoint for each of the generated voices. Ex: You play a single note and the upper voice plays two notes for every one of yours, etc, etc.
3. If you want to get real fancy, make the generated voices perform a canon of your played notes.
Have you been able to try it as well would love to hear what you think! Coming back to the features, regarding 1. you can already choose between soprano, alto, tenor or bass. I have still filed an issue for this, will help me remember to take vet this feature. Sometimes it's not as strict as it should be but that's also something I need to work on. Regarding 2. it's a good idea, helps you be in control of the kind of counterpoint you are doing, filed an issue for the same. Please feel free to comment on the issue. 3. is just feels is a little goofy as well I love it. I haver filed an issue for this as well check https://github.com/contrapunk-audio/contrapunk/issues/
"Realtime" as in "while playing guitar" has some pretty challenging latency requirements. Even if your solution is optimal, hardware specs will play a meaningful role. I'd be really interested if you've solved for this e2e.
Yes, latency was the main problem to solve here. Because of which I opted for Rust. The pipeline is:
- 128-sample cpal audio buffers (~2.7ms at 48kHz)
- Single-cycle pitch detection
- 2-frame McLeod pitch voting for confirmation
- Entire DSP pipeline is Rust, pre-allocated ring buffers with minimal heap pressure
The e2e from pluck to MIDI "note-on" signal, is under 10ms on an M-series Mac. Hardware matters for sure so an audio interface with low-latency drivers (I use an Audient iD14) helped a lot. The web version (app.contrapunk.com) adds AudioWorklet latency on top, so the native Mac app is noticeably tighter. I am still working on figuring out how to have lesser noise and pitch jitter in the final output. Also this works really well for higher notes, bass not so much right now. Still need to figure out how to handle harmonics better. I have created this issue for you for now, let me know if you would like to add anything else to this as well. https://github.com/contrapunk-audio/contrapunk/issues/6.
That's awesome thank you for sharing, really appreacite it ! Hope you have a good time with Contrapunk. Just let me know if you are facing any issues as well.
Sorry about that, you will have to download the app again there was an issue with the midi out for guitar atleast. Just to be sure, have you setup the IAC buses. I usually like to have 4 and then configure them as output devices.
There was a 1970's Indonesian progressive rock band called Contrapunk that released an album called "Putri Mohon Diri" [1]
You can find the recording on YT [2]
They were really unique - blending traditional Indonesian instruments, intense guitar work and classical influences.
Ah banger from the get go. Guess this is what I will be listening to through rest of the day. And I will add this as a fun fact to the website as well. Thank you for sharing :)
Thank you! The idea is not completely mine, I have to give thanks to Abhinav Arora who had this idea initially during the ADCx music hackathon. Kudos to him! Also love the phrase Gradus ad Parnassum! Maybe this should be the motto of contrapunk :)
How are you finding rust for audio development? I have a background in pro audio, and both for the audio and GPU render threads, I used a lot of arena allocators and ring buffers. Do you find yourself fighting with rust's strict semantics?
This is callback heavy audio code so this was the bigger problem for me mainly and learning about lifetimes was a pain initially. cpal's stream callback wants 'static which means you can't just pass references around. You end up using channels (crossbeam / std::sync::mpsc) between the audio thread and everything else. Once I structured around that it got smoother. I also got a lot of help from AI to understand and reimplement a lot of the parts for this as you can tell from the commit messages.
Nicely done! I saw you're building a cloud version so wanted to refer the excellent https://github.com/sonosaurus/sonobus which you might be able to integrate. All the best!
Ah thanks for referring sonobus, hadn't seen it before. Would defintiely be good to get some inspiration from there. Do you use sonobus by any chance ? If so, what has your experience been with it ?
Not gp.
I use it as I'm writing music, so all my ideas, repeats, mistakes, rewinds, instrument changes, etc etc are broadcast. I'll mute and un'utenza other musicians as i feel.
Really appreciate you sharing your workflow honestly.
I have a few follow up questions. When you're using SonoBus, are your musicians running local synths/instruments, or are you working with audio streams? And are you on macOS?
Asking because you can actually use Contrapunk with SonoBus today then route guitar through Contrapunk, send the harmony MIDI to a synth via IAC buses, and pipe the audio into SonoBus through BlackHole. But there's clearly an extra step you probably shouldn't have to do. Your session hears the harmonized output and you can mute/unmute the Contrapunk voices like any other musician. Longer term we can look at native SonoBus integration via the AOO protocol so Contrapunk would show up as a peer in your session directly. What do you think about it ? Would love your input on what that should look liked.
What a cool idea. I don't have a music setup capable of running this right now - perhaps in a couple of months - but if you were to post some sample recordings, I'd gladly listen to them.
How do you generate velocity values for the accompaniment notes?
Given that you already have a pitch tracker, it could be interesting to add key detection; just start playing, instead of telling the machine what key you're in, and it starts following along as soon as it catches on.
Thank you! The best part about this project is that you don't actually need a elaborate setup :) All you need is a DAW like garage band or logic and a few IAC buses configured on your mac. I really like the idea of key detection, it is something I have already thought about as well: based on the song it should pick up the key which you can then jam to with contrapunk and I think it can act as an educational moment for the player as well at the time. https://github.com/contrapunk-audio/contrapunk/issues/4 I have created a github issue over here as well for this. For the velocity currently I am inheriting the onset strength of the input signal for the accompanied notes. The guitar input measures RMS energy in the first ~5ms of each pluck (the attack transient) and maps that to MIDI velocity. I will post more sample recordings on the website soon! Were you able to check the one which was already posted?
Unfortunately not right now, it's in the works. Polyphonic guitar to midi is a problem I am yet to understand and try solving in this one. Jam Origin's Midi Guitar is good like that, I still need to get there.
The quickest workaround for this would be running ` xattr -cr /Applications/Contrapunk.app`. Are you able to use the web version though at app.contrapunk.com
Ah perfect ! Let me know what you think ! Also you can ping me on twitter if required https://x.com/BobadeVibhav/ Not really a twitter user, never been, but would be a easier way to connect obviously.
Thanks for taking a look! Let me know if you have any issues with it, it's still in a nascent stage and has a lot of room to grow especially in the Guitar to MIDI detection. If you don't have a midi controller you can use your keyboard as well !
Would be nice to have it be realtime no, is this something you are looking to have? Do sign up on the cloud waitlist so I can get in touch wrt progression on the chord detection work.
Harmonies: I notice that the Bach mode in C major, given the notes C-D-E, produces Dm7 Em7 Fmaj7. Something like C G7 Am would be more Bach.
If you're interested in adding more scales, I have a wider range in my scale builder: https://xn--langy-yua.net/skala
Very cool!
FYI in this phrase: "AI is not going to kill music till people keep playing music together."
The "till" (until) kind of inverts what I think is the intended meaning.
A better replacement would be "as long as".
Ah, thanks for catching that really appreciate it ! Updated the website now.
I've been thinking of and briefly working on a similar project.
One idea is to analyze timing as well, and "trigger" things after certain sequences (so play 1-3-5 as say eighth notes and then get an in-rhythm arpeggio one octave higher) or detect the beat and play on the upbeat.
I haven't done any Rust, but this might give me a good reason to give a try.
Have you considered making it a plugin? (makes replay easier in my opinion, but brings other pain like relaunching the DAW between builds...)
created https://github.com/contrapunk-audio/contrapunk/issues/8 https://github.com/contrapunk-audio/contrapunk/issues/9, let me know what you think
I had added a metronome and a note generator to this earlier which aren't working well exactly right now. But this would definitely increase playability. I agree that there should be a VST plugin version of this as well. It can also live as a standalone app and a plugin. Could you elaborate on the analyze timing? If you had to play using this, what would you like to hear ?
Cool idea!
I've got a few thoughts for features, if you're open to them:
1. Ability to specify where your "played" voice resides in the voicing: As the bass note, as an inner voice, or as the top line.
2. Options for first species, second species, third, florid, etc counterpoint for each of the generated voices. Ex: You play a single note and the upper voice plays two notes for every one of yours, etc, etc.
3. If you want to get real fancy, make the generated voices perform a canon of your played notes.
Have you been able to try it as well would love to hear what you think! Coming back to the features, regarding 1. you can already choose between soprano, alto, tenor or bass. I have still filed an issue for this, will help me remember to take vet this feature. Sometimes it's not as strict as it should be but that's also something I need to work on. Regarding 2. it's a good idea, helps you be in control of the kind of counterpoint you are doing, filed an issue for the same. Please feel free to comment on the issue. 3. is just feels is a little goofy as well I love it. I haver filed an issue for this as well check https://github.com/contrapunk-audio/contrapunk/issues/
"Realtime" as in "while playing guitar" has some pretty challenging latency requirements. Even if your solution is optimal, hardware specs will play a meaningful role. I'd be really interested if you've solved for this e2e.
Yes, latency was the main problem to solve here. Because of which I opted for Rust. The pipeline is:
- 128-sample cpal audio buffers (~2.7ms at 48kHz) - Single-cycle pitch detection - 2-frame McLeod pitch voting for confirmation - Entire DSP pipeline is Rust, pre-allocated ring buffers with minimal heap pressure
The e2e from pluck to MIDI "note-on" signal, is under 10ms on an M-series Mac. Hardware matters for sure so an audio interface with low-latency drivers (I use an Audient iD14) helped a lot. The web version (app.contrapunk.com) adds AudioWorklet latency on top, so the native Mac app is noticeably tighter. I am still working on figuring out how to have lesser noise and pitch jitter in the final output. Also this works really well for higher notes, bass not so much right now. Still need to figure out how to handle harmonics better. I have created this issue for you for now, let me know if you would like to add anything else to this as well. https://github.com/contrapunk-audio/contrapunk/issues/6.
I’m very excited to try this. I haven’t looked deep into it but Steinberg makes an sdk for vst/au development. It has rust bindings!
https://steinbergmedia.github.io/vst3_doc/vstsdk/index.html
That's awesome thank you for sharing, really appreacite it ! Hope you have a good time with Contrapunk. Just let me know if you are facing any issues as well.
I’ve got a midi watcher. I can get midi in to the app and it’s recognized, can’t see any midi messages out
Sorry about that, you will have to download the app again there was an issue with the midi out for guitar atleast. Just to be sure, have you setup the IAC buses. I usually like to have 4 and then configure them as output devices.
There was a 1970's Indonesian progressive rock band called Contrapunk that released an album called "Putri Mohon Diri" [1]
You can find the recording on YT [2] They were really unique - blending traditional Indonesian instruments, intense guitar work and classical influences.
[1] https://www.discogs.com/release/17424685-Contrapunk-Putri-Mo...
[2] https://www.youtube.com/watch?v=jb1792ZuXcY
Ah banger from the get go. Guess this is what I will be listening to through rest of the day. And I will add this as a fun fact to the website as well. Thank you for sharing :)
> Guess this is what I will be listening to through rest of the day
You won't be alone :)
Such a cool project and the name and thanks for making it open source!
IKR ! and thank you, let me know if you were able to download and play around with it !
Gradus ad Parnassum! What a cool idea, and the fact it's counterpoint gives you a nice little time buffer for any DSP. Super cool
Thank you! The idea is not completely mine, I have to give thanks to Abhinav Arora who had this idea initially during the ADCx music hackathon. Kudos to him! Also love the phrase Gradus ad Parnassum! Maybe this should be the motto of contrapunk :)
Contrapunk is a cool name though.
How are you finding rust for audio development? I have a background in pro audio, and both for the audio and GPU render threads, I used a lot of arena allocators and ring buffers. Do you find yourself fighting with rust's strict semantics?
This is callback heavy audio code so this was the bigger problem for me mainly and learning about lifetimes was a pain initially. cpal's stream callback wants 'static which means you can't just pass references around. You end up using channels (crossbeam / std::sync::mpsc) between the audio thread and everything else. Once I structured around that it got smoother. I also got a lot of help from AI to understand and reimplement a lot of the parts for this as you can tell from the commit messages.
Nicely done! I saw you're building a cloud version so wanted to refer the excellent https://github.com/sonosaurus/sonobus which you might be able to integrate. All the best!
Ah thanks for referring sonobus, hadn't seen it before. Would defintiely be good to get some inspiration from there. Do you use sonobus by any chance ? If so, what has your experience been with it ?
Not gp. I use it as I'm writing music, so all my ideas, repeats, mistakes, rewinds, instrument changes, etc etc are broadcast. I'll mute and un'utenza other musicians as i feel.
Really appreciate you sharing your workflow honestly.
I have a few follow up questions. When you're using SonoBus, are your musicians running local synths/instruments, or are you working with audio streams? And are you on macOS? Asking because you can actually use Contrapunk with SonoBus today then route guitar through Contrapunk, send the harmony MIDI to a synth via IAC buses, and pipe the audio into SonoBus through BlackHole. But there's clearly an extra step you probably shouldn't have to do. Your session hears the harmonized output and you can mute/unmute the Contrapunk voices like any other musician. Longer term we can look at native SonoBus integration via the AOO protocol so Contrapunk would show up as a peer in your session directly. What do you think about it ? Would love your input on what that should look liked.
What a cool idea. I don't have a music setup capable of running this right now - perhaps in a couple of months - but if you were to post some sample recordings, I'd gladly listen to them.
How do you generate velocity values for the accompaniment notes?
Given that you already have a pitch tracker, it could be interesting to add key detection; just start playing, instead of telling the machine what key you're in, and it starts following along as soon as it catches on.
Thank you! The best part about this project is that you don't actually need a elaborate setup :) All you need is a DAW like garage band or logic and a few IAC buses configured on your mac. I really like the idea of key detection, it is something I have already thought about as well: based on the song it should pick up the key which you can then jam to with contrapunk and I think it can act as an educational moment for the player as well at the time. https://github.com/contrapunk-audio/contrapunk/issues/4 I have created a github issue over here as well for this. For the velocity currently I am inheriting the onset strength of the input signal for the accompanied notes. The guitar input measures RMS energy in the first ~5ms of each pluck (the attack transient) and maps that to MIDI velocity. I will post more sample recordings on the website soon! Were you able to check the one which was already posted?
I had not noticed that the screenshot was actually a video, so I hadn't listened - thanks for the hint.
No worries!
Would be useful to have a video demoing it if people don't have a guitar or MIDI instrument handy.
You can use your computer keyboard as well. Setup the IAC buses and use the keyboard and you should be sorted.
How good is the guitar to MIDI detection? Can it recognize chords?
Unfortunately not right now, it's in the works. Polyphonic guitar to midi is a problem I am yet to understand and try solving in this one. Jam Origin's Midi Guitar is good like that, I still need to get there.
totally agree. tabs by ear is still the best way to internalize a song. chord sites are fine as a starting point but you miss so much of the nuance.
Yes sir, that is the best way. Learning music by ear is the best way to get through yes. What does your practice session usually look like ?
Seems like the Mac app has a packaging problem. Tahoe on an M2 Max views it as corrupted and refuses to launch it
The quickest workaround for this would be running ` xattr -cr /Applications/Contrapunk.app`. Are you able to use the web version though at app.contrapunk.com
I was bitten by the same issue. Maybe add a line to the docs.
Added to the README, apologies for that.
Oh, right, forgot Tahoe made the error message for this so generic. Yeah, that did the trick
Ah perfect ! Let me know what you think ! Also you can ping me on twitter if required https://x.com/BobadeVibhav/ Not really a twitter user, never been, but would be a easier way to connect obviously.
Ah sorry about that. Let me see what's up. I am able to run it locally on my M2 Pro. Not running Tahoe. Filed an issue https://github.com/contrapunk-audio/contrapunk/issues/7
The quickest way to fix this is clone the repo and run `cargo tauri dev` I believe.
Wow that's is really cool! Thanks for sharing! I will definitely take a look
Thanks for taking a look! Let me know if you have any issues with it, it's still in a nascent stage and has a lot of room to grow especially in the Guitar to MIDI detection. If you don't have a midi controller you can use your keyboard as well !
Have you seen this? https://github.com/ptnghia-j/ChordMini
I haven't, this looks neat. Have you tried it ?
yeah but it's not real time.
Would be nice to have it be realtime no, is this something you are looking to have? Do sign up on the cloud waitlist so I can get in touch wrt progression on the chord detection work.
This is actually super cool!
Thank you!
finally some good stuff on here
thanks ! go ahead and download the DMG and try it, let me know what you think ! :D