Llama.cpp v1.0 on an RTX 3090: up to 55% faster with speculative decoding

 (homebrewserver.club)

2 points | by aidotguru 8 hours ago ago

1 comments