I made Gutsy to see what quality of inferencing I could get for a Jev style model on laptops and cheap VMs. The result is pretty good for triaging, relevance scoring and yes no type questions. I'd love to hear what latency others are getting. For me it was subsecond inference time for short queries.
I made Gutsy to see what quality of inferencing I could get for a Jev style model on laptops and cheap VMs. The result is pretty good for triaging, relevance scoring and yes no type questions. I'd love to hear what latency others are getting. For me it was subsecond inference time for short queries.