The Htop for LLM Inference

2 comments

• $rbanffy 9 hours ago

  This is super cool. Would be nice if it could be adapted to other architectures besides Nvidia. How is it getting its data?

  BTW, exporting the data to Prometheus would be a killer feature.

• $helasaoudi 11 hours ago

  [flagged]