Describing a 3d scene with text is one way using ML to update an NPC, but I feel like the real killer app would be a multi-modal agent. Deepmind recently showed some impressive progress in this sense: https://deepmind.google/discover/blog/sima-generalist-ai-age...
Thanks for sharing this link!
This would definitely be a silver-bullet, but I'm not aware of such agent having the ability to take and spit out text input!
Another example of this idea would be Midjourney's CEO vocal plans to create models able to natively take in 3d input - but no demo yet :)
Thanks for the comment!
For now, the README.md and the tests files [1] should give you a first idea on how to use our library.
But we're also working with the https://cu.bzh platform to create demos using their engines - those should give you inspiration as well!
Describing a 3d scene with text is one way using ML to update an NPC, but I feel like the real killer app would be a multi-modal agent. Deepmind recently showed some impressive progress in this sense: https://deepmind.google/discover/blog/sima-generalist-ai-age...
Thanks for sharing this link! This would definitely be a silver-bullet, but I'm not aware of such agent having the ability to take and spit out text input! Another example of this idea would be Midjourney's CEO vocal plans to create models able to natively take in 3d input - but no demo yet :)
That's a very exiciting project, do you plan to write more tutorial and demos?
Thanks for the comment! For now, the README.md and the tests files [1] should give you a first idea on how to use our library. But we're also working with the https://cu.bzh platform to create demos using their engines - those should give you inspiration as well!
[1] https://github.com/GigaxGames/gigax/blob/main/tests/test_ste...