I have worked on 3D shopping - this seems like a huge step up from the world of photogrammetry to NeRFs to veo. When it comes to shopping - the realism matteres the most. Its not just about 3D model but shadows of light, texture, how it looks in different settings (for e.g. if it's furniture item in a larger scene). That's where the rubber meets the road. Regardless, I'd be curious - how fast can we create 3D models from existing imagery/videos.
I used to work with e-commerce teams, and without access to a proper photo studio, getting good product visuals was always a struggle.If this kind of tech can generate usable 3D views from just a few images, that’s a pretty big breakthrough.
I’m especially curious how it handles reflective or semi-transparent objects—those are usually a pain even in traditional setups.
If it becomes widely available, I could see it being a real boost for smaller sellers trying to compete.
It’s a shame that they’re trying to poorly frame what is a decent paper as a method of e-commerce
1. All the products shown would have CAD models already and have no need for GenAI to make 3D models. A mesher and decimation would give better results anyway, without risk of hallucination.
2. They don’t really address hallucination at all. Why would a reliable retailer trust an imperfect replica? Who is liable for the potential false advertising?
This really feels like marketing tried to co-opt R&D and missed the mark.
A client we did marketing for produces gaming rigs - the metal frames. He's putting it together in the USA, and is super proud of his product (simfab.com) and the way wires come together, the whole modular design. But he struggles with massive Chinese competition ripping his designs off, and also people not understanding which part of the product is his own, and which are the third parties.
So he has the wires in SolidWorks, and everything, but when he needs to showcase the product in 3D he lacks the meshes of all his partner products. And nobody ever's gonna give these out as it is industrial design intellectual property.
It is super apparent that at some point he starts doing lidar point-clouds and having these third-parties scanned (first), and eventually modeled in 3D again by some mesh artist.
Pretty sure his situation is super common. So this GOOG thing is huge for himself and others, who don't even have the lidar option.
I think you might have it backwards. This team seems very focused on fundamental open-ended NeRF/gaussian/lightfield research. It is a natural and good thing that someone at Google found a reasonably relevant way to apply some of the tech the team has developed. Finding an application of the tech within Google helps justify continued investment in fundamental research and provides a source of usage data and feedback to the team. Also, who wouldn't want to see what they made get used by millions of people instead of living only on a conference demo page?
Communication, Trade, Commerce, Navigation, Education/Learning, are timeless needs for 100s of years. We buy and sell products all the time. And if something that improves our day to day shopping experience then we are solving for one of the core needs for us as humans.
Just because something makes shopping easier (and does this, really?) doesn’t mean it’s a smart or meaningful use of brainpower. Not every small convenience is worth solving when bigger problems are out there.
And I still can't search for a pair of jeans of a specific size, brand and style over all online shops ... Sorry, but jeez what are these brightest minds doing?
This seems like a recipe for disaster, since the inferred 3D views aren't necessarily representative of the actual product.
Yeah :c, I guess it seems to work well but this kind of approach makes me sad. It's anti-truth in a way that NeRFs are not
I have worked on 3D shopping - this seems like a huge step up from the world of photogrammetry to NeRFs to veo. When it comes to shopping - the realism matteres the most. Its not just about 3D model but shadows of light, texture, how it looks in different settings (for e.g. if it's furniture item in a larger scene). That's where the rubber meets the road. Regardless, I'd be curious - how fast can we create 3D models from existing imagery/videos.
I used to work with e-commerce teams, and without access to a proper photo studio, getting good product visuals was always a struggle.If this kind of tech can generate usable 3D views from just a few images, that’s a pretty big breakthrough. I’m especially curious how it handles reflective or semi-transparent objects—those are usually a pain even in traditional setups. If it becomes widely available, I could see it being a real boost for smaller sellers trying to compete.
It’s a shame that they’re trying to poorly frame what is a decent paper as a method of e-commerce
1. All the products shown would have CAD models already and have no need for GenAI to make 3D models. A mesher and decimation would give better results anyway, without risk of hallucination.
2. They don’t really address hallucination at all. Why would a reliable retailer trust an imperfect replica? Who is liable for the potential false advertising?
This really feels like marketing tried to co-opt R&D and missed the mark.
That's not precisely true. Let me illustrate:
A client we did marketing for produces gaming rigs - the metal frames. He's putting it together in the USA, and is super proud of his product (simfab.com) and the way wires come together, the whole modular design. But he struggles with massive Chinese competition ripping his designs off, and also people not understanding which part of the product is his own, and which are the third parties.
So he has the wires in SolidWorks, and everything, but when he needs to showcase the product in 3D he lacks the meshes of all his partner products. And nobody ever's gonna give these out as it is industrial design intellectual property.
It is super apparent that at some point he starts doing lidar point-clouds and having these third-parties scanned (first), and eventually modeled in 3D again by some mesh artist.
Pretty sure his situation is super common. So this GOOG thing is huge for himself and others, who don't even have the lidar option.
The best and brightest minds of our times -- working tirelessly to improve your shopping experience
I think you might have it backwards. This team seems very focused on fundamental open-ended NeRF/gaussian/lightfield research. It is a natural and good thing that someone at Google found a reasonably relevant way to apply some of the tech the team has developed. Finding an application of the tech within Google helps justify continued investment in fundamental research and provides a source of usage data and feedback to the team. Also, who wouldn't want to see what they made get used by millions of people instead of living only on a conference demo page?
If you look past the shopping angle- being able to generate 3D objects out of 2D images is a rather huge milestone.
Yes but by necessity it is making up stuff.
It reminds me of: https://www.youtube.com/watch?v=Vxq9yj2pVWk
and in particular the line "enlarge the Z-axis"
Communication, Trade, Commerce, Navigation, Education/Learning, are timeless needs for 100s of years. We buy and sell products all the time. And if something that improves our day to day shopping experience then we are solving for one of the core needs for us as humans.
Just because something makes shopping easier (and does this, really?) doesn’t mean it’s a smart or meaningful use of brainpower. Not every small convenience is worth solving when bigger problems are out there.
Ok, but you got money to pay to solve those bigger problems?
which bigger problems are being solved though?
And I still can't search for a pair of jeans of a specific size, brand and style over all online shops ... Sorry, but jeez what are these brightest minds doing?
You'd think google products would have this feature considering the enormous developer capacity they have. But i think they're getting there.
Cool. Where can I try? I can't find the link to try this new capability anywhere?
Possibly: https://www.google.com/search?ibp=oshop&q=Google+Pixel+9+Pro...
That doesn't have the glass reflectivity I would expect from this latest technique.
If only there was a universal format to describe 3D models that manufacturers world share with a shop. But then there would be no "AI" in the title.
Legitimate question: Is there any standard format that captures not just the geometry but all the detail of reflectance (subsurface scattering, etc.)?
GLTF standard covers a lot of PBR (physically based rendering) properties for materials.
There is also USD but I still fail to recognize the reason of its existance.