I had a client doing this in 2016. It was ancillary to their main business which was matching movers with homeowners. They eventually pivoted into an insurtech offering using computer vision models to do what you’re suggesting, collecting the data at the point of moving by using the movers to do the “heavy lifting” of scanning the items. They ended up being insurance intermediaries before folding so I’m not sure how valuable the idea is. Maybe things are different now.
We (ML team) were brought in to talk to some folks at our sister company about this possibility. Our conclusion was basically that expectations were way too high - as mentioned in your quote they wanted a specific brand, model, and price for every item (from a grainy photo of a charred object, out of a scraped database of millions of items, with a correct hit in the top three or four results). Frankly I don't know how/if even the expert human appraisers manage it.
If you could achieve those results, it would be tremendously valuable. But I think you'd need a research team and ~~five~~ many years.
Oh I've been thinking about this a lot, from a different perspective.
Now that we can get highly detailed Gaussian Splats to represent spaces in 3D, there has been great work to do segmentation of these datasets. Theres a lot of momentum behind both of these ideas.
The technology is very nearly there, such that you could scan your home from your phone, and get a detailed segmented map of everything you own.
I believe I've also seen someone take a video and input it into Gemini and ask for a list of all the products. Some combination of these ideas really.
Sounds not realistic. Imagine you have this list, how do you know it's complete? What if it's not complete and you think it's complete and you only find out something was missing when it was too late?
edit: You might as well work on software that keeps a list of everything you ever owned so you can just whip it out once your house burns down.
I had a client doing this in 2016. It was ancillary to their main business which was matching movers with homeowners. They eventually pivoted into an insurtech offering using computer vision models to do what you’re suggesting, collecting the data at the point of moving by using the movers to do the “heavy lifting” of scanning the items. They ended up being insurance intermediaries before folding so I’m not sure how valuable the idea is. Maybe things are different now.
We (ML team) were brought in to talk to some folks at our sister company about this possibility. Our conclusion was basically that expectations were way too high - as mentioned in your quote they wanted a specific brand, model, and price for every item (from a grainy photo of a charred object, out of a scraped database of millions of items, with a correct hit in the top three or four results). Frankly I don't know how/if even the expert human appraisers manage it.
If you could achieve those results, it would be tremendously valuable. But I think you'd need a research team and ~~five~~ many years.
Oh I've been thinking about this a lot, from a different perspective.
Now that we can get highly detailed Gaussian Splats to represent spaces in 3D, there has been great work to do segmentation of these datasets. Theres a lot of momentum behind both of these ideas.
The technology is very nearly there, such that you could scan your home from your phone, and get a detailed segmented map of everything you own.
I believe I've also seen someone take a video and input it into Gemini and ask for a list of all the products. Some combination of these ideas really.
This is encouraging!
Sounds not realistic. Imagine you have this list, how do you know it's complete? What if it's not complete and you think it's complete and you only find out something was missing when it was too late?
edit: You might as well work on software that keeps a list of everything you ever owned so you can just whip it out once your house burns down.
Hmm. Perfection is not possible on Day 1, but the status quo isn't perfect either.
I expected handwaving instead of answers to my questions and that's what I got.
You can just write a gpt wrapper that is really bad for the job and get some seed money I guess.
completing the absurdity of this:
an AI assistant that tells you what is in the picture as voice, so you can rest your eyes....
...hold on, that may actually be pretty useful for blind people
Lmaooooooo(on)
It's certainly a brilliant app if it existed.
I'd suggest the decluttering movement, inventory, allergens, tracking lost stuff, would also find it a useful tool.
Decluttering passes Larry Page's "toothbrush test" FWIW
Sorry, can't answer your question.
AI should not be used for anything that requires accuracy.
everything requires accuracy. the question is "how much accuracy?"
Respectfully, that MAY be the case today, but not necessarily tomorrow (where the puck is going)
It will become the case when we will have confidence that it will provide expected output and be able to fix unexpected output if/when it comes up.