This is really clever from Apple. The journalist use case is just the PR story. This will be really useful for identity verification and insurance apps, and has the potential to shift from "you need a smartphone to be able to live normally" to "you need an iPhone to be able to live normally".
There are already plenty of insurances that require you to submit claims through a smartphone app that tries to essentially do this by capturing sensor metadata etc. - those don't need to be nation-state resilient, just Joe the Crackhead Insurance Scammer resilient, so this works. Likewise, more and more things online require identity verification (either officially or disguised as age verification).
Edit: And while "a nation state actor can spoof this" is a problem for the journalism use case, the insurance/ID verification use cases are perfectly fine with anything that raises the bar but could be bypassed with enough effort. Also, the journalism use case suffers from the same fundamental issue all of these use cases suffer from: People will "verify" the picture by looking at the repost of a screenshot of the verification UI, not by verifying the original themselves.
An insurance would either send #1 an insurance agent or mechanic to initially assess the damage (trusted) or #2 ask the customer to send pictures (untrusted).
Tendency is #2 for cost-saving of the insurance, and 3rd party apps are used to execute this.
Now the idea is that the insurance company discontinues the App and the (untrusted) customer must have an iPhone 18 Pro to make an insurance claim?
Or is the insurance agent / mechanic an untrusted entity who will now be required to have an iPhone 18 Pro?
What is the fraud vector here, and how can the insurance service provider continue cost-saving on damage-assessment by offloading to the customer, if the customer is required to own a specific device?
Some banks needs photos of machine readable IDs to verify user details to fight fraud. These IDs can be passports or NFC enabled EU (and compatible) ID cards.
This will allow banks to trust these cameras more on the long run, allowing higher security ID checks.
Okay, this doesn't answer the question on the vector but is another interesting example. Let's expand on that one then:
Banks are offloading the trusted process of ID verification to an untrusted entity (end-user, merchant,...) and compensate for the loss of security by using a trusted service-provider (now Apple AND an iPhone 18 Pro).
This is already happening today in two scenarios:
1. lower-risk scenarios (remotely) with trusted 3rd party service-providers and very low Hardware-requirements ("use this app on your phone to take a picture/video") and
2. higher-risk scenarios (on-site) with trusted 3rd party service-providers ("use THIS expensive device to take a picture/video of the customer/citizen")
Apple now potentially disrupts the service-provider industry of #2 (higher-risk scenarios) by
#a grabbing a part of this hardware/service market that MAY allow the end-user to be in control of the device and
#b replacing the on-site hardware/service with an iPhone "in a box".
They can't disrupt #1 because their cost-saving can't mandate the end-user to buy a 1000+ USD device just for THEM to provide the contracted service. (They can add convenience if you have it, but they can't reject their service if you don't)
Which means they disrupt mainly #2: The industry providing trusted imaging solutions for higher-risk scenarios.
--> So it's the Watch Ultra game all over again.
On Watch Ultra they disrupted the diving-watch market by the sheer scale of selling their development to everyone buying a Watch Ultra, driving down the cost so much that they can undercut every diving-watch company on the market.
Now they use the sheer scale of iPhone 18 Pro sales to enter the trusted-imaging market-segment, undercutting every player there and take that market.
Not the OP, but yes, other vendors will be able to support that as well. But a camera sensor that has
1. a public/private key exchanged during device-production (production-cost),
2. the capability to reboot in a cryptographic mode (R&D / component cost) and
3. a cloud-service which then processes the raw data to create a JPG (operational cost)
comes at a premium. Why should this premium be applied on a 99 USD Smartphone?
Which is my whole puzzle on this vector: If the big benefit is for insurance/ID-verification, which apply cost-saving by offloading their process to the untrusted customer, how much they can offload this by requiring their customer to own a 1000+ USD smartphone to provide THEIR service...?
The most I can imagine is insurances offloading their work to OTHER companies, NOT trusting them and therefore requiring them to own a 1000+ USD Smartphone. But even then, why not use a third party app that also runs on a 3y old iPhone and a 99 USD Android device...?
We have 99USD smartphones with 1080p+ AMOLEDs, massive 5k amp batteries and very performant SOCs (e.g. Galaxy A16) among other costly, but not vital niceties. I struggle to see how cost could be a factor here.
I think you’re on the ball, but also all KYC flows, photo proof for shipping returns (Chinese platforms were getting destroyed on this) etc. etc. It’s a very very clever solution and a very opportune time.
Like a lot of things in the UK, the problem isn't the harshness of the law, it's the lack of prosecution. It doesn't matter what the punishment is when criminals know that the crime is extremely unlikely to be investigated or make it to trial.
I don’t understand what this brings to the table beyond what we’re currently doing.
Insurance companies can have a native app and require the device’s camera. Companies already have tools to combat a liveliness check. Even if you’re using a modified app that pulls from the photo album instead of the camera? A video recording with the appropriate liveness verification easily avoids that mess.
As per opening paragraph of link, AI fakes are a thing.
It's been possible to do a live video deepfake for a long time now, but as with all new tech, law and society are taking their sweet time to understand the risks; IMO this is the other side of the same coin as some infamous tech comments on consumer products: https://news.ycombinator.com/item?id=9224 and https://en.wikiquote.org/wiki/Rob_Malda
NVIDIA suggested AI fakes controlled with face tracking input as a compression technique just for reducing video call bandwidth requirements (to ~117 bytes per frame). They did that six years ago: https://www.dpreview.com/news/5756257699/nvidia-research-dev...
As we're now in an AI race, even NVIDIA's specific technique has flaws which all the current tools can detect, there's never any guarantee of this continuing to be the case.
That said, in the case of Apple, they're historically followers not leaders despite the public image they like to present about innovation, and I'd expect this method to be flawed from day one even if we weren't reading a corporate blog post written in a self-congratulatory tone I find almost as off-putting as when AI write.
> That said, in the case of Apple, they're historically followers not leaders despite the public image they like to present about innovation
While its true Apple usually isn't the first in a product category--not the first mp3 player, not the first smartphone, not the first tablet) but once they get there, they're quite innovative.
When the iPhone 5s was released in 2013, it was the first smartphone with a 64-bit processor, which caught Qualcomm off guard. Even when Qualcomm released a 64-bit processor the following year, it kinda didn’t matter because Android was still 32-bit.
You don’t even need an AI deepfake to edit a video.
AI deepfake or edit video doesn’t pass liveliness checks without all the c2pa or reference image song and pony show.
Insurance companies can monitor the light reflections from the flash that they control or monitor the accelerometer and compare the accelerometer values with the video that they receive.
They could also just update their app to stop accepting photos from the album.
Insurance agencies worried about fraud could just do what they did prior to smartphones: have you bring the car to a claims adjuster at your local office. I’ve brought cars to be inspected, because I got T-boned at an intersection by a careless driver before smartphones existed; it was reasonably quick and hassle-free.
> People will "verify" the picture by looking at the repost of a screenshot of the verification UI, not by verifying the original themselves.
True.
> raises the bar but could be bypassed with enough effort.
Anyone can spoof this.
Apple cannot stop spam iMessages. They can't stop someone from rendering their privileged UI inside a browser viewport. People copy and paste remote script executions from convincing captchas.
This whole provenance thing is a red herring. You agree with me, but there's truly not a single application for this that won't be exploited.
Apple doesn't address the modified photo replay situation, where you take a picture of an already edited image.
Photoshop / AI-gen an image -> display on a high-resolution monitor -> photograph the monitor with iPhone 18 Pro -> valid Apple Reference image.
To get valid reference photos, you can go to the actual physical location, put the iPhone/monitor in a cardboard box to block external light, then photograph the monitor. Paint the inside of the box using Vantablack (stopping reflections) and cover the LiDAR projector with tape.
I can't wait to see Apple Verified™ photos of UFOs flying over the Golden Gate Bridge.
Claim 7 in this patent application describes how depth sensors are used as part of an image authentication process, which would make such a workaround more difficult:
The Apple Reference Image feature is here launched on iPhone 18 Pro and iPhone 18 Pro Max that both have built-in LiDAR sensors that could be used for this process.
Apple's current implementation doesn't integrate LiDAR. And LiDAR wouldn't be enough here, it's trivial to block the projector and hide the dot pattern. No dot pattern = iPhone thinks the object is far away, which is what happens in landscape photos.
A better fix is to take photos with all three iPhone cameras simultaneously, ideally as a 2-3s video, and use the parallax/multiple perspectives to extract depth information. The video files (Possibly audio too) could also be included with the verified image as additional verification.
They can also prevent photos if iPhone detects the LiDAR sensor is covered, similar to how Meta does it with their camera glasses.
> it's trivial to block the projector and hide the dot pattern. No dot pattern = iPhone thinks the object is far away, which is what happens in landscape photos.
I've never looked at the LiDAR hardware, but where is the emitter in relation to the receiver. Why would the LiDAR not reflect off of whatever you're blocking it with and return a very short flight meaning it was very close?
I don't think it's an either or - additional data signals that need to correlate to authenticate will increase confidence. You can use multiple other signals to evaluate whether something is truly a landscape photo, and in that case not require a LiDAR capture, but if you are inside and at close range then you could assume that it should be part of scoring the authentication.
Similarly, LiDAR alone will help disqualify cases where someone is just taking a picture of e.g. a landscape target of the Golden Gate, but that it shown on a screen 1 meter away.
Right but I’d argue that realistically this feature is going to be most useful when taking photos of things reasonably close by, people especially, rather than landscape photography.
Still, that means that either the fake target scene and your screen presenting it would need to be outside of LiDAR sensor bounds, or you'd need to find a way to make the depth sensor data conform with your fake scene, both increasing the difficulty of producing a forgery.
> increasing the difficulty of producing a forgery
The problem with this thinking is twofold:
1) Whether it actually meaningfully increases the difficulty of a forgery remains to be seen. Despite their initial language about discerning real events, we see no details here about what scene information is used.
2) It increases the potential value of a forgery because now your forgery is attested by Apple.
So it either makes it easier to defraud people or more worthwhile to put in the effort to defraud people or both. None of those outcomes are great.
Or, probably just some optics, like a slanted $9 IR mirror [1] in front of it, to direct the lidar to the sky/absorption box. Then you can point at the high res HDR TV that's probably already in your living room.
Well, first, that's only expensive if you're poor. The world is absolutely full of people who can easily piss away your entire annual income throwing a house party.
But I really mean that if the lidar barely works outdoors anyway then actually you don't need to be 16 feet away at all.
Anyway, one may presume that they've thought about this.
Thought about it and also are bright enough not to fall for the “if a single person dies wearing a seat belt, we should abandon seat belts because they do no good at all” fallacy.
It’s almost certainly possible to fool v1 of this system, for some images, in some contexts. It would be shocking if the first implementation was completely perfect. But maybe it’s better than nothing?
> It’s almost certainly possible to fool v1 of this system, for some images, in some contexts. It would be shocking if the first implementation was completely perfect.
Knowing Apple, they've been working on and testing Apple Reference Image for years.
It being perfect isn't the issue; it's that random people on the internet who are just learning about this assume Apple's engineers haven't already thought about everything (and more) mentioned in this thread.
> It being perfect isn't the issue; it's that random people on the internet who are just learning about this assume Apple's engineers haven't already thought about everything (and more) mentioned in this thread.
Given how many bugs there are in macOS and how long they have remained there, I (who have been writing iOS apps from the release of the first retina iPod touch until AI got good) functionally agree with such people; at best, I think Apple's engineers haven't actually solved everything (and more) mentioned in this thread, even if every one of these things may have come up in discussions and even reached an official backlog or task list or similar.
I think this will depend on how it gets used. I can imagine numerous outcomes where it's in fact worse than nothing (significantly more effective blackmail, for instance).
While that is not quite my bar of confidence when implementing wide-reaching technologies that have numerous unexplored knock-on effects, I guess the calculus must have been different on Infinite Loop recently.
It also doesn't prevent you from staging an image or anything that's existed since photography was invented. But that's not the problem they're trying to solve.
> Today, powerful, widely available AI tools allow users to easily generate or alter photorealistic images to a degree that was difficult to imagine just a few years ago.
Photoshop has existed for decades and so has fake images. This is a low friction way to attest "this image came from an iPhone sensor and Apple approved it". It will still take the usual image forensics to determine if the scene it depicts is legitimate.
> "But that's not the problem they're trying to solve."
It is the problem that they say they're trying to solve, though. They specifically say "where the essential role of a photograph is to prove that something actually happened".
It fails the reasonable person test to say that the "something" in that phrase refers to the act of taking the photo itself.
Likewise in "distinguish between photographs that depict real events and...".
So, in your view, photography has been fatally flawed since the late 1800’s, and mere mitigation of AI image gen are insufficient if they don’t also solve actors impersonating real people?
> So, in your view, photography has been fatally flawed since the late 1800’s
Used in a capacity as evidence? Uh, yes? Duh? Do you seriously believe otherwise? Anyway, that scenario is made worse not better by Apple promising captured veracity.
This is so stupid. This makes it like, a thousand times harder to fake a photo than it would otherwise be. You pedants imagining a way to fake it doesn't change that.
> This makes it like, a thousand times harder to fake a photo than it would otherwise be.
The problem with this thinking is twofold:
1) Whether it actually meaningfully increases the difficulty of a forgery remains to be seen. Despite their initial language about discerning real events, we see no details here about what scene information is used.
2) It increases the potential value of a forgery because now your forgery is attested by Apple.
So it either makes it easier to defraud people or more worthwhile to put in the effort to defraud people or both. None of those outcomes are great.
This has been possible since the beginning of photography and yet I can’t think of a single scenario where people have been tricked by a staged photo. Yet every day hundreds of millions of people are being fooled by AI generated photos.
> I can't wait to see Apple Verified™ photos of UFOs flying over the Golden Gate Bridge.
While I'm on board with you about the inabsolute security of this (relative to what's typically expected of cryptographic systems), the fact that their 'verified' state requires a live certification and can be revoked means that the sensor responsible for obviously faked images will see those images and that device no longer certified.
It all relies a lot on trust in Apple, and integration with Apple, and relatively unmotivated attackers.
Sony's analogous solution (https://authenticity.sony.net/camera/en-us/) claims 3d depth information is built in, I'm sure Apple could do the same given at least some iPhone models have LiDAR on the back
This would work for close up shots taken on iPhone, but not landscape shots. The infrared dots the iPhone LiDAR projects are too weak to appear over long distances.
Also the dots can be trivially blocked by putting your finger over the sensor, sometimes improving photo quality. I do this frequently when I want to take a photo through a window. The absence of the dot matrix tells the iPhone to focus on the background far away instead of the windowpane.
You don't even have to travel to the location, you can just spoof GPS. And of course that will only be needed until some eastern european kid gets bored one weekend and the signing keys magically appear on pastebin.
To be fair Apple of all companies have the best shot at pulling it off. They've been perfecting their hardware security for years for other reasons and this is just another way to take advantage of that work. But yes, if someone breaks it then the trust is gone and it casts doubt on all of the photos that were ever captured using the broken system.
Is this really that big of a flaw in this implementation? I don't think it's worth the additional complexity to address it. (Encoding depth information in some way, trying to detect "flat" surfaces, whatever).
Discerning a camera taken image of an image is typically very very easy. The collors/exposure/etc will all be obviously wrong in ways to a human, even without doing any analysis.
You mean that it is sometimes very easy. But it is also sometimes impossible. You seem to be thinking only of poor quality photos of poor quality prints, but there's no basis for assuming those characteristics.
The fundamental issue isn't technical. It's that people will see the "certified real" tag and just take the image for face value of whatever narrative someone wants to convey. They'll see the "Real Photo, Verified by Apple" and their brain will short circuit [0]
I don't think we should have this, for that reason alone (but many others too).
I’m pretty sure “certified real” aren’t the words Apple will use, nor do they use it in this document. The words to describe the technology were chosen with care: semantic verification, attestation, tamper evident, etc.
To me the weakest spot of this whole endeavor is how this will create false confidence in a story just because the accompanying images pass Apple's verification.
Like with the Watch Ultra (attacking the diving-watch market with the sheer volume-scale of selling the development to everyone buying a Watch Ultra), Apple is attacking the trusted-imaging market with the same strategy.
Okay, fine. Will work for sure, this will disrupt the forensic-imaging market and moreover make Apple a service-provider in this industry.
But creating this impression and media-buzz that Apple is now verifying more than just the digital authenticity of an image may shift the public scrutiny of MANY media/online statements:
There is a risk that random claims (and propaganda) will be given more credibility in the public eye just because they came with images that were confirmed to be "taken like this on an iPhone"
That's a lot of words to say "we re-invented C2PA but made worse by getting our servers involved somehow".
Like with C2PA, the entire thing hinges on nobody being able to dump keys or trick the TPM into signing arbitrary image data. The timestamping server is a nice idea (though I don't see why they can't just use a normal timestamping server, I guess to keep control over the protocol) but it doesn't solve the fundamental problem that defeated C2PA.
It looks like the reason for the custom timestamp setup is to assert and upper and lower bound on time. A normal timestamp server can asset it saw the image at a certain time but not that the image wasn’t created much earlier. This setup, the image processing pipeline can immediately attach the last seen timestamp to the photo as a lower bound, and then connect to the network to get the upper bound time.
If there is too much of a gap between the upper and lower bounds then the image becomes suspicious.
The lower bound is specified by the device, you don't need support from the timestamping server for that. Determining if this timestamp is or isn't suspicious can be done at verification time. The timestamping feature itself makes sense from a verification perspective (though the privacy implications are questionable, of course), but I don't think it necessitates an Apple-specific setup.
This approach does have one benefit, which is that Apple gets all the (meta)data to determine if something is or isn't "real", rather than letting the verifier decide beforehand.
I can only imagine the outrage if Google or Microsoft added a "upload all of your photos to us and we will mark them are real or fake" protocol, even with all of the verified compute gaff.
I‘ve been wondering whether the contact tracking features introduced for Covid 19 could be used to verify that pictures of an event where taken by people who were actually around the scene. That way you‘d have some reassurance that a given picture was actually from the event. Combined with pictures from different angles from different people and some
kind of verified photography should make alterations harder.
I’m fairly sure that the contact tracing feature has been removed now. And it wouldn’t be needed anyway, the iPhone location services are far more useful. I imagine the geotag could be included with the verification.
Location services is quite hard to trick. To the point people have gone to the lengths of putting iPhones inside a microwave for RF shielding and setting up fake phone tower signals inside to trick the phone in to unlocking the hearing aid feature on AirPods for unapproved countries.
This is so insanely complex and requires placing trust in the correctness of so many pieces, many of them closed-source. And uploading every verified "developed" image to Apple's servers. And giving up full control of the software and hardware you "own". All to achieve a goal of "verifying" photons, which is only a part of the real problem of verifying the truth of an event that was photographed.
I hope that companies and governments don't start forcing us to use this stuff by requiring it for their services.
That's also why the approach is fundamentally flawed. The open-ish C2PA protocol has been "defeated" by tricking phones into signing arbitrary data already. The even-more-closed Apple version can be defeated the same way and relies on Apple to be the sole arbiter of truth.
This approach assumes that the smartphone in question is not under the user’s control. That should generally not be the case. When I buy a device, I have the right to install whatever I want on it and to make the camera sensors believe whatever I want. If something cannot be implemented securely under these circumstances, it’s not a good idea, and other solutions are needed. I once tested a video identification system for a company that the manufacturer claimed was absolutely secure. All it took was rooting the smartphone and bypassing the root detection. After that, you could play any pre recorded video, which would then be recognized as camera input. Under those conditions, it was easy to manipulate a video so that a company employee would consider it real enough to verify the test subject.
It’s simply not technically possible to verify the authenticity of the camera input with 100% certainty. Pretending that it is possible only creates problems. Then someone fakes evidence, but all the normies who have no clue about technology assume that it must be real. You see this with AI detectors too they recognize random texts as generated, yet an unbelievable number of people believe them.
How do you plan to replace the sensor of your phone's main camera (with a device you need), and let it authenticated by the OS, and then create authenticated photographs with it?
Apple/iOS already have part authentication pipeline on its security sensitive devices (TouchID/FaceID). How can camera sensor can't be considered one of those and needs attestation before enabling?
From the document:
> Apple Reference Image leverages custom-designed image sensors in iPhone 18 Pro and iPhone 18 Pro Max to ensure reliable capture of image data, and relies on Private Cloud Compute, which provides a computational environment for secure photographic processing that cannot be subverted even in the case of device compromise. (emphasis mine)
I think this is really good and kudos to Apple for implementing it. The first question that popped into my mind was "what new scenarios of government X forcing Apple to do 'terrible thing' to 'individual' this enables?", but I can't think of anything. It seems that all government attack vectors this feature enables are of the type "government X forces Apple to do 'terrible thing' to 'Apple'", i.e. a government can try to force Apple into certifying a narrative, and of course Apple is going to fight tooth and nail the lack of credibility that would result from that.
I mean, the obvious one is that they can revoke certification of a photo despite it being real.
The harder one is they can force apple to certify a fake photo.
the part that would be very hard but not outside the realm of plausibility, is that gov could force apple to introduce a bug in its pcc platform to link photos to the photographer in order to track and arrest inconvenient people. Apple says there are a bunch of protections against that but ultimately you are trusting apple to do it the way they say they are.
Though, on second thought, "government X could force Apple to disable feature for members of group Y" seems possible. I'm sure you can come with "Y" quite easily, heard anything about Ed Sheeran's tour?
> Modern cameras rely on sophisticated image-processing algorithms to produce the final viewable image, so certifying that an image accurately reflects what a real camera sensor captured requires a chain of trust covering the sensor as well as the computational photography software that interpreted the capture.
So if you jailbreak or root your phone what happens? Is this a trojan horse into making rooted phone cameras unverified? Just like how Linux machines can't watch Netflix in 4K
You'd need to jailbreak the camera sensor chip and the phone's secure element. Which isn't exactly impossible either, but it's harder. I don't think it has been done yet (but I'm sure it will be at some point).
It all depends on when or if Apple actually activates this system. Unless Apple can prove when a compromise first took place, they would have to retro-actively classify all iPhone pictures taken before discovery of such an exploit as "potentially fake".
Plenty of certification bodies refuse to revoke their given certifications because of brand damage or effects on their customers. That's why cryptographic verification of things like Secure Boot are basically broken on most systems by default.
If an independent security researcher does it and Apple rolls out fixes a month later, I can see it happening. If it turns out a government agency hacked the platform "at some point", I have my doubts Apple will retroactively reject all of their iPhones' signatures.
It seems like the two signatures on device are processed on the camera sensor itself, and then post processing is signed by the SEP. Neither of these would be compromised even if you have a full jailbreak.
Edit^2: On triple reread it sounds like the first pass ("Image Capture") sends the image metadata hash to be timestamped, whereas the second pass (Reference Image Development) sends the image itself but is not what actually creates the timestamp attestation. According to Apple[0][1] it sounds like the second pass (development) only happens when the reference image is actually viewed, which means that your image isn't sent if you never view the reference image?
> When you take a photo in Reference mode after tapping Reference Mode, your device will include reference image information in the photo’s metadata. If you then view that photo and tap the Reference badge on your iOS device or click it on your Mac, the device will send the raw photograph, metadata about the photograph like the sensor’s signatures and the time frame in which the photo was captured, as well as the sensor’s unique hardware identifiers to Private Cloud Compute.
Edit: On reread it seems they do in fact send the actual photographic data to PCC, which I presume has some reason over signing metadata on-device? Original mistaken post is below for transparency.
You can always not use the reference image mode, and according to the article you send a hash of the signature of the photograph, so all they would know is you took a photograph in reference image mode at some point in time before the request.
After my brief read, one of the main reasons they’re using PCC to produce the signed JPEGs, instead of doing everything on device, is that it maintains your privacy.
If you were signing with the iPhone, an attacker could then correlate photos taken with the same device.
Apple certainly has the data that “this sensor, in this device, took this exact photo” in PCC at the time of signing, but they discard that data.
Somewhat tangential but is "most secure consumer mobile device" actually correct? Does an iPhone beat out a grapheneOS android, or would that not be considered consumer because of aftermarket changes? Seems like a pretty bold claim but I know apple is pretty damn good with security (as long as you dont count Apple as a security risk themselves)
The timestamp system seems like it provides more benefit than signing the image data itself, at least in terms of difficulty to fake. As long as rolling back the stored timestamp token is prevented, I would have to find a phone that never updated its timestamp after the time I want to fake. Of course you could potentially find a phone that last connected to Apple's servers with a plausible timestamp. Even then the upper bound of when it signs the photo after reconnecting to the internet will raise eyebrows if you take too long to find the phone and fake the photo, so it effectively raises the bar to having to take the fake photo roughly simultaneously with the time the event purportedly took place anyway.
Apple's protocol differs in that it requires a timestamping server to sign the file and centralising Apple as the single arbiter of truth. An excellent addition, if you trust Apple and the governments they're friendly with (I don't, especially the latter part).
so what happens if you display an extremely high res image of a 100% AI generated fake-something on an 8K display in a photo studio room and take a picture of it with the camera? it gets tagged as authentic.
I skimmed the whole article and I didn’t see a single image so I’m confused. Is there some watermark of some kind or where is this metadata integrated? Because if it’s just metadata then I need to parse each photo I come across manually, and if it’s a watermark it can be faked because I won’t manually validate every single image I come across to prove the watermark isn’t fake.
presumably the metadata reader app would be integrated into the photo viewer app which would verify the digital sigs.
I have my doubts about this scheme but this is not one of them. If the point is that someone in principle could verify, that is enough for it to be useful, even if not everyone does.
Seems to lead us down the slippery slope of requiring an Apple device, or a Google device (e.g., https://cybernews.com/privacy/google-qr-code-recaptcha-requi...), or the device of some other entity (that may be mostly non-aligned with democratic values) in order to participate in society.
The unfortunate result of AI slop is reduced trust, which in turn is responded to with surveillance, which ultimately leads to the loss of liberty. Is it possible to do these sorts of verifications in an open way? I kinda doubt it, since someone has to control the hardware manufacturing process.
Hot damn. I've described this concept before, obviously not to this level of detail, but leaving this comment in here in case I can find my old comments. A bunch of people have poo-poo'd my proposals, but glad to see a serious actor really executing it. Probably no one at Apple ever read my posts, but it sure does feel good to see something executed. Hopefully it sticks.
Am wondering why no one is talking about traceability of Photos. Ex: CSAM which Apple was fighting for a long time. I thought this feature was the answer to provide proof of who shot the picture.
According to the description in the article, "an outside observer cannot determine whether any pair of reference images were taken by the same device." and they "avoid even implicit public association between different photos taken by the same sensor"
I dont think this is relavent to that use case. It seems like this proposal would be an optional off by default feature. They also seem to be going to lengths to make it privacy first so the verified photos cannot be linked to a specific photographer.
Nothing stops you not verifying, or simply stripping the verification off.
What you are prevented from doing is adding a verification to a photo outside of the iOS image pipeline, or modifying the photo with the verification still in tact.
This is cool, but also seem really complex and i'm not sure it makes sense pragmatically.
- it sounds like its an optional mode you have to enable. That kind of defeats the point if you need to prove something after the fact
- i guess you need internet to take a picture. :(
- You are puting a lot of trust in apple's private cloud compute platform.
- apple can revoke certification of a picture. I understand the appeal of this, all security systems eventually have failures, so its important to be robust against this. However if the point is to prove a picture is real (especially politically damaging ones), this is giving a lot of power to apple.
Its meant to be in competition with C2PA, and i guess the idea is its much more secure against complex hardware attacks. However i think its worth asking who the target audience is and what threats they face. The primary issue with AI is it makes fake photos easy, not that it invented fake photos. Even Stalin manipulated photos back in the day. It is not a new thing, the problem is just being overwhelmed with them.
with that in mind, are complex hardware attacks really that important? We just need to increase the difficulty floor, not solve fake photos for all time. No matter what you do, people can still use practical effects.
It seems like this is almost trying to thwart spies and nation state adversaries, well forgetting that such well funded groups have the budget to fake photos the old fashioned way or if they really cared, bribe their way into apple.
> it sounds like its an optional mode you have to enable
It’s opt-in because your photo is sent to Apple’s servers. Only if it were on-device should they even consider making it default.
> i guess you need internet to take a picture
Not really, internet is required to process the reference image, but that can happen later if you’re not currently connected.
> are complex hardware attacks really that important?
No, but the floor shouldn’t be “trivially exploitable” like C2PA[0]. It’d be interesting if there were a middle ground but we don’t have anything like that as of now.
I'm not sure i would describe the linked exploit as "trivial", but nonetheless point taken.
Ultimately though, i think all this might just mean we do not have a practical solution to this problem.
just to throw out some naive ideas, maybe the solution is to just sign the raw camera output and embed it in the metadata. If this is an optional feature meant for photojournalists, does file size really matter?
Apple has to allownpost-manufacruring exchanges of camera due ton right of repair legislation. This requires them to publish pairing tools that are to be used during the repair process to update all the cryptographic vérification chains in the device.
Now the camera module is supposed to generate a key pair internationally and send the public key over the bus. This looks like it is interceptable at repair time and a man in the middle can insert a different public key that they generated externally. Is there a way to stop this?
I feel a little discomfort about this: moving towards a world where photos were plausibly deniable felt good for privacy, this feels like a step further away.
Lots of criticism here but I think this is extremely promising. When this tech is extended to videos and perhaps even other forms of media, I think it has the potential of stopping all slop!
All slop? I'm sure that some "Shrimp Jesus" or "Talking Strawberry" was never considered to be authentic by anybody. There is a lot of useless AI generated content of which everybody knows it's AI generated littering the web. Having it marked as AI will not stop that.
This is really clever from Apple. The journalist use case is just the PR story. This will be really useful for identity verification and insurance apps, and has the potential to shift from "you need a smartphone to be able to live normally" to "you need an iPhone to be able to live normally".
There are already plenty of insurances that require you to submit claims through a smartphone app that tries to essentially do this by capturing sensor metadata etc. - those don't need to be nation-state resilient, just Joe the Crackhead Insurance Scammer resilient, so this works. Likewise, more and more things online require identity verification (either officially or disguised as age verification).
Edit: And while "a nation state actor can spoof this" is a problem for the journalism use case, the insurance/ID verification use cases are perfectly fine with anything that raises the bar but could be bypassed with enough effort. Also, the journalism use case suffers from the same fundamental issue all of these use cases suffer from: People will "verify" the picture by looking at the repost of a screenshot of the verification UI, not by verifying the original themselves.
I don't understand the vector of this:
An insurance would either send #1 an insurance agent or mechanic to initially assess the damage (trusted) or #2 ask the customer to send pictures (untrusted).
Tendency is #2 for cost-saving of the insurance, and 3rd party apps are used to execute this.
Now the idea is that the insurance company discontinues the App and the (untrusted) customer must have an iPhone 18 Pro to make an insurance claim?
Or is the insurance agent / mechanic an untrusted entity who will now be required to have an iPhone 18 Pro?
What is the fraud vector here, and how can the insurance service provider continue cost-saving on damage-assessment by offloading to the customer, if the customer is required to own a specific device?
> Now the idea is that the insurance company discontinues the App and the (untrusted) customer must have an iPhone 18 Pro to make an insurance claim?
In a couple of years it will be almost any iPhone instead of 18 Pro. And if it catches on, other phone vendors will provide a similar service.
Some banks needs photos of machine readable IDs to verify user details to fight fraud. These IDs can be passports or NFC enabled EU (and compatible) ID cards.
This will allow banks to trust these cameras more on the long run, allowing higher security ID checks.
Okay, this doesn't answer the question on the vector but is another interesting example. Let's expand on that one then:
Banks are offloading the trusted process of ID verification to an untrusted entity (end-user, merchant,...) and compensate for the loss of security by using a trusted service-provider (now Apple AND an iPhone 18 Pro).
This is already happening today in two scenarios:
1. lower-risk scenarios (remotely) with trusted 3rd party service-providers and very low Hardware-requirements ("use this app on your phone to take a picture/video") and
2. higher-risk scenarios (on-site) with trusted 3rd party service-providers ("use THIS expensive device to take a picture/video of the customer/citizen")
Apple now potentially disrupts the service-provider industry of #2 (higher-risk scenarios) by
#a grabbing a part of this hardware/service market that MAY allow the end-user to be in control of the device and
#b replacing the on-site hardware/service with an iPhone "in a box".
They can't disrupt #1 because their cost-saving can't mandate the end-user to buy a 1000+ USD device just for THEM to provide the contracted service. (They can add convenience if you have it, but they can't reject their service if you don't)
Which means they disrupt mainly #2: The industry providing trusted imaging solutions for higher-risk scenarios.
--> So it's the Watch Ultra game all over again.
On Watch Ultra they disrupted the diving-watch market by the sheer scale of selling their development to everyone buying a Watch Ultra, driving down the cost so much that they can undercut every diving-watch company on the market.
Now they use the sheer scale of iPhone 18 Pro sales to enter the trusted-imaging market-segment, undercutting every player there and take that market.
> […] the potential to shift from "you need a smartphone to be able to live normally" to "you need an iPhone to be able to live normally".
Why do you believe Android manufacturers and SOC makers like Qualcomm won’t be able to offer a similar solution?
Not the OP, but yes, other vendors will be able to support that as well. But a camera sensor that has
1. a public/private key exchanged during device-production (production-cost),
2. the capability to reboot in a cryptographic mode (R&D / component cost) and
3. a cloud-service which then processes the raw data to create a JPG (operational cost)
comes at a premium. Why should this premium be applied on a 99 USD Smartphone?
Which is my whole puzzle on this vector: If the big benefit is for insurance/ID-verification, which apply cost-saving by offloading their process to the untrusted customer, how much they can offload this by requiring their customer to own a 1000+ USD smartphone to provide THEIR service...?
The most I can imagine is insurances offloading their work to OTHER companies, NOT trusting them and therefore requiring them to own a 1000+ USD Smartphone. But even then, why not use a third party app that also runs on a 3y old iPhone and a 99 USD Android device...?
We have 99USD smartphones with 1080p+ AMOLEDs, massive 5k amp batteries and very performant SOCs (e.g. Galaxy A16) among other costly, but not vital niceties. I struggle to see how cost could be a factor here.
A bunch of them already offer one. Have been a while, actually; the S25 and Pixel 10 came with exactly this.
The timestamping server is the hard part, especially with the verified compute component. It's just not something I see Samsung doing.
I expect Google to show up with a blog post titled "extending C2PA with timestamps for industry-leading authenticity confirmation" any time.
I think you’re on the ball, but also all KYC flows, photo proof for shipping returns (Chinese platforms were getting destroyed on this) etc. etc. It’s a very very clever solution and a very opportune time.
The main way we combat insurance fraud is by throwing people in jail who do it. I dont think AI faked photos is a major cause of fraud.
At least in the UK this seems to be a growing problem and jail isn't a scalable solution. See for example: https://www.bbc.com/news/articles/cm2rr9pg4jzo
Like a lot of things in the UK, the problem isn't the harshness of the law, it's the lack of prosecution. It doesn't matter what the punishment is when criminals know that the crime is extremely unlikely to be investigated or make it to trial.
I don’t understand what this brings to the table beyond what we’re currently doing.
Insurance companies can have a native app and require the device’s camera. Companies already have tools to combat a liveliness check. Even if you’re using a modified app that pulls from the photo album instead of the camera? A video recording with the appropriate liveness verification easily avoids that mess.
As per opening paragraph of link, AI fakes are a thing.
It's been possible to do a live video deepfake for a long time now, but as with all new tech, law and society are taking their sweet time to understand the risks; IMO this is the other side of the same coin as some infamous tech comments on consumer products: https://news.ycombinator.com/item?id=9224 and https://en.wikiquote.org/wiki/Rob_Malda
NVIDIA suggested AI fakes controlled with face tracking input as a compression technique just for reducing video call bandwidth requirements (to ~117 bytes per frame). They did that six years ago: https://www.dpreview.com/news/5756257699/nvidia-research-dev...
As we're now in an AI race, even NVIDIA's specific technique has flaws which all the current tools can detect, there's never any guarantee of this continuing to be the case.
That said, in the case of Apple, they're historically followers not leaders despite the public image they like to present about innovation, and I'd expect this method to be flawed from day one even if we weren't reading a corporate blog post written in a self-congratulatory tone I find almost as off-putting as when AI write.
> That said, in the case of Apple, they're historically followers not leaders despite the public image they like to present about innovation
While its true Apple usually isn't the first in a product category--not the first mp3 player, not the first smartphone, not the first tablet) but once they get there, they're quite innovative.
When the iPhone 5s was released in 2013, it was the first smartphone with a 64-bit processor, which caught Qualcomm off guard. Even when Qualcomm released a 64-bit processor the following year, it kinda didn’t matter because Android was still 32-bit.
You don’t even need an AI deepfake to edit a video.
AI deepfake or edit video doesn’t pass liveliness checks without all the c2pa or reference image song and pony show.
Insurance companies can monitor the light reflections from the flash that they control or monitor the accelerometer and compare the accelerometer values with the video that they receive.
They could also just update their app to stop accepting photos from the album.
Insurance agencies worried about fraud could just do what they did prior to smartphones: have you bring the car to a claims adjuster at your local office. I’ve brought cars to be inspected, because I got T-boned at an intersection by a careless driver before smartphones existed; it was reasonably quick and hassle-free.
> People will "verify" the picture by looking at the repost of a screenshot of the verification UI, not by verifying the original themselves.
True.
> raises the bar but could be bypassed with enough effort.
Anyone can spoof this.
Apple cannot stop spam iMessages. They can't stop someone from rendering their privileged UI inside a browser viewport. People copy and paste remote script executions from convincing captchas.
This whole provenance thing is a red herring. You agree with me, but there's truly not a single application for this that won't be exploited.
Apple doesn't address the modified photo replay situation, where you take a picture of an already edited image.
Photoshop / AI-gen an image -> display on a high-resolution monitor -> photograph the monitor with iPhone 18 Pro -> valid Apple Reference image.
To get valid reference photos, you can go to the actual physical location, put the iPhone/monitor in a cardboard box to block external light, then photograph the monitor. Paint the inside of the box using Vantablack (stopping reflections) and cover the LiDAR projector with tape.
I can't wait to see Apple Verified™ photos of UFOs flying over the Golden Gate Bridge.
Claim 7 in this patent application describes how depth sensors are used as part of an image authentication process, which would make such a workaround more difficult:
https://image-ppubs.uspto.gov/dirsearch-public/print/downloa...
The Apple Reference Image feature is here launched on iPhone 18 Pro and iPhone 18 Pro Max that both have built-in LiDAR sensors that could be used for this process.
Apple's current implementation doesn't integrate LiDAR. And LiDAR wouldn't be enough here, it's trivial to block the projector and hide the dot pattern. No dot pattern = iPhone thinks the object is far away, which is what happens in landscape photos.
A better fix is to take photos with all three iPhone cameras simultaneously, ideally as a 2-3s video, and use the parallax/multiple perspectives to extract depth information. The video files (Possibly audio too) could also be included with the verified image as additional verification.
They can also prevent photos if iPhone detects the LiDAR sensor is covered, similar to how Meta does it with their camera glasses.
> it's trivial to block the projector and hide the dot pattern. No dot pattern = iPhone thinks the object is far away, which is what happens in landscape photos.
I've never looked at the LiDAR hardware, but where is the emitter in relation to the receiver. Why would the LiDAR not reflect off of whatever you're blocking it with and return a very short flight meaning it was very close?
Paint the stopper vantablack then.
I knew keeping a pot of vantablack in my jacket pocket would come in handy.
I don't think it's an either or - additional data signals that need to correlate to authenticate will increase confidence. You can use multiple other signals to evaluate whether something is truly a landscape photo, and in that case not require a LiDAR capture, but if you are inside and at close range then you could assume that it should be part of scoring the authentication.
Similarly, LiDAR alone will help disqualify cases where someone is just taking a picture of e.g. a landscape target of the Golden Gate, but that it shown on a screen 1 meter away.
Right but I’d argue that realistically this feature is going to be most useful when taking photos of things reasonably close by, people especially, rather than landscape photography.
This approach makes me wonder if the future is actually going to move towards visual cryptography.
iPhone lidar only works up to like 16 feet in the easiest lighting conditions (indoors) and may be functionally ineffective outdoors.
Still, that means that either the fake target scene and your screen presenting it would need to be outside of LiDAR sensor bounds, or you'd need to find a way to make the depth sensor data conform with your fake scene, both increasing the difficulty of producing a forgery.
https://news.ycombinator.com/item?id=49721878
> increasing the difficulty of producing a forgery
The problem with this thinking is twofold:
1) Whether it actually meaningfully increases the difficulty of a forgery remains to be seen. Despite their initial language about discerning real events, we see no details here about what scene information is used.
2) It increases the potential value of a forgery because now your forgery is attested by Apple.
So it either makes it easier to defraud people or more worthwhile to put in the effort to defraud people or both. None of those outcomes are great.
So you need a big enough screen to cover the entire field of view at 16 ft? Sounds expensive
Or, probably just some optics, like a slanted $9 IR mirror [1] in front of it, to direct the lidar to the sky/absorption box. Then you can point at the high res HDR TV that's probably already in your living room.
[1] https://commonlands.com/products/ir-cut-filters-csp650?srslt...
Well, first, that's only expensive if you're poor. The world is absolutely full of people who can easily piss away your entire annual income throwing a house party.
But I really mean that if the lidar barely works outdoors anyway then actually you don't need to be 16 feet away at all.
Anyway, one may presume that they've thought about this.
Thought about it and also are bright enough not to fall for the “if a single person dies wearing a seat belt, we should abandon seat belts because they do no good at all” fallacy.
It’s almost certainly possible to fool v1 of this system, for some images, in some contexts. It would be shocking if the first implementation was completely perfect. But maybe it’s better than nothing?
The problem is that if defeating it is trivial, then it _authenticates_ fake images.
The problem is that it makes it easier to fool people and provide "cryptographic" evidence of validity, backed by big tech.
It's purpose is to stop bad actors from passing of fake as real just as much as it is to prevent real images being dismissed as fake.
> It’s almost certainly possible to fool v1 of this system, for some images, in some contexts. It would be shocking if the first implementation was completely perfect.
Knowing Apple, they've been working on and testing Apple Reference Image for years.
It being perfect isn't the issue; it's that random people on the internet who are just learning about this assume Apple's engineers haven't already thought about everything (and more) mentioned in this thread.
> It being perfect isn't the issue; it's that random people on the internet who are just learning about this assume Apple's engineers haven't already thought about everything (and more) mentioned in this thread.
Given how many bugs there are in macOS and how long they have remained there, I (who have been writing iOS apps from the release of the first retina iPod touch until AI got good) functionally agree with such people; at best, I think Apple's engineers haven't actually solved everything (and more) mentioned in this thread, even if every one of these things may have come up in discussions and even reached an official backlog or task list or similar.
If they thought about it, then why isn't this very obvious failure point even mentioned in the technical breakdown?
> But maybe it’s better than nothing?
I think this will depend on how it gets used. I can imagine numerous outcomes where it's in fact worse than nothing (significantly more effective blackmail, for instance).
> maybe it’s better than nothing?
While that is not quite my bar of confidence when implementing wide-reaching technologies that have numerous unexplored knock-on effects, I guess the calculus must have been different on Infinite Loop recently.
I like that seatbelt argument example.
I’ve seen that type of argument a million times, and I’ll certainly reuse that.
Put some sugar on it. Please.
Def Leppard, is that you?
furthermore, couldnt you do parallax from the multiple cameras as well as flicker the flash?
seems pretty easy to make it sufficiently difficult to trick the system
It also doesn't prevent you from staging an image or anything that's existed since photography was invented. But that's not the problem they're trying to solve.
> Today, powerful, widely available AI tools allow users to easily generate or alter photorealistic images to a degree that was difficult to imagine just a few years ago.
Photoshop has existed for decades and so has fake images. This is a low friction way to attest "this image came from an iPhone sensor and Apple approved it". It will still take the usual image forensics to determine if the scene it depicts is legitimate.
> "But that's not the problem they're trying to solve."
It is the problem that they say they're trying to solve, though. They specifically say "where the essential role of a photograph is to prove that something actually happened".
It fails the reasonable person test to say that the "something" in that phrase refers to the act of taking the photo itself.
Likewise in "distinguish between photographs that depict real events and...".
Yes, it is proving something actually happened, that is your monitor screen showing something you photographed.
Lol.
No security control is perfect. The point is to increase costs to the point its unfeasible.
After all, if money is no object, you could just bribe every single apple employee involved in the project.
So, in your view, photography has been fatally flawed since the late 1800’s, and mere mitigation of AI image gen are insufficient if they don’t also solve actors impersonating real people?
> So, in your view, photography has been fatally flawed since the late 1800’s
Used in a capacity as evidence? Uh, yes? Duh? Do you seriously believe otherwise? Anyway, that scenario is made worse not better by Apple promising captured veracity.
This is so stupid. This makes it like, a thousand times harder to fake a photo than it would otherwise be. You pedants imagining a way to fake it doesn't change that.
> This makes it like, a thousand times harder to fake a photo than it would otherwise be.
The problem with this thinking is twofold:
1) Whether it actually meaningfully increases the difficulty of a forgery remains to be seen. Despite their initial language about discerning real events, we see no details here about what scene information is used.
2) It increases the potential value of a forgery because now your forgery is attested by Apple.
So it either makes it easier to defraud people or more worthwhile to put in the effort to defraud people or both. None of those outcomes are great.
You worry that a technology that you have never used nor evaluated might not work in practice...
Therefore because of your worry (which is based on remarkably little information), it's a bad technology?
Come the fuck on. That's beyond luddite bullshit.
> because of your worry (which is based on remarkably little information)
You must be new around here. ;-)
Is this you? https://news.ycombinator.com/item?id=49685271
This has been possible since the beginning of photography and yet I can’t think of a single scenario where people have been tricked by a staged photo. Yet every day hundreds of millions of people are being fooled by AI generated photos.
This sounds like it could be done, but the costs for doing so are comparably high.
I think the idea is to control the easy, cheap mass production of AI gen picture and not 100% coverage.
That’s a tradeoff I can live with.
> I can't wait to see Apple Verified™ photos of UFOs flying over the Golden Gate Bridge.
While I'm on board with you about the inabsolute security of this (relative to what's typically expected of cryptographic systems), the fact that their 'verified' state requires a live certification and can be revoked means that the sensor responsible for obviously faked images will see those images and that device no longer certified.
It all relies a lot on trust in Apple, and integration with Apple, and relatively unmotivated attackers.
> Paint the inside of the box using Vantablack
But you're only allowed to do that if your name if Anish Kapoor
Sony's analogous solution (https://authenticity.sony.net/camera/en-us/) claims 3d depth information is built in, I'm sure Apple could do the same given at least some iPhone models have LiDAR on the back
This would work for close up shots taken on iPhone, but not landscape shots. The infrared dots the iPhone LiDAR projects are too weak to appear over long distances.
Also the dots can be trivially blocked by putting your finger over the sensor, sometimes improving photo quality. I do this frequently when I want to take a photo through a window. The absence of the dot matrix tells the iPhone to focus on the background far away instead of the windowpane.
> I do this frequently when I want to take a photo through a window.
I feel really dumb for not having thought of this.
>I can't wait to see Apple Verified™ photos of UFOs flying over the Golden Gate Bridge.
There’s no such thing as a Golden Gate Bridge.
Prove it.
Won't the focus length of the camera be wrong?
I suspect they have ways to ID at least some things like this somehow in ways that will lead to key revocation.
It's less about proving a photo's truth than about attesting it.
Yeah, such systems have been tried (and been hacked) for decades now.
https://www.elcomsoft.com/news/428.html
https://blog.elcomsoft.com/2011/04/nikon-image-authenticatio...
You don't even have to travel to the location, you can just spoof GPS. And of course that will only be needed until some eastern european kid gets bored one weekend and the signing keys magically appear on pastebin.
It's funny to see Apple fall into this same trap.
To be fair Apple of all companies have the best shot at pulling it off. They've been perfecting their hardware security for years for other reasons and this is just another way to take advantage of that work. But yes, if someone breaks it then the trust is gone and it casts doubt on all of the photos that were ever captured using the broken system.
Is this really that big of a flaw in this implementation? I don't think it's worth the additional complexity to address it. (Encoding depth information in some way, trying to detect "flat" surfaces, whatever).
Discerning a camera taken image of an image is typically very very easy. The collors/exposure/etc will all be obviously wrong in ways to a human, even without doing any analysis.
You mean that it is sometimes very easy. But it is also sometimes impossible. You seem to be thinking only of poor quality photos of poor quality prints, but there's no basis for assuming those characteristics.
Exactly, a wave of “verified” fake images are coming.
The fundamental issue isn't technical. It's that people will see the "certified real" tag and just take the image for face value of whatever narrative someone wants to convey. They'll see the "Real Photo, Verified by Apple" and their brain will short circuit [0]
I don't think we should have this, for that reason alone (but many others too).
[0]: https://imgur.com/fVPkpuQ
I’m pretty sure “certified real” aren’t the words Apple will use, nor do they use it in this document. The words to describe the technology were chosen with care: semantic verification, attestation, tamper evident, etc.
To me the weakest spot of this whole endeavor is how this will create false confidence in a story just because the accompanying images pass Apple's verification.
Like with the Watch Ultra (attacking the diving-watch market with the sheer volume-scale of selling the development to everyone buying a Watch Ultra), Apple is attacking the trusted-imaging market with the same strategy.
Okay, fine. Will work for sure, this will disrupt the forensic-imaging market and moreover make Apple a service-provider in this industry.
But creating this impression and media-buzz that Apple is now verifying more than just the digital authenticity of an image may shift the public scrutiny of MANY media/online statements:
There is a risk that random claims (and propaganda) will be given more credibility in the public eye just because they came with images that were confirmed to be "taken like this on an iPhone"
That's a lot of words to say "we re-invented C2PA but made worse by getting our servers involved somehow".
Like with C2PA, the entire thing hinges on nobody being able to dump keys or trick the TPM into signing arbitrary image data. The timestamping server is a nice idea (though I don't see why they can't just use a normal timestamping server, I guess to keep control over the protocol) but it doesn't solve the fundamental problem that defeated C2PA.
It looks like the reason for the custom timestamp setup is to assert and upper and lower bound on time. A normal timestamp server can asset it saw the image at a certain time but not that the image wasn’t created much earlier. This setup, the image processing pipeline can immediately attach the last seen timestamp to the photo as a lower bound, and then connect to the network to get the upper bound time.
If there is too much of a gap between the upper and lower bounds then the image becomes suspicious.
The lower bound is specified by the device, you don't need support from the timestamping server for that. Determining if this timestamp is or isn't suspicious can be done at verification time. The timestamping feature itself makes sense from a verification perspective (though the privacy implications are questionable, of course), but I don't think it necessitates an Apple-specific setup.
This approach does have one benefit, which is that Apple gets all the (meta)data to determine if something is or isn't "real", rather than letting the verifier decide beforehand.
I can only imagine the outrage if Google or Microsoft added a "upload all of your photos to us and we will mark them are real or fake" protocol, even with all of the verified compute gaff.
I‘ve been wondering whether the contact tracking features introduced for Covid 19 could be used to verify that pictures of an event where taken by people who were actually around the scene. That way you‘d have some reassurance that a given picture was actually from the event. Combined with pictures from different angles from different people and some kind of verified photography should make alterations harder.
I’m fairly sure that the contact tracing feature has been removed now. And it wouldn’t be needed anyway, the iPhone location services are far more useful. I imagine the geotag could be included with the verification.
Location services is quite hard to trick. To the point people have gone to the lengths of putting iPhones inside a microwave for RF shielding and setting up fake phone tower signals inside to trick the phone in to unlocking the hearing aid feature on AirPods for unapproved countries.
This is so insanely complex and requires placing trust in the correctness of so many pieces, many of them closed-source. And uploading every verified "developed" image to Apple's servers. And giving up full control of the software and hardware you "own". All to achieve a goal of "verifying" photons, which is only a part of the real problem of verifying the truth of an event that was photographed.
I hope that companies and governments don't start forcing us to use this stuff by requiring it for their services.
Because it’s impossible to implement this feature in open source and out in the open. It relies on a locked down image pipeline and hidden key.
That's also why the approach is fundamentally flawed. The open-ish C2PA protocol has been "defeated" by tricking phones into signing arbitrary data already. The even-more-closed Apple version can be defeated the same way and relies on Apple to be the sole arbiter of truth.
This approach assumes that the smartphone in question is not under the user’s control. That should generally not be the case. When I buy a device, I have the right to install whatever I want on it and to make the camera sensors believe whatever I want. If something cannot be implemented securely under these circumstances, it’s not a good idea, and other solutions are needed. I once tested a video identification system for a company that the manufacturer claimed was absolutely secure. All it took was rooting the smartphone and bypassing the root detection. After that, you could play any pre recorded video, which would then be recognized as camera input. Under those conditions, it was easy to manipulate a video so that a company employee would consider it real enough to verify the test subject.
It’s simply not technically possible to verify the authenticity of the camera input with 100% certainty. Pretending that it is possible only creates problems. Then someone fakes evidence, but all the normies who have no clue about technology assume that it must be real. You see this with AI detectors too they recognize random texts as generated, yet an unbelievable number of people believe them.
How do you plan to replace the sensor of your phone's main camera (with a device you need), and let it authenticated by the OS, and then create authenticated photographs with it?
Apple/iOS already have part authentication pipeline on its security sensitive devices (TouchID/FaceID). How can camera sensor can't be considered one of those and needs attestation before enabling?
From the document:
> Apple Reference Image leverages custom-designed image sensors in iPhone 18 Pro and iPhone 18 Pro Max to ensure reliable capture of image data, and relies on Private Cloud Compute, which provides a computational environment for secure photographic processing that cannot be subverted even in the case of device compromise. (emphasis mine)
I think this is really good and kudos to Apple for implementing it. The first question that popped into my mind was "what new scenarios of government X forcing Apple to do 'terrible thing' to 'individual' this enables?", but I can't think of anything. It seems that all government attack vectors this feature enables are of the type "government X forces Apple to do 'terrible thing' to 'Apple'", i.e. a government can try to force Apple into certifying a narrative, and of course Apple is going to fight tooth and nail the lack of credibility that would result from that.
I mean, the obvious one is that they can revoke certification of a photo despite it being real.
The harder one is they can force apple to certify a fake photo.
the part that would be very hard but not outside the realm of plausibility, is that gov could force apple to introduce a bug in its pcc platform to link photos to the photographer in order to track and arrest inconvenient people. Apple says there are a bunch of protections against that but ultimately you are trusting apple to do it the way they say they are.
This entire system relies on trusting apple
Though, on second thought, "government X could force Apple to disable feature for members of group Y" seems possible. I'm sure you can come with "Y" quite easily, heard anything about Ed Sheeran's tour?
Apple would have to be the only company around and by the time Apple “loses” in court its too late.
> Modern cameras rely on sophisticated image-processing algorithms to produce the final viewable image, so certifying that an image accurately reflects what a real camera sensor captured requires a chain of trust covering the sensor as well as the computational photography software that interpreted the capture.
So if you jailbreak or root your phone what happens? Is this a trojan horse into making rooted phone cameras unverified? Just like how Linux machines can't watch Netflix in 4K
You'd need to jailbreak the camera sensor chip and the phone's secure element. Which isn't exactly impossible either, but it's harder. I don't think it has been done yet (but I'm sure it will be at some point).
On top of that, they have a revocation system in place to try and deal with that eventuality.
It all depends on when or if Apple actually activates this system. Unless Apple can prove when a compromise first took place, they would have to retro-actively classify all iPhone pictures taken before discovery of such an exploit as "potentially fake".
Plenty of certification bodies refuse to revoke their given certifications because of brand damage or effects on their customers. That's why cryptographic verification of things like Secure Boot are basically broken on most systems by default.
If an independent security researcher does it and Apple rolls out fixes a month later, I can see it happening. If it turns out a government agency hacked the platform "at some point", I have my doubts Apple will retroactively reject all of their iPhones' signatures.
It seems like the two signatures on device are processed on the camera sensor itself, and then post processing is signed by the SEP. Neither of these would be compromised even if you have a full jailbreak.
Seems kind of concerning that using this at all means you send your image to Apple’s PCC machines.
Presumably you would only do this for images you plan on sharing to social media anyways, to prove that it's not AI generated.
PCC is quite good, about as close to private remote compute we can get without doing HME.
If homomorphic encryption is not involved, how does Apple not have access to the raw image data being sent to PCC? (Genuine question)
It would be incorrect to presume that.
Edit^2: On triple reread it sounds like the first pass ("Image Capture") sends the image metadata hash to be timestamped, whereas the second pass (Reference Image Development) sends the image itself but is not what actually creates the timestamp attestation. According to Apple[0][1] it sounds like the second pass (development) only happens when the reference image is actually viewed, which means that your image isn't sent if you never view the reference image?
[0] https://support.apple.com/guide/iphone/view-reference-images...
[1] https://www.apple.com/legal/privacy/data/en/reference-image/
> When you take a photo in Reference mode after tapping Reference Mode, your device will include reference image information in the photo’s metadata. If you then view that photo and tap the Reference badge on your iOS device or click it on your Mac, the device will send the raw photograph, metadata about the photograph like the sensor’s signatures and the time frame in which the photo was captured, as well as the sensor’s unique hardware identifiers to Private Cloud Compute.
Edit: On reread it seems they do in fact send the actual photographic data to PCC, which I presume has some reason over signing metadata on-device? Original mistaken post is below for transparency.
You can always not use the reference image mode, and according to the article you send a hash of the signature of the photograph, so all they would know is you took a photograph in reference image mode at some point in time before the request.
> some reason over signing metadata on-device
After my brief read, one of the main reasons they’re using PCC to produce the signed JPEGs, instead of doing everything on device, is that it maintains your privacy.
If you were signing with the iPhone, an attacker could then correlate photos taken with the same device.
Apple certainly has the data that “this sensor, in this device, took this exact photo” in PCC at the time of signing, but they discard that data.
Somewhat tangential but is "most secure consumer mobile device" actually correct? Does an iPhone beat out a grapheneOS android, or would that not be considered consumer because of aftermarket changes? Seems like a pretty bold claim but I know apple is pretty damn good with security (as long as you dont count Apple as a security risk themselves)
> or via software-level jailbreak of the device.
I’m surprised that even Apple calls jailbreaking, jailbreaking. Doesn’t that imply their own software is a jail?
The timestamp system seems like it provides more benefit than signing the image data itself, at least in terms of difficulty to fake. As long as rolling back the stored timestamp token is prevented, I would have to find a phone that never updated its timestamp after the time I want to fake. Of course you could potentially find a phone that last connected to Apple's servers with a plausible timestamp. Even then the upper bound of when it signs the photo after reconnecting to the internet will raise eyebrows if you take too long to find the phone and fake the photo, so it effectively raises the bar to having to take the fake photo roughly simultaneously with the time the event purportedly took place anyway.
Hey I predicted this awhile ago. Although it is kind of an obvious solution so I can’t claim much insight hehe.
https://news.ycombinator.com/item?id=44135416
By the time you made your comment, such a system had already been invented, even partially rolled out: https://en.wikipedia.org/wiki/Content_Credentials
Apple's protocol differs in that it requires a timestamping server to sign the file and centralising Apple as the single arbiter of truth. An excellent addition, if you trust Apple and the governments they're friendly with (I don't, especially the latter part).
At this point it'd be easier to return to film.
Some of us never left :D
so what happens if you display an extremely high res image of a 100% AI generated fake-something on an 8K display in a photo studio room and take a picture of it with the camera? it gets tagged as authentic.
Was photo not authentic? Think of it "as seen by an iPhone", not "this is authentic event" verification.
But Apple likely would reject such photo because of inappropriate depth map / LiDAR data.
The timestamp wouldn’t match the event being depicted and the geotag would show the studio. And the depth sensing would show the image as flat.
That’s a lot of money and effort for a fake photo
I skimmed the whole article and I didn’t see a single image so I’m confused. Is there some watermark of some kind or where is this metadata integrated? Because if it’s just metadata then I need to parse each photo I come across manually, and if it’s a watermark it can be faked because I won’t manually validate every single image I come across to prove the watermark isn’t fake.
presumably the metadata reader app would be integrated into the photo viewer app which would verify the digital sigs.
I have my doubts about this scheme but this is not one of them. If the point is that someone in principle could verify, that is enough for it to be useful, even if not everyone does.
Seems to lead us down the slippery slope of requiring an Apple device, or a Google device (e.g., https://cybernews.com/privacy/google-qr-code-recaptcha-requi...), or the device of some other entity (that may be mostly non-aligned with democratic values) in order to participate in society.
The unfortunate result of AI slop is reduced trust, which in turn is responded to with surveillance, which ultimately leads to the loss of liberty. Is it possible to do these sorts of verifications in an open way? I kinda doubt it, since someone has to control the hardware manufacturing process.
Hot damn. I've described this concept before, obviously not to this level of detail, but leaving this comment in here in case I can find my old comments. A bunch of people have poo-poo'd my proposals, but glad to see a serious actor really executing it. Probably no one at Apple ever read my posts, but it sure does feel good to see something executed. Hopefully it sticks.
A photograph by itself should never be considered proof of anything.
Don’t know why this was downvoted but this is the right take. A photograph is evidence, but isn’t a proof in and of itself.
Am wondering why no one is talking about traceability of Photos. Ex: CSAM which Apple was fighting for a long time. I thought this feature was the answer to provide proof of who shot the picture.
According to the description in the article, "an outside observer cannot determine whether any pair of reference images were taken by the same device." and they "avoid even implicit public association between different photos taken by the same sensor"
Whatever that means in detail...
I dont think this is relavent to that use case. It seems like this proposal would be an optional off by default feature. They also seem to be going to lengths to make it privacy first so the verified photos cannot be linked to a specific photographer.
Nothing stops you not verifying, or simply stripping the verification off.
What you are prevented from doing is adding a verification to a photo outside of the iOS image pipeline, or modifying the photo with the verification still in tact.
Or have I understood the feature's capability completely wrong?
This is cool, but also seem really complex and i'm not sure it makes sense pragmatically.
- it sounds like its an optional mode you have to enable. That kind of defeats the point if you need to prove something after the fact
- i guess you need internet to take a picture. :(
- You are puting a lot of trust in apple's private cloud compute platform.
- apple can revoke certification of a picture. I understand the appeal of this, all security systems eventually have failures, so its important to be robust against this. However if the point is to prove a picture is real (especially politically damaging ones), this is giving a lot of power to apple.
Its meant to be in competition with C2PA, and i guess the idea is its much more secure against complex hardware attacks. However i think its worth asking who the target audience is and what threats they face. The primary issue with AI is it makes fake photos easy, not that it invented fake photos. Even Stalin manipulated photos back in the day. It is not a new thing, the problem is just being overwhelmed with them.
with that in mind, are complex hardware attacks really that important? We just need to increase the difficulty floor, not solve fake photos for all time. No matter what you do, people can still use practical effects.
It seems like this is almost trying to thwart spies and nation state adversaries, well forgetting that such well funded groups have the budget to fake photos the old fashioned way or if they really cared, bribe their way into apple.
> it sounds like its an optional mode you have to enable
It’s opt-in because your photo is sent to Apple’s servers. Only if it were on-device should they even consider making it default.
> i guess you need internet to take a picture
Not really, internet is required to process the reference image, but that can happen later if you’re not currently connected.
> are complex hardware attacks really that important?
No, but the floor shouldn’t be “trivially exploitable” like C2PA[0]. It’d be interesting if there were a middle ground but we don’t have anything like that as of now.
[0] https://www.da.vidbuchanan.co.uk/blog/android-c2pa.html
I'm not sure i would describe the linked exploit as "trivial", but nonetheless point taken.
Ultimately though, i think all this might just mean we do not have a practical solution to this problem.
just to throw out some naive ideas, maybe the solution is to just sign the raw camera output and embed it in the metadata. If this is an optional feature meant for photojournalists, does file size really matter?
Apple has to allownpost-manufacruring exchanges of camera due ton right of repair legislation. This requires them to publish pairing tools that are to be used during the repair process to update all the cryptographic vérification chains in the device.
Now the camera module is supposed to generate a key pair internationally and send the public key over the bus. This looks like it is interceptable at repair time and a man in the middle can insert a different public key that they generated externally. Is there a way to stop this?
It only works in one of the 3 lenses though.
85 comments, 5 days ago: https://news.ycombinator.com/item?id=49649222
What if someone take the photo of the forged photo displayed on another device, doesn’t the forged photo become an authentic one?
Sure, if it’s believable that the shot was perfectly flat at, what, 2 feet away?
I feel a little discomfort about this: moving towards a world where photos were plausibly deniable felt good for privacy, this feels like a step further away.
Lots of criticism here but I think this is extremely promising. When this tech is extended to videos and perhaps even other forms of media, I think it has the potential of stopping all slop!
All slop? I'm sure that some "Shrimp Jesus" or "Talking Strawberry" was never considered to be authentic by anybody. There is a lot of useless AI generated content of which everybody knows it's AI generated littering the web. Having it marked as AI will not stop that.
How do you figure?
More sales to Apple, to prove that your image is real.
Hardware wins.
NFTs by another name...
terrible idea...
but im sure it will popular with 60 year olds watermarking their pictures of sunsets.
Waiting for "Apple verified" photo of some important politician doing something wildly inappropriate.
Scrapped in 3..2..1..