This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Well, there are some caveats there - if they are actually using homomorphic encryption to run the classifier, that means that Apple's servers do not at any point learn what the landmark is. If the goal is to report it to the FBI if a picture being sent has been labelled "child porn", accordingly, the phone would have to be wired up to report/send the image if the data it received, once decrypted, indicates that it was classified as such. How do you stop people from blocking this reporting functionality on their end? Adding additional user-unmuteable snitching logic to end-user devices comes with all sorts of legal, technological and security risks (and quickly puts you in a league with North Korean computing equipment that comes with daemons watermarking every document you touch, which they make it illegal to disable).
That being said, I see your argument at least insofar as the case for "it can't be done" is overstated and oversold, but I am not enough of an idealist to agree with this "just be truthful to the ruling classes and try to defend what you want on principle, the truth always wins in the end" thinking. I'm pessimistic about the prospect of a principled stand - we'll get the mandatory surveillance rectangle reporting on wrongthink eventually, because the powers-that-be really want it, and the majority of our fellow citizens probably already want it as well, or else the ruling classes will have all the opportunity on their side to manufacture the conditions that will make them want to, be it by propaganda, dissolving the cohesion of their opposition (note how effectively they split the tech anarchist scene into those who still want to keep the government out and those who think that the Nazis who want to keep the government out are the real danger) or creating real problems to which they are the solution (people want less government spying -> import scary foreigners into what to them is a scary foreign land -> old natives want more government spying to keep them safe from scary newcomers, newcomers want more government spying to keep them safe from racist natives). As far as I am concerned, the better choice at this point is just to lie and obstruct all the way. This buys time for some technical or societal deus ex machina solution to emerge, or else at least lets us spend a bigger fraction of our remaining time on this mortal coil out of bondage.
Are they currently claiming they do this today? My understanding of homomorphic encryption (admittedly a bit outside my wheelhouse) is that it's nowhere near as well-trod a space as, say, RSA. When I last looked, it was possible -- with a bunch of caveats -- to do simple things with a whole bunch of overhead, and certainly no equivalent of a NIST standard (if you trust those: say "Dual EC" with me) for it.
I didn't think the technology to do this well was ready for prime time today, but maybe I'm just a bit out of date. Do they have a white paper, or, better, a bunch of academic papers?
My general take is that the main use of homomorphic encryption is to make cryptocurrencies look respectable by comparison.
It's main purpose seems to be to get people to move their data into the cloud, while pretending that you can do useful computations on the data without learning anything useful about it, because it is all encrypted.
More options
Context Copy link
Yup. They even open sourced a library. They're citing the BFV papers from 2012.
Thanks. I suppose publishing a white paper at least opens them up to more serious scrutiny (which I've seen in serious-ish forums like HN). But my initial response is rather skeptical still, though. The encryption methods seem like they should be far more expensive than Apple is letting on [1], and they say they're using this for querying remote databases with encrypted queries. They're less clear on how these databases (for photographic landmarks, URLs, and such) are encrypted in a manner that actually hides the query from Apple. Are the encryption keys different per-device? If so, how do they avoid needing a separate database per device? And if not, it seems there's a lot of trust that they would be unable to figure out which rows matched.
I know Google's approach to similar issues has been focused more on device side ML models. Pixel phones support offline song recognition (I've noticed it's fairly limited to popular songs), and Google Translate can work (in a limited fashion) offline. Why does Apple need to do cloud-based POI recognition in photos? The whitepaper only shows 6 very well known landmarks, but it seems like it'd be easy enough (and secure!) to do this on-device. Given the known computational costs of FHE, it might even be better for battery life.
Oh, I would read it differently. I think they're letting on that it's pretty expensive, which is why they're doing all the mess with sharding and DP.
My understanding of BFV is that when the device does its keygen (unique for each query, I assume), it produces and then passes an "evaluation key" as part of the public key. One has to design the scheme so that there is sufficient expressivity in the set of evaluation functions that can take in the evaluation key and perform the desired operations on the encrypted message. I don't believe this involves completely encrypting the entire database from scratch using the public key every time; it just requires running the same operations on the same underlying database, but with the evaluation key in the operations. They call out that it's important that the BFV scheme has many of the operations they want, expressible in, shall we say "evaluation key parameterized form".
I think a lot of the work in library building is essentially building up a set of these "evaluation key parameterized" operations. You have to start with extremely simple operations and then build your way up to more usable tools that are composed of those simple operations.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
If they were designing a system to enable this, they might not include all of the features they currently have. It would probably be easier to just make one that does it than to start from what they have and add it in. For example, they wouldn't go to all the trouble to try to hide the correspondence between a device ID and the encrypted bag of photos, either (or they'd have to similarly specially puncture it).
Also, most of the concerns about "can't people just turn it off" forget how strong Apple's control is over their walled garden. Ultimately, this usually descends into arguments over Apple's update process, which is where they are the strongest. I'm mentally skipping some steps, but I could imagine designing a system where, when the device requests the latest update, their request must come with certain proofs, and one could imagine building in that those proofs show that they're doing things that Apple wants (it can check by, say, leaving a blob on their server, one that still doesn't reveal the contents of the photos, but can be used to verify that they're being truthful about them), and denying the update if they can't prove that they're doing what they're told. (Probably still easier to just do it the first time, though.)
That discussion usually descends into questions of society, not tech. Whether people will still use Apple devices. Whether people will eschew updating their devices with Apple software and try to go their own way, etc. All of those discussions mostly amount to, "Yes, you can leave Apple's walled garden and go your own way," but that's true of any piece of electronics if you try hard enough. For any devices that stay in Apple's walled garden, Apple is nearly omnipotent (absent possibly nation state level efforts).
Fair enough on the strategy piece. One clarification, though. I did not say:
Often times, the truth doesn't win. Many of those times, it's because the people who could have the credibility to fight have publicly burned it until the truth doesn't matter anymore.
More options
Context Copy link
More options
Context Copy link