My point is that she is appropriately characterizing what she's being asked to do.
> Real harm is happening
... and these proposals would be very unlikely to materially reduce that harm. They WOULD, however, create a bunch of other harm. Including harm to children. In fact even including sexual harm to children.
> boogeymen
You said it, I didn't.
> Technical people are supposed to find optimal technical solutions, not pretend to be technocrats or underdogs fighting against governments who are all unjust and out there to get us.
Nobody's job disqualifies them from having political opinions. Technical expertise MAY, on the other hand, give one a better than average understanding of the actual impact of making a technology change. As may actually running an affected system and knowing how things play out on it, for that matter.
> and these proposals would be very unlikely to materially reduce that harm. They WOULD, however, create a bunch of other harm. Including harm to children. In fact even including sexual harm to children
I would want to hear your explanation for that but more importantly, you should convince lawmakers about this instead of claiming the tech is impossible.
> You said it, I didn't.
Yes I did, hope you got my point then.
> Nobody's job disqualifies them from having political opinions. Technical expertise MAY, on the other hand, give one a better than average understanding of the actual impact of making a technology change. As may actually running an affected system and knowing how things play out on it, for that matter.
I fully support tech people opoposing this law, so long as they don't claim apps can't scan messages before transmission which is obviously a lie. Of course the apps security is reduced and they can have opinions by how much, but so long as the security property and risks are communicated to policy makers and users honestly I have no problem with it. What I would have a problem with is if Signal as a corporation chooses to remain in the UK market and refuse to comply with UK law.
But if you read my comment, apple and google should do this, not Signal, but then again them or telcos pushing code that modifies appls like Signal would be much more insecure than Signal itself doing it nice and neat.
> I would want to hear your explanation for that but more importantly,
The people who want it need to prove that it will help.
I had started in to list paragraphs and paragraphs of failure modes, unintended consequences, and directly bad effects, but then I realized that that implicitly accepted an inappropriate burden of proof.
I am tired of hearing "Something must be done. This is something. Therefore this must be done".
Tell me, in detail, how any of these proposals will help. Include all of the steps from the initial scanning through the final removal of harm. Do not stop at "Authority(TM) will know about X, so they can fix it". Tell me how they will fix it, and how they will do that without doing harm. Not how they theoretically could. How they actually will. And make it plausible.
After you've given me that, I'll be happy to rip it to shreds for you.
Otherwise, if you can't figure out for yourself how it's harmful to give platforms massive incentives to shut down anybody who raises costs by triggering too many false positives, how it's equally harmful to give them massive incentives to broaden their definition of what's a "true" positive, how it's harmful to have thousands of random people viewing false "hits", how it's damaging to enable Authority(TM) to come down on kids for communicating among themselves, how it's dangerous to create a massive database of blackmail material on potentially vulnerable people, how it's trivial to repurpose a spying infrastructure for different targets, or how it's useless to give more reports to law enforcement that's overwhelmed with the reports it already has, then you'll have to content yourself with reading the vast volume of stuff other people have written up about the OSB and all the related proposals.
> you should convince lawmakers about this
Many of us have been trying. The OSB has not in fact passed yet. The EU and US proposals still less.
> instead of claiming the tech is impossible.
Nobody, not me, not Meredith Whittaker, not anybody, has claimed that client side scanning is impossible, or that you can't turn around and encrypt a copy of a message after you've spied on it. That is a straw man. It is entirely your fantasy.
What they've said is that it's stupid and contrary to the whole point of having the encryption in the first place. It leaks information that's supposed to be private, including false positives, and paints a target on the most sensitive stuff. It creates infrastructure for abuse. Those are the "security properties" being "communicated". And that is 100 percent true.
When she says "back doors", she's talking about system back doors, not necessarily protocol back doors. Although the protocol would have to have some way to report the data.
Anybody with a remotely clear-headed view of the situation understands that sending off plaintext copies of the stuff you then turn around and encrypt is in tension with the purpose of doing the encryption. They also understand that once you have either the scanning or the reporting system in place, there's nothing that controls how it can be used.
And they understand that the proponents of this stuff have no detailed story at all about how the leaked plaintext is supposed to be secured after the leaking is done... especially if it's actually going to be used for the stated purposes. That is a VAST technical and organizational problem that's constantly handwaved.
> What I would have a problem with is if Signal as a corporation chooses to remain in the UK market and refuse to comply with UK law.
That's another fantasy of yours. She's been very clear that if the UK does this, Signal will exit the UK market.
However, that's her and her corporation. It's not binding on the rest of us.
The fact is that "lawmakers" do not have infinite legitimate power. It is sometimes right and maybe even morally required to directly break the law.
Personally, I'm looking forward to the widespread use of P2P tools that will make Signal look like an open book... whether they're legal or not. At this point, laws even in the "free world" are descending into insanity.
> After you've given me that, I'll be happy to rip it to shreds for you.
Rip this to shreds: if at least one victim of a crime can be prevented by catching a witless criminal then it is worth it. Also this: Homomorphic encryption and differential privacy are a thing, if certificate transparency logs can prove no certs are issued maliciously then similar logs can be published publicly to prove that the scanning list (similar to a CRL) does not contain hashes/patterns that haven't gone through appropriate legal/regulatory checks and balances. E2EE can also be used to secure the message between the gov agency and the device. Certificate authorities already have similar exposure that can compromise all your traffic (including the signal app download/install), "shred" this and show me why a similar risk model is inapplicable to gov scan lists.
> Nobody, not me, not Meredith Whittaker, not anybody, has claimed that client side scanning is impossible, or that you can't turn around and encrypt a copy of a message after you've spied on it. That is a straw man. It is entirely your fantasy.
> What they've said is that it's stupid and contrary to the whole point of having the encryption in the first place. It leaks information that's supposed to be private, including false positives, and paints a target on the most sensitive stuff. It creates infrastructure for abuse. Those are the "security properties" being "communicated". And that is 100 percent true.
No, the message you have been spreadning is that it is impossible to secure a message and spy in it which you just admitted there. That is false, that is not a strawman on my end, you just admitted it! The infrastructure can be abused? So can signal's source code repo infrastructure. Your lie is that of omission and context framing while fully knowing how non-technical policymakers will interpret what you are saying. It is absolutley possible to implement a scanning infrastructure that has the same security properties as the app's code or app store download/signature security. If you are claiming the gov end can be abused by malicious humans then that is beyond your exertise to police humans breaking laws, so long as transparency logs can be produced to criminally punish violators.
> Anybody with a remotely clear-headed view of the situation understands that sending off plaintext copies of the stuff you then turn around and encrypt is in tension with the purpose of doing the encryption.
That is not what was suggeste and I am sure you are aware that it is possible to scan for messages without sending off a copy of the message off device and also without informing anyone of false positive hits. Or even contents of true positive hits (requiring them to get a proper warrant and target the device for intrusive collection).
> That's another fantasy of yours. She's been very clear that if the UK does this, Signal will exit the UK market.
It's not a fantasy, it is possibility and I am glad signal will exit the UK if this happens. I support this kind of boycotting.
> It is sometimes right and maybe even morally required to directly break the law.
Maybe, but not everytime you don't like a law, that undermimes the rule of law. Only active and imminent harm to humans is a reason for civil disobedience not mere speculation and disagreement. You spoke of burden of proof, the other side has proof of harm to humans you are on the opposite side of civil disobedience here. You are on the side of the oblivious technocrats profiting from harming people.
> Personally, I'm looking forward to the widespread use of P2P tools that will make Signal look like an open book... whether they're legal or not. At this point, laws even in the "free world" are descending into insanity.
Me too, but for opposite reasons: so that you people finally get that technocrats and civil disobedience is not a thing. You cannot solve politics with technical rebellion. You have to actually convince your peers. Here I am as your peer engaging with you while being aware of most of the technical facts and you are not convincing me. Your strategy of using technical expertise to deceive people will backfire too and so will downvoting people like me attempting to engage in civil discourse with you in good faith. I think you will just end up facilitating the building of "GFW" level national firewalls in the west with your techniques. They are not going to stop trying to find a solution to reduce CSAM and other illicit crimes, a privacy preserving solution is possible and fighting it is a lose-lose scenario.
> Rip this to shreds: if at least one victim of a crime can be prevented by catching a witless criminal then it is worth it.
You didn't come close to what I asked for. You gave no details on how even one crime would be prevented. But I'll give you that it probably would prevent a few.
It's still crazy to say that outweighs any other consideration.
You have to prevent more evil than you cause. That includes the opportunity cost of the resources you put into it, by the way.
"If it saves just one child" is a childish argument and I'm not going to engage with it any further.
... and if you think no children will be sexually blackmailed using the reports you're trying to generate, you are insane.
> Also this: Homomorphic encryption and differential privacy are a thing,
You seem to have slipped from the idea of client-side scanning into using homomorphic encryption to run ML in zero knowledge on the server side. That won't work.
There's a sharp limit on how many operations you can do on the encrypted data before you lose the ability to recover the result. That limit is NOWHERE NEAR the number you need for any useful ML. Plus of course the incredible compute, communication and storage load.
I have no IDEA how you'd expect to use differential privacy for any of this. And I very much doubt that you do either.
So let's stick to client-side scanning, since we at least know how to build that. Anyway, the fatal problems are with scanning in general, not with any particular way of going about it.
> if certificate transparency logs can prove no certs are issued maliciously then similar logs can be published publicly to prove that the scanning list (similar to a CRL) does not contain hashes/patterns that haven't gone through appropriate legal/regulatory checks and balances.
I think you may now have moved again, to perceptual hashes. But I guess maybe you could do something like that for an ML model too.
The "checks and balances" I've seen so far have been not so much crappy as nonexistent. Including having private groups create the lists with no real oversight.
Once the infrastructure was in place, I'd expect some countries to PUBLICLY AND OPENLY expand the scope, so auditing is irrelevant anyway. In fact, the OSB already covers more than CSA and would probably require scanning for more than CSA.
> E2EE can also be used to secure the message between the gov agency and the device.
These proposals, especially the OSB, generally call for platforms to manually vet the "hits" before they go to any government agencies, or even to the private advocates.
They kind of have to, since--
1. Most of the government agencies are too swamped to actually follow up on most of the reports they already get, and
2. The bills all demand going beyond looking for specific, already known and vetted files, into looking for things that "look suspicious". Once you go there, the number of false positives will be more than the number of true positives. Especially when you make them terrified to have any false negatives.
But sure, you can encrypt the stuff at each in-motion hop. Which has nothing to do with the main exposures.
> Certificate authorities already have similar exposure that can compromise all your traffic (including the signal app download/install),
That's one reason high-security applications don't trust CAs. Signal doesn't, for instance, but Signal's not special in that way.
> No, the message you have been spreadning is that it is impossible to secure a message and spy in it which you just admitted there.
Well, yes, it is. At least to any reasonable standard.
> So can signal's source code repo infrastructure.
One risk doesn't justify taking on another risk. Especially not a much greater risk.
> It is absolutley possible to implement a scanning infrastructure that has the same security properties as the app's code or app store download/signature security.
OK, this is truly insane.
One system tries to keep messages tightly compartmented, exposing them only to their senders and recipients. It keeps the messages encrypted except on the senders' and recipients' devices, and actually being read.
Another system does the same, except that it also sends some of those messages into a central database. It does so for the purpose of having them read by third parties, and by this I mean humans. In practice, that database has to have a long retention time and a huge number of authorized users. If those users decide that the messages were true positives, they get forward into yet another database.
The second system has every exposure the first system has, plus a bunch of other, worse exposures. It vastly expands the spatial and temporal areas where sensitive data are kept. It puts all users and devices' data into the same compartment. It has probably more than twice the total code of the first system. It trusts thousands more people. And the database is not only huge, but rich in abusable material, so it's a gigantic target that will attract attackers.
Those two systems do not have "the same security properties". They don't have anything close to "the same security properties". Using the phrase "the same security properties" anywhere near those two systems shows that you know nothing about what you're raving about.
And, yes, that second system is what you will get.
> If you are claiming the gov end can be abused by malicious humans then that is beyond your exertise to police humans breaking laws, so long as transparency logs can be produced to criminally punish violators.
Not only am I entitled to my opinions as a member of the (world) polity, but that as an actually competent security specialist, I do have real expertise in designing security systems around how humans actually act.
Unlike you, obviously.
> I am sure you are aware that it is possible to scan for messages without sending off a copy of the message off device and also without informing anyone of false positive hits.
It's a false positive because you don't know it's not a true positive. You therefore have to treat it exactly the same as a true positive.
> Or even contents of true positive hits (requiring them to get a proper warrant and target the device for intrusive collection).
You won't meet a probable cause standard with those hits. I'm sure you could get a warrant in Iran.
Nor do most law enforcement agencies have the resources to get warrants and raid people "on spec" like that.
... but in fact nobody's going to try to build that. The "workflow" you will actually get with this stuff is
1. Device gets a hit.
2. Device sends the data the hit was based on to the platform
3. Optional, but likely to be common because it limits risk for the platform: automation disables the user's account until the platform gets around to reviewing the hit. This may take weeks, especially if some score is borderline.
4. Platform employee reviews the hit (with little context, which matters a lot especially for the text scanning people are demanding)
5. If the hit looks criminal, platform employee forwards it to law enforcement or whoever. With all the data unless legally prevented.
6. If the hit looks borderline, or the user looks like they "might be a risk", or maybe even like they "might generate a bunch more false positives we have to review", platform employee ends the business relationship with the user.
7. If the hit looks completely false, platform employee reenables the user's account.
On hits with very low scores, you might have the device just refuse to send the message or whatever. In that case, either the innocent user is screwed, or the guilty user tries other ways until one works.
> Only active and imminent harm to humans is a reason for civil disobedience not mere speculation and disagreement.
We're seeing people disappeared constantly in a bunch of countries (not the UK so far). I'm OK with calling that active harm.
We're seeing stuff like teenagers jailed for sexting in the US... which this nonsense would definitely greatly increase. And we're people worldwide driven off of platforms. They "look too much like" abusive users to a computer, you see... especially to a computer programmed by some clueless whitebread idiot.
> You are on the side of the oblivious technocrats profiting from harming people.
That stupid bullshit again. News flash: CSA is not profitable for platforms. They don't get paid for it, and if it's visible it drives away profitable users. At MOST it's profit-neutral, usually negative.
> Here I am as your peer engaging with you while being aware of most of the technical facts and you are not convincing me.
That's because you're an obvious fanatic.
> Your strategy of using technical expertise to deceive people
Nobody's tried to deceived anybody. Except maybe you.
> downvoting people like me attempting to engage in civil discourse with you in good faith.
I haven't downvoted you. I have wasted my time engaging with you. I'm not going to waste any more, though.
> I think you will just end up facilitating the building of "GFW" level national firewalls in the west with your techniques.
> Real harm is happening
... and these proposals would be very unlikely to materially reduce that harm. They WOULD, however, create a bunch of other harm. Including harm to children. In fact even including sexual harm to children.
> boogeymen
You said it, I didn't.
> Technical people are supposed to find optimal technical solutions, not pretend to be technocrats or underdogs fighting against governments who are all unjust and out there to get us.
Nobody's job disqualifies them from having political opinions. Technical expertise MAY, on the other hand, give one a better than average understanding of the actual impact of making a technology change. As may actually running an affected system and knowing how things play out on it, for that matter.