felsiq

To be clear, I wasn’t saying you’re wrong. I just like homomorphic encryption a lot and love a chance to tell people about it lmao

There are no LLMs that process encrypted tokens.

Check out homomorphic encryption! AFAIK it’s not used in any LLMs just yet but the plans are in place and it’s tantalizingly close

Did you use a heavily quantized version? Those models are much smaller than the state of the art ones to begin with, and if you chop their weights from float16 to float2 or something it reduces their capabilities a lot more

Yep, the OpenAI api and/or the ollama one work for this no problem in most projects. You just give it the address and port you want to connect to, and that port can be localhost, lan, another server on another network, whatever.

Oof rip sorry lol, thought I’d seen a post about boost and piefed

I’ve seen a couple of other clients add support for piefed recently as well, although I don’t follow them closely cuz I’m happy with voyager. Maybe boost or mlem supports this with piefed already?

It’s come a long way recently, I’m currently using piefed on voyager rn and it works great

It works in the sense that the operations are performed on binary numbers, so text handling works the same way it normally does assuming the handler function is encrypted to match. Once you have multiplication and addition, you can make logic gates and general computing follows from there - although with the noise being amplified thru each logic gate, the more complex the functions the more bootstrapping is required and the less I see this being doable in the short term.

For a working example, check out apple’s homomorphic encryption page, they use it for landmark identification and afaik will be using it for siri whenever they get to that update. It’s slow but it’s already usable - I’m not personally convinced it’ll be used everywhere, but the technology is super cool and I hope it shows up more

Called the blobfish

This seems like the perfect use case for federated software to me