• 2 Posts
  • 70 Comments
Joined 2 years ago
cake
Cake day: July 4th, 2023

help-circle



  • I went through and built a license, then read through it.

    I don’t think most of the things contained would be legally enforceable. We barely even have traditional open licensing that works, much less one that tries to enforce an ethical framework. Instead of this, we should work toward wide-reaching law that protects people’s rights, something that has teeth. Asking people to please not enslave someone with your library will never work, they will do it anyway or just not use your library, as they already do with copyleft licenses.


  • Arch on desktop/laptop because I’m very comfortable with it, and I can set it up the way I like.

    Debian on servers because it’s stable and nearly everything has a package available, or at least instructions for building.

    Same as OP, but I’m not likely to change them out. I’ve tried a lot of distros over the years and this is what works best for me.






  • I know you said consumer GPU, but I run a used Tesla P40. It has 24 GB of vram. The price has gone up since I got it a couple years ago, there might be better options in the same price category. Still, it’s going to be cheaper than a modern full fat consumer gpu, with a reasonable performance hit.

    My use case is text generation, chat kind of things. In most cases, the inference is more than fast enough, but it can get slow when swapping out large context lengths.

    Mostly I run quantized 8-20B models with the sweet spot being around 12. For specialized use cases outside of general language, you can run more compact models. The general output is quite good, and I would have never had thought it was possible 10 years ago.

    ETA: I paid about $200 USD for the P40 a couple years ago, plus the price for a fan and 3d printed shroud.


  • I would do FDE yeah. My current laptop setup is with systemd-boot and a special initramfs that allows me to unlock it with a yubikey, with fallback to password. Fair warning, this exact configuration is not particularly easy to setup.

    There are also modules which enable early network connectivity along with a SSH server, meaning you login and unlock it remotely. I have not tried this.

    Debian does not frequently require rebooting under normal circumstances. Kernel updates are not that frequent, and you can usually put it off for a bit if you don’t want to deal with it.







  • Yeah, you can turn off registration without a token. Then, if you want someone to register you can issue them a registration token, or manually create their account.

    Federation can be turned on, on a case by case basis.

    You can set rooms to invite only and not discoverable. Alternately, you can use an invite-only space that allows users to join rooms from there.

    The first two parts are done in the server config, see the synapse docs. The last is done once the server is setup and running as an admin.