Scroll down to see the original post Alright, I’ll consider the maintenance complete. Here are the updates:
-
Lemmy backend has been upgraded to version 0.17.4 which is the latest available. This added some database optimizations, applied some security patches and potentially fixed some bugs although the lemmy devs have very vague release notes for this version and mention that more information will be given in the next version.
-
I have also hooked up the server’s backups to cloud storage. This should have no visible impact but it gives us an extra redundancy layer in case a backup needs to be used. Now that the instance is growing this is a must.
-
Finally there’s one issue which I wanted to do today but had to postpone: media storage.
Yesterday the instance crashed because we filled up the virtual machine’s hard drive completely. This should not have happened so early. For some reason we’ve been caching a lot of media of external instances. This could be a bug in the way lemmy handles media. Unfortunately this is pretty much “beta software” and we have to roll with the punches.
My intention was to hook up the server to object storage for media files where we’d have plenty of space, but there’s currently two ways of doing it: a good way and and a proper way. The proper way requires upgrading the pict-rs component of lemmy to the latest version… but a) it’s not a final version, but a release candidate. b) I have doubts on how this will impact the lemmy upgrade process since it is pretty much automated and I’m worried there might be some sort of conflict or issue for manually upgrading one of the components.
For now it might be best to wait a little bit more until I have more information. I doubled the hard drive and it should last us for some time. Rest assured that this has nothing to do with the images that we upload in our communities, but with caching remote media which should not be cached.
Anyways, this concludes today’s maintenance. I do have some pending updates about moderation, personal communities, and plans for the future, but it’ll have to wait until tomorrow. I will give you a quick heads up in bullet point format:
- I’m at the point where I’ll open up applications for moderators. Stay tuned.
- Please don’t use NSFW content for community icons or banners. No one has done so, but I remembered that I need to put this in the rules. This is because they appear as profile icons and profile banner on other clients such as Mastodon and are visible to everyone.
- When I said that I want Yiffit to become a community space, I meant it. !malekart@yiffit.net is the first “personal community” and this is perfectly fine. I’ll soon give more details about how this fits into the larger vision as well as the challenges this presents. I want to give you, the users, the tools to carve out your own space.
Most importantly
, please tag NSFW very liberally. We do this as a courtesy to remote users from different parts of the world and cultures. Since everyone on yiffit has access to this content and is over 18, it’s not meant to be a filter to keep minors away from sensitive images, but more as a respectful gesture for other users who may not share our values. I’ll share more info about it tomorrow. If you have any concerns, please send me a DM.
Please note this is not a restriction since every yiffit user or remote user interested in yiffit should already have an account capable of viewing NSFW tagged content. This is just a measure to get highly sensitive remote users off our backs :P.
There’s a point to be made about us having the moral high ground, but if you’ll post some sort of cropped yiff, pin-up, sensual, etc… image, even if it should be safe, it’s probably best to tag it as NSFW. And don’t worry too much if you forget.
Example:
spoiler: potentially nsfw. These are tagged "safe" on e6 but it'd be best if we tag them as nsfw
But as I said, more info about all this tomorrow!
PS: I have been reading all your comments and I am humbled by the amount of support I’ve been receiving. Actually they were popping up on screen as notifications while doing maintenance work. It felt like one of those anime scenes where you’ve got friends and family shouting messages of hope and determination ///. Thank you everyone!
Original post First of all we’ll be upgrading to the new version of lemmy (v.17.4) which does not have major changes but some database optimizations and might change some bugs.
As far as I know its the next one which will have much expected bug fixes, but we might be surprised with some improvements in this one as well. In any case this update needs to be done to prepare for the next.
Furthermore I will also do some server maintenance in ensuring that the instance is more robust and capable of handling our growth. I’ll be able to give more details about our current setup and technical details once the upgrade concluded.
I hope to keep the downtime to a minimum. It could be less than 5 minutes for each, but could also be around 30min in a bad-case scenario. (Worst case if is something goes wrong and I need to restore a backup, but that is unlikely to happen).
Than you very much Wander-sensei for all this. ヾ(・ω・)メ(・ω・)ノ
Good thing to know about planned upgrades for both Lemmy software and for the server upgrades. ( ̄ω ̄;)
And take as much time as needed to ensure everything is right. (^ω~)
Thank you so much for being here ^^ I couldn’t do this all without you, regardless of how many servers I had.
appreciate you wander 💙 this is so nice to let us know and give us details. it’s a lot to manage and i appreciate all your efforts for the benefit of the community. you the real mvp 💙
Thank you for all your support <3
The more time you and we all here invest into this lemmy instance, the more growth our community will get. Thank you, Wander, for keeping it all going. :)
Thank you so much for all the work and for being such a good admin
Thank you for creating this lovely instance and putting all this work into it :3
Woo! Upgrades!
Thanks for workin on your weekend to keep us up and running :D
thank you so much for the transparency and the work you’re putting into this instance!!
object storage can be expensive D: is there any way we can contribute to maintenence of our instance?
Not right now. Once things are stable and running smoothly, I’ll consider taking donations. Right now I feel like I need to earn it first :P
Fortunately my object storage provider is quite affordable and doesn’t break my bank. Regarding reliability, I’ve been using them for 6 months on my Mastodon instance and can’t complain at all.
okay :3 please keep me updated on this 💙 may i ask what block storage you use, please? i’ve been trying to decide on one to go with my homelab backup flow. ty 💙
I’m running things on a dedicated server with proxmox as hypervisor on OVH. Hard drives are two SSDs on a RAID-1 setup.
oh, i meant for your object storage :3
Here you go! https://www.idrive.com/object-storage-e2/
ty o: how is it so cheap? how’s your experience with it been?
It’s been flawless so far. That said, I haven’t needed that much reliability, although it’s worth mentioning that I haven’t experienced any problem or downtime either. I’ve been using it at packmates.org for over 6 months now and it caches thousands of images a day.
Biggest problem I’ve had was documentation which was a bit sparse six months ago, although it seems to be better now.
The most important thing is that it has no egress fees.
weird on the storage thing, I kinda figured that it would just pull from the original communities instance, though i guess some level of caching would keep a popular instance from melting down.
wonder if maybe it does but lemmy isnt clearing out its cache properly
(I havent read how it works, just guessing from the peanut gallery)
In the Lemmy admins chat they were saying the same thing and also had no idea how it cached so much.
That said, it shouldn’t be a problem. I’ve doubled the disk of the VM and in the next days will have media files upload to object storage just as I do with Mastodon. I just need to figure out whether I should wait for pict-rs release or mount the storage to the files directory using s3fs which is what I’ve seen recommended in some Lemmy discussiond from about a year ago.