Looks like it works.

Edit still see some performance issues. Needs more troubleshooting

Update: Registrations re-opened We encountered a bug where people could not log in, see https://github.com/LemmyNet/lemmy/issues/3422#issuecomment-1616112264 . As a workaround we opened registrations.

Thanks

First of all, I would like to thank the Lemmy.world team and the 2 admins of other servers @[email protected] and @[email protected] for their help! We did some thorough troubleshooting to get this working!

The upgrade

The upgrade itself isn’t too hard. Create a backup, and then change the image names in the docker-compose.yml and restart.

But, like the first 2 tries, after a few minutes the site started getting slow until it stopped responding. Then the troubleshooting started.

The solutions

What I had noticed previously, is that the lemmy container could reach around 1500% CPU usage, above that the site got slow. Which is weird, because the server has 64 threads, so 6400% should be the max. So we tried what @[email protected] had suggested before: we created extra lemmy containers to spread the load. (And extra lemmy-ui containers). And used nginx to load balance between them.

Et voilà. That seems to work.

Also, as suggested by him, we start the lemmy containers with the scheduler disabled, and have 1 extra lemmy running with the scheduler enabled, unused for other stuff.

There will be room for improvement, and probably new bugs, but we’re very happy lemmy.world is now at 0.18.1-rc. This fixes a lot of bugs.

  • Dong64@lemmy.worldEnglish
    9·
    2 years ago

    Having some issues with logging in, but wanted to detail what worked for me:

    1. Try clearing cache (this worked for some users, but not me)

    2. Try different web browser

    3. Worked for me: reset password. Interestingly enough, the password reset link sent me to a version of the page in French, but this method worked both times I was unable to log in today.

    Also looks like wefwef is still not working after this troubleshooting fix.

    • Saundsr@lemmy.worldEnglish
      3·
      2 years ago

      Thanks

      • Clearing cache didn’t work for me.
      • Different web browser didn’t work for me
      • Resetting did work for Lemmy on desktop

      but can’t log into any app at this point.

      We’ll see what happens in a bit.

      Edit: slowly opening up I think.

      Edit 2: I can now get into all of the big 3 (Connect, Wefwef, Jerboa).

      • Dong64@lemmy.worldEnglish
        1·
        2 years ago

        Out of curiosity, was the reset password link in French? Also, which apps are you using? I finally was able to get into wefwef, and hadn’t really tried another app up to this point.

        • Saundsr@lemmy.worldEnglish
          1·
          2 years ago

          Password reset was not in french. (I’m in US and reset email and the actual page was in English).

          I’m using the Big Three. (Connect = Working, Wefwef = Working, Jerboa = not working)