Warning: The default setup WILL remove admin priveliges. Best to create a second admin account beforehand. A work around fix is found at the bottom of this article.
Follow the instructions to integrate authentik with Jellyfin here.
Step 9 for role claims needs some extra info if you want to log into Jellyfin with admin access, namely the Admin Roles.

I’ve gone with the default group for Authentik of “authentik Admins”.
You can use a custom group or create one in Authentik by going to:
Admin Interface -> Directory -> Groups -> Create
Name it “jellyfin” and leave all as default. Go to users at the top and add the correct users you want admin access to jellyfin.
Enter “jellyfin” in the Admin Roles in the Jellyfin OIDC plugin.
Enjoy 🙂
Fixing broken admin with the OIDC Admin Roles
Credit to Spyros from the comments
If you have set this up with the official instrcutions and need to still add the ‘Admin Roles’ for authentik, navigate to your data at /jellyfin/data/plugins/configurations/SSO-Auth.xml
Edit the lines at <AdminRoles> to the following
<AdminRoles>
<string>authentik Admins</string>
</AdminRoles>
Or to whatever your custom group is named, then save and restart 🙂

thank you a lot for your instructions.
In case you already did the booboo and you are locked out (if you haven’t created a second user), find the jellyfin file `plugins/configurations/SSO-Auth.xml` and edit it. in Adminroles XML line, replace it with
authentik Admins
Save, exit edit, and restart jellyfin.
refresh page and you should have back the admin rights.
thats awesome, thanks for this.
I’ll add this in with credit when work slows down a little 🙂