Tag: proxmox

Posted on

Proxmox Unprivelliged LXC with shared Intel ARC GPU for Plex or Jellyfin transcoding

This is to create Plex and/or Jellyfin using docker within an Unprivilleged LXC container using Proxmox with access to the LXC’s host Intel ARC GPU.

I’m using Debian Trixie LXC for this example. This assumes you have:

Check the render device group on the host

ls -l /dev/dri/render*


Example output:

crw-rw---- 1 root render 226, 128 Nov 16 21:02 /dev/dri/renderD128
crw-rw---- 1 root render 226, 129 Nov 16 21:02 /dev/dri/renderD129


This is for the iGPU and the ARC GPU, the ARC GPU is renderD129. Both are part of the device GROUP 226 this will be needed later on.

Find the GID of the render group on the host – HOST_GID

getent group render


Example output:

render:x:993:


We also need the render group of the LXC – LXC_GID

getent group render


Example output:

render:x:105:



On the proxmox host, edit the config for the lxc

nano /etc/pve/lxc/<CTID>.conf


Add these lines in, remembering:
GROUP: 226
LXC_GID: 105
HOST_GID: 993
SUBUID_START: 100000 (constant)
BASE_RANGE: 65536 (constant)

lxc.cgroup2.devices.allow: c 226:0 rwm
lxc.cgroup2.devices.allow: c 226:128 rwm
lxc.mount.entry: /dev/dri/renderD128 dev/dri/renderD128 none bind,optional,create=file
lxc.cgroup2.devices.allow: c 226:129 rwm
lxc.mount.entry: /dev/dri/renderD129 dev/dri/renderD129 none bind,optional,create=file
lxc.idmap: u 0 100000 105
lxc.idmap: g 0 100000 105
lxc.idmap: u 105 993 1
lxc.idmap: g 105 993 1
lxc.idmap: u 106 100106 65430
lxc.idmap: g 106 100106 65430


This is to share both the iGPU and ARC GPU with 226 being the group.
The idmaps are rather complicated but you should be fine if you follow the rules 

lxc.idmap: u 0 SUBUID_START BASE_RANGE = lxc.idmap: u 0 100000 105
lxc.idmap: g 0 SUBUID_START BASE_RANGE = lxc.idmap: g 0 100000 105

lxc.idmap: u LXC_GID HOST_GID 1 = lxc.idmap: u 105 993 1
lxc.idmap: g LXC_GID HOST_GID 1 = lxc.idmap: g 105 993 1

lxc.idmap: u (LXC_GID+1) (SUBUID_START + LXC_GID + 1) (BASE_RANGE - (LXC_GID+1)) = lxc.idmap: u 106 100106 65430
lxc.idmap: g (LXC_GID+1) (SUBUID_START + LXC_GID + 1) (BASE_RANGE - (LXC_GID+1)) = lxc.idmap: g 106 100106 65430


Save and exit the config.

We need to add the HOST_GID to the proxmox host subuid and subgid files

nano /etc/subuid


This should be as follows, then save and close

root:100000:65536
root:993:1


The same for the subgid

nano /etc/subgid
root:100000:65536
root:993:1


Restart the LXC container so the new config is loaded. Nearly there..

Docker Compose

For both Plex and Jellyfin the ARC GPU needs to be passed through into the container, add these lines into the docker compose file 

    devices:
      - /dev/dri:/dev/dri

Plex

For Plex, go to Settings, Transcoder then scroll down to Hardware transcoding device your ARC should be there. Mine shows Intel DG2 [Arc A310]. If it’s not here, double check the idmaps in the LXC config from the proxmox host

Test the transcoding by playing any video, then in the Play Settings change the quality to something lower, then go to Activity, Dashboard and see if its transcoding with a hardware (hw) tag

NOTE: Hardware transcoding does require a subscription.
 

Jellyfin

For Jellyfin, top left 3 bars, Adminstration, Dashboard
Then Playback, Transcoding
Under Hardware Acceleration select Intel Quicksync (QSV)
With QSC Device enter /dev/dri/renderD129 for your ARC GPU
(or /dev/dri/renderD128 if you dont have iGPU)

To test play a video, go to the Play Settings, lower the quality, then click Playback Info
Look through the info for Play method Transcoding.
When Jellyfin transcoding is incorrectly setup videos wont play at all, check the idmaps in the LXC config from the proxmox host

Posted on

Tailscale split dns by domain for secure home server access

This is how you can still have public facing web apps but still maintain private web apps while you’re connected to Tailscale.

My setup:
Domain is richay.au
Adguard (10.10.10.10) as my home network DNS / ad blocker – All webapps have a DNS cname record for *.richay.au to rewrite to my Traefik IP (so internal connections stay internal)
Traefik v3 (10.10.10.20) as my reverse proxy – All webapps are published here under subdomains for richay.au
Cloudflare tunnels to get around CGNAT for public facing web apps – Only public facing apps such as my blog are listed in my tunnels, and the private IP set to Traefik (10.10.10.20)
Proxmox for my hypervisor for Linux Debian VM’s.
Tailscale to make the magic happen.

The first step is to install Tailscale (tailscale.com) on either the Traefik host machine or the DNS host machine (or both). If only installing once you can use advertise a route to the other VM within Tailscale, this is the option I am choosing and install on my Traefik host of 10.10.10.20 – then advertising a route to my Adguard DNS of 10.10.10.10 this is done using this command once Tailscale has been setup:

sudo tailscale set --advertise-routes=10.10.10.10/32


You will need to go to the Tailscale admin console and locate the Traefik entry then click the 3 circles to open the menu.
Click on “Edit Route Settings”.

Confirm 10.10.10.10/32 and accept the new route to activate, now Traefik can talk to your DNS server through Tailscale.

The second step is to allow Tailscale to use IP forwarding in Linux, these instructions can be found here.

After these have been configured you will need to return to the Tailscale admin console to configure the Tailscale DNS settings.
Under nameservers click Add nameserver.
In the pop up enter your home Adguard DNS 10.10.10.10, check the box for split DNS and enter your domain.


Hit save, and now you should have an active split dns while connected to tailscale.
Add your mobile device to tailscale and you will be able to access secure web apps from your phone wherever you are.

Posted on

Proxmox cluster with Traefik

With Traefik you can load balance between the different nodes of your cluster with one web page.

ie, https://proxmox.richay.au could access either one of the three nodes I have available, but it comes with one issue, you wont be able to use the shell as the websocket it uses needs to be upgraded in Traefik.

This is my dynamic config.yaml in traefik

http:
  routers:
    proxmox:
      entryPoints:
        - "https"
      rule: "Host(`proxmox.richay.au`)"
      middlewares:
        - https-redirectscheme
        - websocket-upgrade
      tls: {}
      service: proxmox
 
  services:
    proxmox:
      loadBalancer:
        sticky:
          cookie: {}
        servers:
          - url: "https://10.10.10.1:8006" # IP of node 1
          - url: "https://10.10.10.2:8006" # IP of node 2
          - url: "https://10.10.10.3:8006" # IP of node 3
        passHostHeader: true

  middlewares:
    https-redirectscheme:
      redirectScheme:
        scheme: https
        permanent: true
    websocket-upgrade:
      headers:
        CustomRequestHeaders:
          Upgrade: websocket
          Connection: Upgrade


This will allow the proxmox shells to work 🙂

Posted on

Ansible – updating proxmox host kernel with LXC shared GPU

This is to automate the updating of proxmox host when there is a kernel update which will break the LXC link to the GPU.

It just requires you to reinstall the graphics driver and do a reboot otherwise.

This is after you have done an update / upgrade of your proxmox host.
You will have to change the IP Addresses for your setup.

For me..
10.77.69.2 – Proxmox Host
10.77.69.103 – LXC Plex

########
- hosts: nvidia
  become: true
  become_user: root
  tasks:
    - name: Wait for 10.77.69.2 to become available
      wait_for_connection:
        delay: 5
        timeout: 300

    - name: Check if NVIDIA kernel module is loaded
      shell: lsmod | grep -q '^nvidia'
      register: nvidia_module_check
      ignore_errors: true

    - name: Set NVIDIA module check result as fact
      set_fact:
        nvidia_module_rc: "{{ nvidia_module_check.rc }}"

    - name: Reinstall NVIDIA driver if module is not loaded
      shell: sh /root/NVIDIA-Linux-x86_64-535.154.05.run --silent
      args:
        executable: /bin/bash
      when: nvidia_module_check.rc != 0

    - name: Set fact if NVIDIA driver was installed
      set_fact:
        driver_installed: true
      when: nvidia_module_check.rc != 0

    - name: Reboot system if NVIDIA driver was reinstalled
      reboot:
      when: nvidia_module_check.rc != 0

    - name: Wait for 10.77.69.2 to become available after reboot
      wait_for_connection:
        delay: 10
        timeout: 600
      when: nvidia_module_check.rc != 0

########
- hosts: plex
  become: true
  become_user: root
  tasks:
    - name: Install NVIDIA driver in LXC
      shell: sh /root/NVIDIA-Linux-x86_64-535.154.05.run --no-kernel-module --silent
      args:
        executable: /bin/bash
      when: hostvars['10.77.69.2'].driver_installed | default(false)

    - name: Reboot 10.77.69.103
      reboot:
      when: hostvars['10.77.69.2'].driver_installed | default(false)

    - name: Wait for 10.77.69.103 to become available
      wait_for_connection:
        delay: 10
        timeout: 300
      when: hostvars['10.77.69.2'].driver_installed | default(false)


This will check to see if the kernels for nvidia (my gpu) has been loaded, if not it will reinstall in silent mode. This will also flag a GPU install in ansible to also reinstall the GPU driver in the LXC, only if it needs though.