Here is what I did on Arch Linux with nitropy and base32 installed in case anyone else needed a bit of extra help figuring this out.

Install and enable the smartcard daemon (PCSCD) using your package manager/init system. On Arch, that would be:

sudo pacman -S ccid opensc

sudo systemctl start pcscd.socket

sudo systemctl enable pcscd.socket

Confirm it is running with:

systemctl status pcscd.socket

Generate a 20-byte random string with:

dd if=/dev/urandom of=/tmp/nk bs=20 count=1

Encode this and add to your nitrokey in a slot (slot 2 here):

nitropy nk3 secrets add-challenge-response 2 $(base32 /tmp/nk)

Repeat this last step on your backup nitrokey if you have one. Copy your KeePass database so you have a backup. Add the challenge-response authentication to KeePassXC under database -> database security -> challenge-response Test opening it with both nitrokeys. Be sure to add "Hardware Key" on the unlock screen the first time (I had to open KeePassXC after plugging it in then click refresh to get it to appear the first time). If it does not appear, there may be an issue with pcscd not running.

This also worked on NixOS after adding services.pcscd.enable = true;

If you are using the KeePassXC FlatPak, make sure to enable socket=pcsc (e.g. with FlatSeal).

I don't think the feature is implemented on KeePassDX yet for anyone using that on mobile.