Ahh, well I have found the Jellyfin clients to be less than perfect on appleTV. You may want to try another endpoint client if you don’t want to try another server side software.

Since you are against using plex even with using it only locally I would suggest you look into the Jellyfin clients support. Specifically look at each client device documentation and take a look at what audio and video codex they natively support. If there is one they both support then I suggest you do some testing and stream a movie or show that is encoded with that specific format. This will let you know if it will work across both clients whether it’s a stream issue hardware issue or Wi-Fi issue.

This will prevent transcoding that can cause audio sync issues.

Edit - if that works then I suggest you convert you library to that format.