FFMPEG HEVC NVENC (yuv420)p10le at 800Mbps not encoding

Question

I'm working on a large immersive project where the clients have requested H.265 media at 800Mbps (requires level 6.2). Due to the hardware used (I believe it's AvoLites based) for playback media compatibility is important.

Test GPU:

NVIDIA GeForce RTX 3060 
Driver: 550.76
CUDA: 12.4

FYI - Adobe Media Encoder can produce compatible media (GPU, main10, level 6.2, 800Mbps) but due to the amount and length of the work we are looking to use FFMPEG to automate and take advantage of our GPU render farm.

Example compatible Adobe Media Encoder media:

ffprobe ame.mp4 
ffprobe version 7.0 Copyright (c) 2007-2024 the FFmpeg developers
  built with gcc 9.3.0 (GCC)
  configuration: --prefix=/images/tmp/jenkins/install/forge/install/ffmpeg/ffmpeg.org/ffmpeg/linux_64/7.0 --cc=//packages/vendor/industrial/kiln.platform/5.1/CentOS_Linux_7.9/x86_64/gcc/9.3.0/bin/gcc --cxx=//packages/vendor/industrial/kiln.platform/5.1/CentOS_Linux_7.9/x86_64/gcc/9.3.0/bin/g++ --extra-cxxflags='-std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0' --enable-shared --enable-avfilter --enable-postproc --enable-pthreads --enable-cuda --enable-gpl --enable-libaom --enable-libfdk_aac --enable-libmp3lame --enable-libvpx --enable-libfreetype --enable-libx264 --enable-libx265 --enable-libopus --enable-cuvid --enable-nvdec --enable-nvenc --enable-nonfree --enable-ffplay --enable-libpulse --extra-libs=-lpthread --extra-libs=-ldl --extra-libs=-lm --extra-libs=-lstdc++
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'ame.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41
    creation_time   : 2024-05-01T17:02:05.000000Z
  Duration: 00:01:23.40, start: 0.000000, bitrate: 798977 kb/s
  Stream #0:0[0x1](eng): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt709), 3840x2160 [SAR 1:1 DAR 16:9], 798654 kb/s, 25 fps, 25 tbr, 25k tbn (default)
      Metadata:
        creation_time   : 2024-05-01T17:02:05.000000Z
        handler_name    : ?Mainconcept Video Media Handler
        vendor_id       : [0][0][0][0]
        encoder         : HEVC Coding
  Stream #0:1[0x2](eng): Audio: aac (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 317 kb/s (default)
      Metadata:
        creation_time   : 2024-05-01T17:02:05.000000Z
        handler_name    : #Mainconcept MP4 Sound Media Handler
        vendor_id       : [0][0][0][0]

So far FFMPEG maxes out at 400Mbps (6.1?). When going above that it errors:

ffmpeg -i ./prores4444.mov -f mp4 -ac 2 -b:a 320.0k -b:v 800M -brand mp43 -c:a aac -c:v hevc_nvenc -g 0 -level 6.2 -pix_fmt p010le -preset 18 -profile:v main10 -rc 2 -shortest -strict experimental -tag:v hvc1 -tier high -tune 1 ./800Mbps.mp4 -y
ffmpeg version 7.0 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 9.3.0 (GCC)
  configuration: --prefix=/images/tmp/jenkins/install/forge/install/ffmpeg/ffmpeg.org/ffmpeg/linux_64/7.0 --cc=//packages/vendor/industrial/kiln.platform/5.1/CentOS_Linux_7.9/x86_64/gcc/9.3.0/bin/gcc --cxx=//packages/vendor/industrial/kiln.platform/5.1/CentOS_Linux_7.9/x86_64/gcc/9.3.0/bin/g++ --extra-cxxflags='-std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0' --enable-shared --enable-avfilter --enable-postproc --enable-pthreads --enable-cuda --enable-gpl --enable-libaom --enable-libfdk_aac --enable-libmp3lame --enable-libvpx --enable-libfreetype --enable-libx264 --enable-libx265 --enable-libopus --enable-cuvid --enable-nvdec --enable-nvenc --enable-nonfree --enable-ffplay --enable-libpulse --extra-libs=-lpthread --extra-libs=-ldl --extra-libs=-lm --extra-libs=-lstdc++
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './prores4444.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf60.3.100
  Duration: 00:03:00.72, start: 0.000000, bitrate: 1107315 kb/s
  Stream #0:0[0x1]: Video: prores (4444) (ap4h / 0x68347061), yuv444p12le(bt709, progressive), 3840x2160, 1099008 kb/s, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 12800 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : FFMP
  Stream #0:1[0x2]: Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (prores (native) -> hevc (hevc_nvenc))
  Stream #0:1 -> #0:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
[hevc_nvenc @ 0xcf0e40] InitializeEncoder failed: invalid param (8): Invalid Level.
[vost#0:0/hevc_nvenc @ 0xcf0900] Error while opening encoder - maybe incorrect parameters such as bit_rate, rate, width or height.
[vf#0:0 @ 0xd0c2c0] Error sending frames to consumers: Invalid argument
[vf#0:0 @ 0xd0c2c0] Task finished with error code: -22 (Invalid argument)
[vf#0:0 @ 0xd0c2c0] Terminating thread with return code -22 (Invalid argument)
[vost#0:0/hevc_nvenc @ 0xcf0900] Could not open encoder before EOF
[vost#0:0/hevc_nvenc @ 0xcf0900] Task finished with error code: -22 (Invalid argument)
[vost#0:0/hevc_nvenc @ 0xcf0900] Terminating thread with return code -22 (Invalid argument)
[out#0/mp4 @ 0xcefa40] Nothing was written into output file, because at least one of its streams received no packets.
frame=    0 fps=0.0 q=0.0 Lsize=       0KiB time=N/A bitrate=N/A speed=N/A    
[aac @ 0xd1bb00] Qavg: nan
Conversion failed!

If I let FFMPEG pick the output pix_fmt it chooses yuv444p16le, switches from main10 to Rext and starts encoding:

08:55:07 william@pika : ffmpeg -i ./prores4444.mov -f mp4 -ac 2 -b:a 320.0k -b:v 800M -brand mp43 -c:a aac -c:v hevc_nvenc -g 0 -level 6.2 -preset 18 -profile:v main10 -rc 2 -shortest -strict experimental -tag:v hvc1 -tier high -tune 1 ./800Mbps.mp4 -y
ffmpeg version 7.0 Copyright (c) 2000-2024 the FFmpeg developers
  built with gcc 9.3.0 (GCC)
  configuration: --prefix=/images/tmp/jenkins/install/forge/install/ffmpeg/ffmpeg.org/ffmpeg/linux_64/7.0 --cc=//packages/vendor/industrial/kiln.platform/5.1/CentOS_Linux_7.9/x86_64/gcc/9.3.0/bin/gcc --cxx=//packages/vendor/industrial/kiln.platform/5.1/CentOS_Linux_7.9/x86_64/gcc/9.3.0/bin/g++ --extra-cxxflags='-std=c++17 -D_GLIBCXX_USE_CXX11_ABI=0' --enable-shared --enable-avfilter --enable-postproc --enable-pthreads --enable-cuda --enable-gpl --enable-libaom --enable-libfdk_aac --enable-libmp3lame --enable-libvpx --enable-libfreetype --enable-libx264 --enable-libx265 --enable-libopus --enable-cuvid --enable-nvdec --enable-nvenc --enable-nonfree --enable-ffplay --enable-libpulse --extra-libs=-lpthread --extra-libs=-ldl --extra-libs=-lm --extra-libs=-lstdc++
  libavutil      59.  8.100 / 59.  8.100
  libavcodec     61.  3.100 / 61.  3.100
  libavformat    61.  1.100 / 61.  1.100
  libavdevice    61.  1.100 / 61.  1.100
  libavfilter    10.  1.100 / 10.  1.100
  libswscale      8.  1.100 /  8.  1.100
  libswresample   5.  1.100 /  5.  1.100
  libpostproc    58.  1.100 / 58.  1.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from './prores4444.mov':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf60.3.100
  Duration: 00:03:00.72, start: 0.000000, bitrate: 1107315 kb/s
  Stream #0:0[0x1]: Video: prores (4444) (ap4h / 0x68347061), yuv444p12le(bt709, progressive), 3840x2160, 1099008 kb/s, SAR 1:1 DAR 16:9, 25 fps, 25 tbr, 12800 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : FFMP
  Stream #0:1[0x2]: Audio: pcm_s16le (sowt / 0x74776F73), 48000 Hz, stereo, s16, 1536 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (prores (native) -> hevc (hevc_nvenc))
  Stream #0:1 -> #0:1 (pcm_s16le (native) -> aac (native))
Press [q] to stop, [?] for help
Output #0, mp4, to './800Mbps.mp4':
  Metadata:
    major_brand     : qt  
    minor_version   : 512
    compatible_brands: qt  
    encoder         : Lavf61.1.100
  Stream #0:0: Video: hevc (Rext) (hvc1 / 0x31637668), yuv444p16le(tv, bt709, progressive), 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 800000 kb/s, 25 fps, 12800 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : FFMP
        encoder         : Lavc61.3.100 hevc_nvenc
      Side data:
        cpb: bitrate max/min/avg: 0/0/800000000 buffer size: 1600000000 vbv_delay: N/A
  Stream #0:1: Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 320 kb/s (default)
      Metadata:
        handler_name    : SoundHandler
        vendor_id       : [0][0][0][0]
        encoder         : Lavc61.3.100 aac
frame=   44 fps= 22 q=-1.0 size=   68864KiB time=00:00:01.76 bitrate=320530.8kbits/s speed=0.88x     

[q] command received. Exiting.

Sadly this output is not compatible with the client hardware.

It's unclear what the issue is other than FFMPEG + the hevc_nvenc encoder not fully supporting the 6.2 spec.?

Any help/ideas/pointers would be hugely appreciated.

Thanks

Which GPU and driver are you using? It may be a limitation of the underlying hardware (abstraction). — Zac67, Commented May 30 at 11:26
Just for testing try: ffmpeg -an -i input.mp4 -b:v 440M -c:v hevc_nvenc -tier high -level 6.2 -pix_fmt p010le -profile:v main10 440Mbps.mp4 -y. It's working in my machine with 440Mbps, but my GPU model is GTX 1650. Adding -loglevel debug may show more details. When I try 450Mbps the error is (also) InitializeEncoder failed: invalid param (8): Invalid Level. (the error seems to be related to the GPU / GPU Driver / NVIDIA Video Codec SDK). — Rotem, Commented May 30 at 15:50
Can you share the output of ffmpeg -i FILE for a 800 mbps file produced by AME? — Gyan, Commented May 31 at 4:47
Thanks for the feedback. Zac67 & Gyan info added. Rotem, that worked. So audio is impacting max bitrate. But still not hitting 6.1 max of 480. Hmmm. — William Marshall Wilkinson, Commented May 31 at 9:57

Stack Exchange Network

FFMPEG HEVC NVENC (yuv420)p10le at 800Mbps not encoding

0

You must log in to answer this question.

Browse other questions tagged
linux
ffmpeg
video
gpu
h.265
.

Hot Network Questions

FFMPEG HEVC NVENC (yuv420)p10le at 800Mbps not encoding

0

You must log in to answer this question.

Browse other questions tagged linuxffmpegvideogpuh.265.

Related

Hot Network Questions

Browse other questions tagged
linux
ffmpeg
video
gpu
h.265
.