FFMPEG - Set custom PTS on mp4 file

Question

I have an ffmpeg command setting custom pts on an input video.
It works fine for .mkv file but not on .mp4 file. When running the following command on an mp4 file, most of frames are dropped. Although my pts is monotonic and always increasing.

ffmpeg -i video.mp4 -filter_complex "[v]setpts='0*eq(N,0)+160*eq(N,1)+191*eq(N,2)+211*eq(N,3)...'[out];" -map [out] out_video.mp4

Is it possible to setpts on mp4 file ?

Thanks!

Answer 1

We may add -fps_mode passthrough, set the MP4 timebase explicitly, and divide the PTS by the timebase:

ffmpeg -y -i input.mp4 -fps_mode passthrough -filter_complex "[v]setpts='(0*eq(N,0)+160*eq(N,1)+191*eq(N,2)+211*eq(N,3))/TB/1000'[out]" -video_track_timescale 100000 -map [out] out_video.mp4

---

• -fps_mode passthrough - Each frame is passed with its timestamp from the demuxer to the muxer (prevents frame duplication and dropping).
• /TB - Dividing by the timebase is used for converting the PTS to seconds.
  /TB/1000 - Used for converting the PTS to milliseconds (assuming the numbers applies milliseconds).
• -video_track_timescale 100000 - Set explicit timebase for the MP4 as described here.
  In most cases it is not required, but there may be cases when the timestamps are going to be rounded due to low timebase accuracy.

---

Testing:

Create sample input video for testing:
ffmpeg -y -f lavfi -i testsrc=size=192x108:rate=100:duration=0.04 input.mp4

Execute the command with the setpts filter:
ffmpeg -y -i input.mp4 -fps_mode passthrough -filter_complex "[v]setpts='(0*eq(N,0)+160*eq(N,1)+191*eq(N,2)+211*eq(N,3))/TB/1000'[out]" -video_track_timescale 100000 -map [out] out_video.mp4

Viewing the PTS timestamps of out_video.mp4 using FFprobe:
ffprobe -select_streams v -of default=noprint_wrappers=1:nokey=1 -show_entries frame=pts_time out_video.mp4

Output:

0.000000
0.160000
0.190000
0.210000

---

There is a "catch":

The above solution is not going to work if the "frame period" of the the input frames is too long.

The command: ffmpeg -y -f lavfi -i testsrc=size=192x108:rate=100:duration=0.04 input.mp4, produceses video at 100fps (100Hz).
The "frame period" is 1/100 = 10msec.

When creating 1Hz video for example: ffmpeg -y -f lavfi -i testsrc=size=192x108:rate=1:duration=4 input_1fps.mp4, the "frame period" is 1sec.

We may check the "frame period" of input_1fps.mp4 using FFprobe:
ffprobe -select_streams v -of default=noprint_wrappers=1:nokey=1 -show_entries packet=duration_time input_1fps.mp4

Output:

1.000000
1.000000
1.000000
1.000000

Now, executing the setpts with the 1fps video:
ffmpeg -y -i input_1fps.mp4 -fps_mode passthrough -filter_complex "[v]setpts='(0*eq(N,0)+160*eq(N,1)+191*eq(N,2)+211*eq(N,3))/TB/1000'[out]" -video_track_timescale 100000 -map [out] out_video.mp4

And FFprobe: ffprobe -select_streams v -of default=noprint_wrappers=1:nokey=1 -show_entries frame=pts_time out_video.mp4.

The output is not as intended:

0.000000
0.000020
0.000030
0.000010

---

Why isn't it working???

1. setpts sets the PTS timestamps, but doesn't modify the "frame period".
2. The delta time between two consecutive timestamps can't exceed the "frame period".

Why setpts doesn't modify the "frame period"?
The reason is that the "frame period" is part of the encoded stream (say H.264), and the timestamps are part of the container (say MP4).

How can we modify the "frame period"?

We may use Bitstream Filter for modifying the "frame period".
Note: there is setts filter, but it looks like it's not allows modifying the "frame period"...

---

It looks like the simplest solution is converting the frames to sequence of PNG images:

ffmpeg -i input_1fps.mp4 frame_%08d.png

Convert to MP4 using the setpts filter:

ffmpeg -y -framerate 1000 -i frame_%08d.png -fps_mode passthrough -filter_complex "[v]setpts='(0*eq(N,0)+160*eq(N,1)+191*eq(N,2)+211*eq(N,3))/TB/1000'[out]" -map [out] out_video.mp4

---

Checking:
ffprobe -select_streams v -of default=noprint_wrappers=1:nokey=1 -show_entries frame=pts_time out_video.mp4

Output:

0.000000
0.160000
0.191000
0.211000