[IMPROVEMENT] feat(mp4): add FFmpeg/libavformat backend for MP4 demuxing

[gaurav02081]

[IMPROVEMENT] feat(mp4): add FFmpeg/libavformat backend for MP4 demuxing

In raising this pull request, I confirm the following (please check boxes):

• I have read and understood the contributors guide.
• I have checked that another pull request for this purpose does not exist.
• I have considered, and confirmed that this submission will be valuable to others.
• I accept that this submission may not be used, and the pull request closed at the will of the maintainer.
• I give this submission freely, and claim no ownership to its content.
• I have mentioned this change in the changelog.

My familiarity with the project is as follows (check one):

• I have never used CCExtractor.
• I have used CCExtractor just a couple of times.
• I absolutely love CCExtractor, but have not contributed previously.
• I am an active contributor to CCExtractor.

---

Add optional FFmpeg-based MP4 parser as an alternative to GPAC

This PR introduces an alternative MP4 parsing backend using FFmpeg's libavformat, while keeping the existing GPAC-based implementation unchanged and as the default.

Motivation

In a previous discussion (Gsoc meeting 2 MARCH) we talked about updating the GPAC dependency used for MP4 processing in CCExtractor. One suggestion was to explore whether there is a Debian-friendly alternative rather than only focusing on upgrading GPAC.

FFmpeg is already used in other parts of the codebase (for example in the demuxing/decoding integration and in the HardsubX module), so extending its use for MP4 parsing seemed like a reasonable option to explore.

Implementation

A new implementation (mp4_ffmpeg.c) was added which uses FFmpeg's libavformat to open and parse MP4 containers.

The general workflow is:

• Open the MP4 container with avformat_open_input()
• Discover streams using avformat_find_stream_info()
• Read packets sequentially using av_read_frame()
• Dispatch packets based on stream type

Video packets (H.264 / HEVC) are passed to the existing do_NAL() processing logic, while caption tracks (CEA-608 / CEA-708) and subtitle tracks (tx3g) continue to use the existing CCExtractor parsing functions.

One difference from the GPAC implementation is that FFmpeg reads packets sequentially across all streams, whereas the GPAC implementation reads samples per track. The downstream caption extraction pipeline remains unchanged.

For H.264 / HEVC streams, codec configuration data is obtained from the stream extradata (avcC / hvcC) in order to determine the NAL unit length prefix size and extract SPS/PPS before processing packets.

Build configuration

This backend is optional and controlled through a compile-time flag:

-DUSE_FFMPEG_MP4=ON

• Default build → uses GPAC (mp4.c)
• FFmpeg build → uses the new implementation (mp4_ffmpeg.c)

The runtime behavior of CCExtractor remains unchanged — the difference only affects how the MP4 container is parsed internally.

Summary

This PR:

• Adds an FFmpeg-based MP4 parser
• Keeps GPAC as the default implementation
• Introduces a compile-time option to switch between the two backends
• Leaves the caption extraction pipeline unchanged

This provides a potential alternative MP4 backend using a widely available multimedia framework while preserving the existing behavior.

[gaurav02081]

Update: Added a new CI job to build CCExtractor with the optional FFmpeg MP4 backend.

The workflow now performs two builds:

Default build using the GPAC implementation

FFmpeg build using -DUSE_FFMPEG_MP4=ON

Both builds run --version to verify the binaries execute correctly, and separate build directories are used to avoid CMake cache conflicts.

[cfsmp3]

Thanks for the PR — the implementation is clean and well-structured, and the dedicated CI job is a nice touch.

However, we're actively moving the codebase from C to Rust, so we can't accept new C modules. We'd need the FFmpeg MP4 demuxer to be implemented in Rust. See #2170 for an example of how this can be done using rsmpeg (Rust FFmpeg bindings) with a thin C bridge for the callbacks into the existing C code.

If you'd like to rework this in Rust we'd be happy to review it. The overall approach (avformat_open_input → av_read_frame → dispatch by stream type) is sound, it's just the implementation language that needs to change.

[gaurav02081]

FFmpeg-based implementation written in Rust using rsmpeg, activated via -DWITH_FFMPEG=ON. The GPAC path
remains the default and is completely untouched when FFmpeg is not enabled.

Architecture

The implementation follows the same 3-layer pattern as PR #2170:

= Layer 1 — Rust Core (src/rust/src/demuxer/mp4.rs)

• Opens MP4 via rsmpeg::AVFormatContextInput
• Classifies streams: AVC/H.264, HEVC/H.265, CEA-608, CEA-708, tx3g
• Parses SPS/PPS/VPS from codec extradata
• Main av_read_frame loop dispatches packets to C bridge functions

= Layer 2 — FFI Exports (src/rust/src/mp4_ffmpeg_exports.rs)

• ccxr_processmp4() — replaces processmp4()
• ccxr_dumpchapters() — replaces dumpchapters()

= Layer 3 — C Bridge (src/lib_ccx/mp4_rust_bridge.c)

• Flat FFI-safe wrappers around existing C processing functions
• ccx_mp4_process_avc_sample() → NAL parsing via do_NAL()
• ccx_mp4_process_hevc_sample() → NAL parsing + store_hdcc() flush
• ccx_mp4_process_cc_packet() → CEA-608 via process608(), CEA-708 via ccdp_find_data() + ccxr_dtvcc_process_data()
• ccx_mp4_process_tx3g_packet() → 3GPP timed text
• No GPAC structs cross the FFI boundary — only primitive types

Build System

cmake -DWITH_FFMPEG=ON -DWITH_OCR=ON -DWITH_HARDSUBX=ON ../src

• ENABLE_FFMPEG_MP4 compile flag gates all new code
• Cargo feature enable_mp4_ffmpeg pulls in rsmpeg
• Corrosion passes the feature to Cargo when WITH_FFMPEG is on
• FFmpeg libs are added as INTERFACE_LINK_LIBRARIES of ccx_rust to ensure correct GNU ld link order
• Bridge functions use --undefined linker flags (same pattern as existing decode_vbi/do_cb/store_hdcc)

Files

New (5):

src/rust/src/demuxer/mp4.rs         Rust FFmpeg demuxer core      
 src/rust/src/mp4_ffmpeg_exports.rs  C-callable FFI exports        
 src/lib_ccx/mp4_rust_bridge.c       Thin C bridge                 
 src/lib_ccx/mp4_rust_bridge.h       Bridge header                 
 src/lib_ccx/ccx_gpac_types.h        Minimal GPAC-compatible types 

Modified (13):```
Cargo.toml, build.rs, lib.rs, demuxer/mod.rs, wrapper.h, rust/CMakeLists.txt, src/CMakeLists.txt, lib_ccx/CMakeLists.txt, ccx_mp4.h,
ccextractor.c, build_linux.yml, build_mac.yml

  Testing

  - GPAC-only build (cmake ../src) — compiles and links, no regression
  - FFmpeg build (cmake -DWITH_FFMPEG=ON -DWITH_OCR=ON -DWITH_HARDSUBX=ON ../src) — compiles and links
  - Runtime: ./ccextractor tests/samples/BBC1.mp4 produces identical output to GPAC path
  - CI: Linux and macOS cmake_ffmpeg_mp4 jobs added

[ccextractor-bot]

CCExtractor CI platform finished running the test files on linux. Below is a summary of the test results, when compared to test for commit 5c87a33...:

Report Name	Tests Passed
Broken	10/13
CEA-708	2/14
DVB	4/7
DVD	3/3
DVR-MS	2/2
General	27/27
Hardsubx	1/1
Hauppage	3/3
MP4	3/3
NoCC	10/10
Options	79/86
Teletext	20/21
WTV	13/13
XDS	34/34

Your PR breaks these cases:

• ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...
• ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9...

NOTE: The following tests have been failing on the master branch as well as the PR:

• ccextractor --out=srt --latin1 --autoprogram 73d9313d64..., Last passed:

  Test 8738

• ccextractor --out=ttxt --latin1 001dd8cdf7..., Last passed:

  Test 8738

• ccextractor --out=srt --latin1 4d4e938ef6..., Last passed:

  Test 8738

• ccextractor --service 1 --out=txt --no-bom --no-rollup ea83ff7bcb..., Last passed:

  Test 8738

• ccextractor --service 1 --out=txt f17524b53f..., Last passed:

  Test 8738

• ccextractor --service 1 --out=txt 80848c45f8..., Last passed:

  Test 8738

• ccextractor --service 1 --out=txt --no-bom --no-rollup b5d6aad89f..., Last passed:

  Test 8738

• ccextractor --service 1[EUC-KR] --out=txt --no-rollup b5d6aad89f..., Last passed:

  Test 8738

• ccextractor --service 1 --out=srt da904de35d..., Last passed:

  Test 8738

• ccextractor --service 1 --out=sami da904de35d..., Last passed:

  Test 8738

• ccextractor --service 1[EUC-KR] b5d6aad89f..., Last passed:

  Test 8738

• ccextractor --service 1[EUC-KR] --no-rollup b5d6aad89f..., Last passed:

  Test 8738

• ccextractor --service all da904de35d..., Last passed:

  Test 8738

• ccextractor --service all[EUC-KR] b5d6aad89f..., Last passed:

  Test 8738

• ccextractor --service 1,2[UTF-8],3[EUC-KR],54 --out=txt da904de35d..., Last passed:

  Test 8738

• ccextractor --autoprogram --out=srt --latin1 d41b53b504..., Last passed:

  Test 8738

• ccextractor --stdout --quiet --no-fontcolor 79a51f3500..., Last passed:

  Test 8738

• ccextractor --stdout --quiet --no-fontcolor 767b546f96..., Last passed:

  Test 8738

• ccextractor --service 1 c83f765c66..., Last passed:

  Test 8738

• ccextractor --myth c83f765c66..., Last passed:

  Test 8738

• ccextractor --in=raw fb79021542..., Last passed:

  Test 8738

• ccextractor --mp4vidtrack 5df914ce77..., Last passed:

  Test 8738

• ccextractor --xmltv=3 --out=null 96efd279cf..., Last passed:

  Test 8738

• ccextractor --datapid 2310 --autoprogram --out=srt --latin1 e639e54550..., Last passed:

  Test 8738

Congratulations: Merging this PR would fix the following tests:

• ccextractor --autoprogram --out=srt --latin1 --quant 0 85271be4d2..., Last passed: Never
• ccextractor --autoprogram --out=ttxt --latin1 --ucla dab1c1bd65..., Last passed: Never
• ccextractor --out=srt --latin1 --autoprogram 29e5ffd34b..., Last passed: Never
• ccextractor --out=spupng c83f765c66..., Last passed: Never
• ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
• ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
• ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never

It seems that not all tests were passed completely. This is an indication that the output of some files is not as expected (but might be according to you).

Check the result page for more info.

[ccextractor-bot]

CCExtractor CI platform finished running the test files on windows. Below is a summary of the test results, when compared to test for commit 5c87a33...:

Report Name	Tests Passed
Broken	10/13
CEA-708	2/14
DVB	4/7
DVD	3/3
DVR-MS	2/2
General	27/27
Hardsubx	1/1
Hauppage	3/3
MP4	3/3
NoCC	10/10
Options	81/86
Teletext	20/21
WTV	13/13
XDS	34/34

NOTE: The following tests have been failing on the master branch as well as the PR:

• ccextractor --out=srt --latin1 --autoprogram 73d9313d64..., Last passed:

  Test 8611

• ccextractor --out=ttxt --latin1 001dd8cdf7..., Last passed:

  Test 8611

• ccextractor --out=srt --latin1 4d4e938ef6..., Last passed:

  Test 8611

• ccextractor --service 1 --out=txt --no-bom --no-rollup ea83ff7bcb..., Last passed:

  Test 8611

• ccextractor --service 1 --out=txt f17524b53f..., Last passed:

  Test 8611

• ccextractor --service 1 --out=txt 80848c45f8..., Last passed:

  Test 8611

• ccextractor --service 1 --out=txt --no-bom --no-rollup b5d6aad89f..., Last passed:

  Test 8611

• ccextractor --service 1[EUC-KR] --out=txt --no-rollup b5d6aad89f..., Last passed:

  Test 8611

• ccextractor --service 1 --out=srt da904de35d..., Last passed:

  Test 8611

• ccextractor --service 1 --out=sami da904de35d..., Last passed:

  Test 8611

• ccextractor --service 1[EUC-KR] b5d6aad89f..., Last passed:

  Test 8611

• ccextractor --service 1[EUC-KR] --no-rollup b5d6aad89f..., Last passed:

  Test 8611

• ccextractor --service all da904de35d..., Last passed:

  Test 8611

• ccextractor --service all[EUC-KR] b5d6aad89f..., Last passed:

  Test 8611

• ccextractor --service 1,2[UTF-8],3[EUC-KR],54 --out=txt da904de35d..., Last passed:

  Test 8611

• ccextractor --autoprogram --out=srt --latin1 d41b53b504..., Last passed:

  Test 8611

• ccextractor --stdout --quiet --no-fontcolor 79a51f3500..., Last passed:

  Test 8611

• ccextractor --stdout --quiet --no-fontcolor 767b546f96..., Last passed:

  Test 8611

• ccextractor --service 1 c83f765c66..., Last passed:

  Test 8611

• ccextractor --myth c83f765c66..., Last passed:

  Test 8611

• ccextractor --in=raw fb79021542..., Last passed:

  Test 8611

• ccextractor --mp4vidtrack 5df914ce77..., Last passed:

  Test 8611

• ccextractor --xmltv=3 --out=null 96efd279cf..., Last passed:

  Test 8611

• ccextractor --datapid 2310 --autoprogram --out=srt --latin1 e639e54550..., Last passed:

  Test 8611

Congratulations: Merging this PR would fix the following tests:

• ccextractor --autoprogram --out=srt --latin1 --quant 0 85271be4d2..., Last passed: Never
• ccextractor --autoprogram --out=ttxt --latin1 --ucla dab1c1bd65..., Last passed: Never
• ccextractor --out=srt --latin1 --autoprogram 29e5ffd34b..., Last passed: Never
• ccextractor --out=spupng c83f765c66..., Last passed: Never
• ccextractor --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
• ccextractor --startcreditsnotbefore 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
• ccextractor --startcreditsnotafter 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
• ccextractor --startcreditsforatleast 1 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never
• ccextractor --startcreditsforatmost 2 --startcreditstext "CCextractor Start crdit Testing" c4dd893cb9..., Last passed: Never

This PR does not introduce any new test failures. However, some tests are failing on both master and this PR (see above).

Check the result page for more info.