framework using OpenMAX DL

(1)

(2)

URBAN WP 1

Implementation of the flexible compression

framework using OpenMAX DL

(3)

Work package 1: Digital Surveying

• Digital Surveying

– Processing on board of the van

• Multi-video capture

– de-bayering and coding

– Processing on the servers

• 3d reconstruction

– decoding, automatic feature extraction and matching…

(4)

Patrice Rondao Alface

imec restricted 2008 4

Work package 1: Digital Surveying

• Main Goal:

Algorithm optimizations for lower computing complexity and improved efficiency

bitrate quality

Speed-up

Moore’s law?

ILP, memory and power walls Parallelization

(5)

Hardware and Parallelization

• Parallel programming

– Not all algorithms are parallelizable (e.g. entropy coding…)

– Modifications to enable parallelism at the expense of quality performances – Different algorithms call for different programming patterns for different

strategies and hardware…

• CPU:

• Multithread/Multicore: OpenMP

• SIMD: MMX/SSE e.g. Intel’s Performance Primitives lib. (IPP)

• GPU

– General Purpose GPU programming – CUDA programming model

(6)

Task 1.1: On-board the recording van

D.1.1.2 Implementation of the flexible

compression framework using OpenMAX DL

(7)

Khronos Group

(8)

Khronos Group: standards

(9)

OpenMAX: portability

(10)

OpenMAX DL

• OpenMAX DL (Development Layer)

– API (initially for mobile multimedia platforms) with audio, video and imaging core functions

– to be implemented and optimized on new processors by silicon vendors and used by codec vendors.

– enables the optimized implementation of codecs such as MPEG-4, H.264, MP3, AAC and JPEG.

– AVC/H.264

• Baseline Profile only!

• Decoder/encoder

• Available ARM implementations

• Functions defined at Macroblock-level

• … Limitations

(11)

OpenMAX DL

• Functional units (at Macroblock level)

– Intra Prediction – Interpolation – Deblocking Filter

– VLC and CAVLC decoding – Inverse Quantization – Transform

– Add Residual

– Motion Estimation

• Motion Estimation Helper

• Motion Estimation Low-level

• Motion Estimation High-level

(12)

OpenMAX DL

• Limitations : OpenMAX is still under development…

– Initialization:

• No parameter describing the resolution of the video

• No way of transmitting image or video features

• No “reset” foreseen

• No parameter differentiating Luma and Chroma channels: loss in compression efficiency

– Transform and quantization functions

• Only make sense at the decoding side

(13)

Algorithmic Optimizations

• Motion Estimation:

– Motion Estimation

• Low-level and high-level Motion Estimation functions cannot share SAD values, only weighted cost:

– preventing threshold-based skip of the fractional ME

• Storing pre-interpolated or on-demand half and quarter pixel values is not allowed:

– many redundant interpolations

• Features such as edges enable fast partition-decisions – but cannot be computed globally at initialization: MB local

convolution induces complexity overhead

• …

(14)

Results

• OpenMAX DL compliance offers portability but limits algorithmic optimizations

no standard API OpenMAX DL Compliant

Sequence PSNR bitrate fps PSNR bitrate fps

Crew QCIF 15 fps 200 kbps 38.335 202.712 39.88 38.335 203.056 25.08 Foreman CIF 30 fps 800 kbps38.504 790.84 13.21 38.825 788.96 8.42 Crew CIF 30 fps1 Mbps 39.111 1008.648 9.92 39.151 1007.288 8.24 Crew 4CIF 30 fps 6 Mbps 40.346 6010.408 2.62 40.384 6009.848 2.15 Jets 720p 60 fps 10 Mbps 41.948 10015.63 2.87 41.793 10011.17 2.28 City 720p 60 fps 10 Mbps 35.669 7969.744 2.65 35.219 8070.424 1.83 Crew 720p 60 fps 10 Mbps 39.569 10009.83 1.08 39.079 10000.29 1.33

(15)

Results

• Comparison with the JM reference encoder

(16)

Results

• Comparison with the JM reference encoder (continued)

(17)

Conclusions

• The OpenMAX DL APIs, the freely available ARM OpenMAX DL library as well as Intel’s Performance Primitives (IPP) allow to quickly bring up a executable behavioral model (in C) of the AVC Baseline encoder.

• As the algorithmic details of the VFUs are hidden below the OpenMAX API, the focus is on the dataflow at

framework level.

• However, the OpenMAX DL APIs are still under

development and their current definition bring some

(18)

Conclusions

• Solving those OpenMAX API's issues and applying a set of high-level optimizations

brought the compression efficiency at a similar level as the reference JM encoder.

• Our algorithmic optimizations further improved the encoding framerate, but induced sensitivity to the video sequence type resulting in a quality loss for some test cases.

• Next deliverable D1.1.3 (June‘09): optimized

compression framework

(19)