SCE2: Combination of inter-layer syntax prediction and motion data compression

SCE2: Combination of inter-layer syntax prediction and motion data compression

1. SCE2 summary and general discussion

15.0.0.1.1.1.1.1.141JCTVC-N0032 SCE2: Summary report of SHVC core experiment on combination of inter-layer syntax prediction and motion data compression [Christophe Gisquet, Kazushi Sato]

Three basic methods were investigated as follows:

The collocated position of the base layer is calculated as;

xRL = ( ( xRL + R ) >> S ) << S

yRL = ( ( yRL + R ) >> S ) << S

where originally R=4, S=4 , subsampling is 4:1

• 
  Tests 1.1, 1.4 (combination with 2.2): R=-2, S=3 (subsampling is 2:1);
  
• 
  Test 1.2 (combination with 2.3): R=4, S=4 (compression is 4:1, i.e. unchanged)
  

In terms of complexity impact, 2.1 should be roughly identical to SHM (two additional additions), no additional memory accesses or usage. It is just re-defining the position from the BL MV memory that is used to generate the motion field mapping.

The question was raised whether it could happen that an MV is accessed which is outside of the BL MV memory. It is confirmed that the specification text includes a clipping operation preventing this.

Decision: Adopt (2.1 = JCTVC-N0139).

2.2 would require additional memory. In the analysis made by the last meeting the additional MV memory was assessed to be increased by 40% (SNR scalability), 10-15% (spatial scalability). Analysis for on-the-fly computation (as suggested by AHG17 and confirmed by JCT-VC to be the common ground for complexity analysis) is not available yet, but likely lower. Provide analysis for on-the-fly computation.

Further analysis was provided and discussion on this issue was performed in Track B on Sunday afternoon (JRO). “On-the-fly” computation in the case of MV would mean that the MV memory corresponding to the EL’s BL reference is written by the time when the base layer MV is decoded. This would in worst case require an additional memory for MV at the EL resolution, regardless whether uncompressed or compressed BL MV are used. However, in a “HLS only implementation” concept, a bigger problem could arise that uncompressed BL MV are only available on the chip and are not accessible.

This was discussed in plenary on Mon. 29th.

Estimated benefit of accessing internal MV memory is 0.7% for 2x scalability; less for 1.5x scalability, nothing for SNR scalability.

Several experts expressed the opinion that it is undesirable to require access to the uncompressed MV memory, as this would require alteration of the BL decoder . It was agreed not to access the finer granularity motion vectors. No action or further investigation was planned on 2.2.

It was also noted that the memory for MV compression is not counted (and there is no specification about limits) in the DPB, as it is usually around 3%. Therefore, several experts suggested whether it would be reasonable to re-consider using the uncompressed MV field (see JCTVC-M0142 et al.). Analysis for memory consumption assuming on-the-fly computation should also be provided for that case.

It was also asked whether the two-stage compression (2:1->4:1) provides the same 4:1 compressed MV field as the base spec. It was later confirmed that this is the case.

Gives gain mostly for 2X scalability; overall the gain is marginal.

For frame-based implementation and worst-case SNR scalability, the MV memory requirements would need to be understood.

2.3: For the motion field mapped from the base layer the motion information is replaced with motion information from a candidate enhancement layer motion field, if the candidate enhancement layer motion information is valid and does not reference its own base layer. In particular, the motion information includes - prediction mode, motion vector, reference picture POC, reference picture used as long term status.

The replacement process is controlled at the sequence and slice level using syntax elements sps_override_mfm_flag and el_collocated_enabled_flag respectively. The candidate enhancement layer motion field is identified using slice level syntax elements el_collocated_from_flag and el_collocated_ref_idx.

To be provided: Analysis about worst-case computation and memory accesses. JCTVC-N0233 is an extension of the (N0252) method which is said to be simpler, should also be included in this analysis. Further

A high-level analysis was shown in Track B on Sunday afternoon, which showed that different items need to be checked (some at picture level, some at 16x16 block level), but did not contain sufficient detail to understand the worst-case complexity of additional comparisons.

Gives gain for 2X and 1.5X cases (a bit less for the latter)

Further worst case complexity analysis was presented Thursday evening:

• 
  2 checks at slice level
  
• 
  4 checks at block level, each for L0 and L1
  

No gain was shown for AI and LD cases; however a smart encoder could potentially switch the tool off for these cases.

Currently we do not use joint optimization of base and enhancement layer – how would the tool perform in that case?

Further study was encouraged – it would be desirable to also achieve gain in the LD case.

Specification text for 2.2 and 2.3 was not available initially. It was presented in Track B Sunday afternoon for both proposals. The text change for 2.2 is minimal (change in 2 shift operations when determining the collocated MV), whereas the text change for 2.3 reflects the additional checks that are necessary on various syntax elements.
As a general remark, 2.2 and 2.3 provide interesting gains, which should be considered against the additional complexity with the additional information requested
Results:

Average, also including optional LD P:

Further discussion N0252 / N0233 Thu p.m. (GS):

Blending of EL & BL motion data

0.5% improvement reported on RA2x.

Complexity analysis presented in revision of N0252. Four checks for each list at 16×16 block level, more checks (~15) at slice level. Negligible runtime perf impact.

Gain is only in RA. The feature was actually disabled in the LD case, as it was not useful there.

It was commented that this requires an extra motion field buffer in the SNR scalabiity case.

The topic was agreed to be for further study.

2. SCE2 primary contributions

3. SCE2 cross checks

3. 
   
   Directory: sc29 -> open -> 29view
   29view -> International organisation for standardisation organisation internationale de normalisation
   29view -> International organisation for standardisation organisation internationale de normalisation
   
   
   
   Download 8.47 Mb.
   
   Share with your friends: