International organisation for standardisation organisation internationale de normalisation

Generic HLS issues

15.0.0.1.1.1.1.1.356JCTVC-N0374 BoG report on SHVC/MV-HEVC HLS topics [J. Boyce]

Joint BoG with JCT-3V.

(Initially reviewed Sun 28th Track A (GJS).)

The BoG met 27 July to review the following contributions:

• 
  6.4.4.5 Reference picture list construction
  JCTVC-N0082, JCTVC-N0095, JCTVC-N0216, JCTVC-N0316
  
• 
  6.4.3.3 Efficient parameter set parameters signalling
  JCTVC-N0162, JCTVC-N0200, JCTVC-N0212
  
• 
  6.6.2 Layer presence and dependency change SEI messages
  JCTVC-N0173, JCTVC-N0174
  

• 
  Adopt JCTVC-N0082, initialization process of reference picture lists for HEVC extensions.
  
• 
  Adopt JCTVC-N0371, scaling list prediction in SPS and PPS (harmonization of JCTVC-N0162 and JCTVC-N0200 variant 3).
  
• 
  Layer dependency change SEI message (which originated from JVT-S080) be removed from specification (since a new VPS could be sent to change the layer dependency). If the SEI message does remain, to adopt JCTVC-N0174 (with some editorial improvements).
  

Decision: Agreed to these BoG recommendations (assuming 3V confirmation).

The BoG then met again to further discuss initialization of reference picture lists: Revision to JCT3V-D0220, and harmonization of JCTVC-N0095 and JCTVC-N0216, with experimental results compared to JCTVC-N0082

The BoG met 28 July to review the following contributions:

• 
  6.4.4.3 General inter-layer RPS signalling and derivation
  
  • 
    N0059, N0081, N0118, N0154, N0195, N0217, N0131
    
• 
  Portions of 6.4.3.1 General
  
  • 
    N0130, N0195, N0217
    
• 
  6.4.4.6 Management of resampled or filtered inter-layer reference pictures
  
  • 
    N0128, N0282, N0056
    

Decision: The JCT-VC endorsed the BoG recommendations of the following:

1. 
   Adopt from JCTVC-N0057 second proposal, change decoding process to add the condition that when SamplePredEnabledFlag equal to 1, don't include picture in the motion pred ref list.
   
2. 
   Adopt a condition on signalling inter_layer_pred_layer_idc[ i ], to avoid sending when NumDirectRefLayers equals NumActiveRefLayerPics, and instead infer values. From JCTVC-N0081, JCTVC-N0195 proposal 1, JCTVC-N0154, and JCTVC-N0217.
   
3. 
   Adopt an Inter Layer Reference Picture (ILRP) presence flag in the VPS, conditioning the presence of ILRP syntax elements in the slice segment header, similar to JCTVC-N0195 proposal 2.
   
4. 
   Add a constraint when splitting_flag is used, that the sum of the lengths be less than or equal to 6, from JCTVC-N0195 5^th proposal.
   

Decision (Ed): The JCT-VC endorsed the BoG recommendation that the editors consider the following:

1. 
   A text bug fix identified in JCTVC-N0059, in which motion resampling is currently only invoked when alt_collocated_idc flag is equal to 1, while it should be invoked whenever inter-layer motion prediction is performed and the current and reference layer differ in resolution.
   
2. 
   Add an editorial note for SHVC encoders to avoid use of TMVP when the inter-layer reference picture is the only one in the list.
   

1. 
   JCTVC-N0128 Proposal 2 be reviewed in the track, as the topic was difficult to address in the BoG. This topic is further discussed above, and AHG study was planned on HRD-related topics.
   
2. 
   The Track further discuss the concept of sending IL RPS candidates in VPS with slice indication, as in JCTVC-N0118and JCTVC-N0081. Regarding JCTVC-N0118, it was commented that this is just about saving bits. Another participant indicated that this was not for use in common cases. The amount of potential savings was unclear, so the topic was agreed to be for further study.
   

The BoG also met again on 30 July to review the following contributions

• 
  6.4.4.5 Reference picture list construction (4)
  
  • 
    Revisit of JCTVC-N0316
    
• 
  6.4.4.2 Sub-layer related inter-layer prediction signalling (4)
  
  • 
    Revisits of JCTVC-N0120, N0109, N0196
    
• 
  6.4.5 Tiles and parallel processing (4)
  
  • 
    JCTVC-N0158, JCTVC-N0160, JCTVC-N0199
    

1. 
   Adopt a presence flag for signalling of max_tid_il_ref_pics_plus1[i], from JCTVC-N0120.
   
2. 
   Move num_ilp_restricted_ref_layers and related offset delay syntax elements from SPS VUI to VPS VUI, and change to a num_ilp_restricted_ref_layers flag per direct dependent layer for each layer, from JCTVC-N0160.
   
3. 
   Modify calculation of xRef[ ] and yRev[ ] variables in semantics for offset delay syntax element semantics, from JCTVC-N0160
   
4. 
   Adopt JCTVC-N0199 proposal 2 variant 2 (also in JCTVC-N0160) to move tile boundaries alignment flag from the SPS VUI to the VPS VUI, and also signal the flag per direct dependent layer for each layer, from JCTVC-N0199.
   

The BoG met again on 31 July to cover the following topics:

• 
  6.6.1 Motion and prediction constrained SEI messages (5)
  

• 
  JCTVC-N0088, JCTVC-N0117, JCTVC-N0236
  
• 
  Revisits of JCTVC-N0069, JCTVC-N0159
  
• 
  6.2.5 Sampling position
  
  • 
    Revisit of JCTVC-N0334
    

Decision: The JCT-VC endorsed the BoG recommendation of the following:

1. 
   Add inter-layer prediction tile constraint indication to the motion constrained tile information SEI message. Exact syntax to be provided in new contribution JCTVC-N0383, which harmonizes JCTVC-N0069, JCTVC-N0087, JCTVC-N0088, and JCTVC-N0236.
   

• 
  6.4.1 Generic HLS issues
  
  • 
    JCTVC-N0355, JCTVC-N0356, JCTVC-N0357, JCTVC-N0135, JCTVC-N0065, JCTVC-N0244
    
• 
  6.4.2 Random access, layer switching structures and cross-layer alignment of pictures types
  
  • 
    JCTVC-N0066, JCTVC-N0147
    
• 
  6.5.1 SHVC Generic HLS issues
  
  • 
    JCTVC-N0108
    
• 
  6.6.1 Motion and prediction constrained SEI messages
  
  • 
    JCTVC-N0383
    

1. 
   Adopt JCTVC-N0244, to use a reserved slice header bit for a POC reset flag, plus signal POC LSB in enhancement layer IRAP pictures from JCTVC-N0065, to maintain POC alignment between layers when IRAP pictures are not aligned.
   
   • 
     Clarifying note: The bit is not necessary to be present when alignment is established per item 3 below.
     
2. 
   Adopt JCTVC-N0066 version 2 layer-wise start up decoding process.
   
3. 
   Add a flag in VPS extension to indicate if all IRAP pictures are aligned in set of dependent layers, from JCTVC-N0147.
   
4. 
   Add explicit constraint that sample resampling may be done once per enhancement layer picture, and motion field resampling may be done once per enhancement layer picture, from JCTVC-N0108.
   
   • 
     The BoG noted that the SHVC track may also wish to discuss whether sample prediction and motion field prediction from different reference layers can be performed when coding an enhancement layer picture in the Scalable Main profile.
     
   • 
     Refinement agreed by JCT-VC: Establish a profile constraint not to allow sample prediction from a different inter-layer reference picture than motion field prediction.
     
5. 
   Adopt JCTVC-N0383 inter-layer predication constraint SEI message, plus modification to motion constrained tile set SEI message to align syntax.
   

15.0.0.1.1.1.1.1.357JCTVC-N0135 MV-HEVC/SHVC HLS: Extended maximum number of layers [B. Choi, Y. Cho, M. W. Park, J. Y. Lee, H. Wey, C. Kim (Samsung)] [late]

(Reviewed Thu. 25th plenary)

This is a follow-up proposal of JCTVC-M0164. To support having more than 64 layers, proposes to use three of the reserved bits in slice headers for extra bits of nuh_layer_id. Approximately 500 layers can be represented with extra 3 bits of nuh_layer_id.

Primarily motivated by "super multiview" application for displays with many views.

Question: Would such an application use one layer per view?

Suggestion to put additional bits into slice header, further investigation necessary whether this is a good place. The current proposal suggests 15 additional bits for layer_id, which according to some experts' opinion might be excessive.

It was remarked that the proposed syntax requires parsing the PPS before being able to access the bits of the extended layer ID, which may be undesirable.

It was noted that we have substantial syntax freedom for non-base layers, although we are constrained by compatibility for layer ID zero.

Near-term profiles likely will not need to support many layers, but it is important to provide extensibility in terms of the number of layers. Some way of allowing extension is highly desirable. Compatibility with existing decoders would be desirable, e.g. for decoding a subset of views.

Another possibility would be to assign one value of layer_id as “reserved for extension”.

Further offline discussion about best way to achieve this

Not clear whether there is any need for immediate action w.r.t. the MV-HEVC draft – likely not.

It was remarked that JCT3V-E0092, JCT3V-E0223 and JCT3V-E0224 are related (not submitted as JCT-VC contributions), where it is suggested to put additional bits into parameter sets. These proposals will also be registered as JCT-VC docs, and were discussed in the context of generic HLS issues.

See BoG report N0374 and related notes.

15.0.0.1.1.1.1.1.358JCTVC-N0355 / JCT3V-E0092 3D/MV-HEVC HLS: Extending the supported number of layers [K. Suehring, G. Tech, R. Skupin, T. Schierl (FhG HHI)] [late]

(Reviewed Thu. 25th plenary)

This contribution proposes an extension mechanism for layer identifiers to increase the number of supported layers in MV-HEVC and 3D-HEVC. The range of nuh_layer_id is extended by an additional syntax element within the NAL units. The concept of so-called layer clusters allows using the existing extraction processes to select groups of related layers as proposed during the 4th meeting. The syntax has been modified slightly to ensure a backward compatible base layer and to be aligned with MV-HEVC Draft 4.

Again focused on "super multiview" with many views.

Proposes to signal, in VPS, a number of extra bits of layer ID.

It was commented that just having reserved fields might be adequate from a syntax perspective.

It was commented that some form of syntax involving "if( layer_id != 0)" branch in the syntax could be used.

It was commented that using layer ID equal to 63 as an escape code indication could be an alternative way to deal with the layer ID range.

Having an extended NUH for use with some particular profile was also discussed.

It was remarked that having a view-subset decoding capability for a lower-capability decoder is desirable. The proponent suggested that having a "clustering" of views to indicate which subset to decode is also desirable.

It was generaly agreed that support for some extensibility to more views with subset capabilities would be desirable, but we don't want to burden "mainstream" decoders with significant extra work to accomplish that, and we don't want the standard to contain purely "hypothetical syntax" that is normatively forbidden to be used.

See BoG report N0374 and related notes.

15.0.0.1.1.1.1.1.359JCTVC-N0244 / JCT3V-E0075 MV-HEVC/SHVC HLS: Cross-layer POC alignment [Y. Chen, Y.-K. Wang, A. K. Ramasubramonian (Qualcomm)]

(Reviewed Thu. 25th plenary)

In this proposal, a mechanism is proposed in order to ensure that the POC values all pictures of each access unit are the same even when it is allowed that access units for some pictures are IRAP pictures with NoRaslOutputFlag equal to 1 while others are not. Draft text was provided.

The contribution proposes a "poc_reset_flag" syntax flag.

When set to 1, the flag changes the POC of the previously-decoded pictures of the same layer in the DPB, by subtracting an offset from their POC values. It was remarked that this has loss resilience implications (when the picture which causes the POC reset is lost), and further study of that aspect was encouraged (e.g. to add some SEI message to improve the detection and handling of lost pictures).

Base layer decoder would work different from a single-layer (version 1) decoder (which is likely not critical).

The handling of long term pictures was discussed, and this was claimed to be solved.

See BoG report N0374 and related notes.

15.0.0.1.1.1.1.1.360JCTVC-N0356 / JCT3V-E0223 3D/MV-HEVC HLS: Dependency signalling for extending the supported number of layers [G. Tech, K. Suehring, R. Skupin, Y. Sanchez, T. Schierl (HHI)] [late]

Related to JCTVC-N0355 / JCT3V-E0092, considered in JCT-3V.

See also BoG report N0374 and related notes.

15.0.0.1.1.1.1.1.361JCTVC-N0357 / JCT3V-E0224 3D/MV-HEVC HLS: Flexible layer clustering for extending the supported number of layers [G. Tech, K. Suehring, R. Skupin, Y. Sanchez, T. Schierl (HHI)] [late]

Related to JCTVC-N0355 / JCT3V-E0092, considered in JCT-3V.

See also BoG report N0374 and related notes.

15.0.0.1.1.1.1.1.362JCTVC-N0267 / JCT3V-E0087 MV-HEVC/SHVC HLS: On changing of the highest layer ID across AUs and multi-mode bitstream extraction [Y.-K. Wang, Y. Chen (Qualcomm)]

(Reviewed Thu. 25th plenary)

This document discusses allowing changing of the highest value of nuh_layer_id across AUs within a CVS (which is currently allowed, due to an adoption intended to allow ARC), and proposes a multi-mode bitstream extraction process. The contribution raises a number of issues relating to this topic.

Related to N0110.

Question: Does “higher layer” usually mean equal or higher resolution? Currently, yes. However, in the case of using the “single_layer_for_non_irap_flag” it might also make sense to allow prediction of lower resolution from higher resolution, which would require defining decimation filters for prediction.

See BoG report N0374 and related notes.

2. 
   
   Directory: sc29 -> open -> 29view
   29view -> International organisation for standardisation organisation internationale de normalisation
   29view -> International organisation for standardisation organisation internationale de normalisation
   
   
   
   Download 8.47 Mb.
   
   Share with your friends: