International organisation for standardisation organisation internationale de normalisation
Task group activities Joint Meetings
Download 8.47 Mb.
|
- Navigate this page:
- With Systems on MP4FF Enhanced Audio Support Wed 14:00-16:00
- With Requirements on DRC Fri 11:00 – 11:30
- Task Group discussions
- MPEG Surround
Task group activities
At the 104th MPEG meeting the 3DG Chair summarized the needs of an Audio Augmented Reality system as follows:
The 3DG Chair clarified that less than 50 ms is a realistic requirement for 3D audio latency. The 3DG Chair gave an example of Augmented Reality: a person has a mobile device with a transparent visual display. The person sees and hears the real world around him, and can look through the transparent visual display (i.e. “see through mode) to see the augmented reality (e.g. with an avatar rendered in the scene). The avatar has a pre-coded audio stream (e.g. MPEG-4 AAC bitstream) that it can play out, and ARAF knows its spatial location and orientation (i.e. which way it is facing). The required the audio metadata is:
The avatar audio could be presented via headphones or ear buds. The Audio Chair noted that ARAF may want to incorporate a head tracker so that the avatar can remain fixed within the physical world coordinate system. The 3DG Chair noted that if “see through” mode is used then the orientation of the mobile device screen would be sufficient. In that case, audio could be presented via the mobile device speakers. The Audio Chair noted that the avatar could function as a “virtual loudspeaker” in MPEG-H 3D Audio such that the 3D Audio renderer could be used for presentation via headphones. However, 3D Audio is not able to add environmental effects (e.g. room reverberation) that are separate from the avatar sound. Furthermore, 3D Audio cannot add such environmental effects based on listener location or orientation (e.g. reverberation changes when the listener moved closer to a reflecting surface). Yeshwant Muthusamy, Samsung, noted that KHRONOS and OpenSELS provide a 3D Audio API that does support e.g. reverberation based on user location and might offer a solution. However, the Audio Chair noted that ARAF does not know the acoustic properties (i.e. surface reflective properties) of the real-world environment (unless it can deduce them from the visual scene) and thus audio effects based on real-world environment is not possible, whether using MPEG-H 3D Audio or KHRONOS. 3DG experts will consider this problem and report back at a future joint meeting.
The following documents were reviewed:
It was decided to remove audio-related technology from these documents and put them in a working draft that can be better managed by Audio and can progress on an independent time line.
Audio experts are well aware that there is DRC capability integrated with the MPEG-4 Advanced Audio Coding profile coder, as in the AAC MDCT-based multi-band DRC. Audio experts do not want to needlessly confuse the marketplace, but are nevertheless interested in considering new technology having performance needed for diversity of the marketplace (e.g. listening in the living room home theatre or in a mobile phone on the street). While many actions concerning Dynamic Range Control were discussed during the week, the Audio subgroup decided to issue a Call for Proposals on Program Level and Dynamic Range Control. Submissions are asked to compare the performance of proposed technology to that of MPEG-4 AAC-based DRC technology. The Call will be issued at this meeting and submissions due at the next meeting. The text of the Call was reviewed in this joint session and approved by Requirements.
The proponent binaurization complexity figures were reviewed and agreed to be as follows:
Calculation of Figure of Merit Using the binauralization complexity figures shown above, the Figure of Merit was calculated for CfP responses using the CO and HOA data sets. Audio experts checked the FoM (and revised and corrected) and the results are shown here: Channel/Object (CO) Signal Set
Higher Order Ambisonics (HOA) signal set
Discussion The FoM data suggests a “best” system for each signal set. For these “best” systems, Audio experts looked at overall system performance and also performance for individual items in individual tests and did not find evidence of inconsistent performance (i.e. exceptionally poor performance for one or more individual items). No such issues were identified. For the Channel/Object (CO) signal set, it was the consensus of the Audio subgroup to select Sys1, which is the FhG-IIS submission, as the CO RM0. For the Higher Order Ambisonics (HOA) signal set, it was the consensus of the Audio subgroup to select Sys2, which is the Technicolor submission, as the HOA RM0. The Chair noted that the Technicolor submission is joint work between Technicolor and Orange. If was agreed to define a first Core Experiment with a workplan at this meeting to investigate merging HOA Sys1 technology, submitted by Qualcomm, into the HOA RM0. As specified in the Call for Proposals, it is expected that proponents submit RM0, as decoder technical specification and decoder source code implementation, prior to the beginning of the next MPEG meeting (see the Call for details). 3D Audio Requirements Akio Ando, NHK, presented
The contribution responds to a WG11 request at the 104th MPEG meeting. In May, 2013 the Next Generation TV Forum (NexTV-F) was establish to promote 8K-resolution television broadcast. They have committed to 8K broadcasting in 2016. The requirements from NHK are: (4K broadcasting?) Service:
Performance:
Functional:
Year 2016 MPEG-4 based model
SHV broadcast schedule (to be promoted by NexTV-F)
The Chair stated that MPEG Audio could commit to delivering an MPEG-H 3D Audio Version 1 that can be used for both experimental and examination broadcast. However, he asked NHK experts if it will be easy to move to MPEG-H 3D Audio Version 2 for 2020 full-service broadcast. NHK experts were not able answer the question at this time, but indicated that Japanese broadcasters might want to use the newest technology. Thomas Sporer, FhG-IDMT, noted that ITU-R Recommendation BS.1548 defines “broadcast quality” The Chair encouraged Audio delegates to reflect on the issues raised and additionally to think of the content of these possible output documents for MPEG-H Version 1: Public:
Private:
Jan Plogsites, FhG-IIS, presented
The contribution presented timeline requirements for ARIB, ATSC 3.0, both of which appear to be well served by an MPEG-H 3D Audio Version 1 International Standard in January 2015. A timeline for the European marketplace is less clear. Candidate Requirements
In summary, FhG-IIS feels that it is important to adhere to an aggressive timeline for Version 1 to address the needs of the worldwide marketplace. In this respect, a clear set of requirements (of what is in and what is not) for Version 1 facilitates meeting this aggressive timeline. Thomas Sporer, FhG-IDMT, presented
The document is a joint contribution from FhG-IDMT and Iosono. The presenter noted that many requirements in this contribution have already be covered in the previous two contributions. The proposed requirements are:
The contribution urges MPEG to make a careful technology selection, particularly for Version 1, so that it is not locked into old technology. In addition, there must be a clear plan e.g. requirements, bitstream architecture to move from Version 1 to Version 2. Andreas Silzle, FhG-IIS, presented
The presenter noted that downmix is a very important functionality in MPEG-H 3D Audio. The fact that one might downmix from 22 channels to 5 or even 2 channels gives ample opportunity for phase alignment of correlated signal peaks and hence large values that lead to signal clipping. Downmix may effect
And dowmix may result in
To avoid these problems, downmix may need to be dynamic, rather than static (over the entire audio program). Dynamic downmix might be both signal-dependent and also frequency-variant. The presentation gave an example spectrogram of a static downmix as compared to an adaptive downmix, in which the static downmix demonstrated deep comb-filtering at 1 kHz intervals. Such comb-filtering is expected to be prevelant in 22.2 channel programs because a mix of high, mid, front with a strong discrete source would have a very small time delay, leading to comb-filtering. Gregory Pallone, Orang Labs, asked if the FhG active downmix used delay compensation (i.e. some aspect of phase in addition to gain) and the presenter stated that the mixer was signal-adaptive, but worked in real-time. He further noted that HOA signals do not have such a downmix problem.
Marc Emerit, Orange, presented
The contribution notes that a format for interfacing head-related transfer functions is currently being developed as an AES standard under project AES-X212 in AESSC working group AES SC-08-02 (www.aes.org/standards). The contribution brings information on the Spatially Oriented Format for Acoustics (SOFA) format, which aims to store space-related audio data in a very general way. It supports a very rich description of the measurement conditions for a HRTF/BRIR, including listener position, emitter position, and IP rights comments. It also supports multiple representations for the HRTF/BRIR data. The contribution recommends that the AES-X212 work be considered for possible adoption by MPEG-H 3D Audio as an interchange format. It is expected that the AES standard will issue in 2014. Thomas Sporer, FhG-IDMT, noted that AES work sometimes progresses slowly and so the 2014 date could be later. The presenter noted that the first format for AES-X212 would likely be PCM BRIR waveforms. The Chair endorsed the idea of a simple BRIR interchange format, with a “second step” of converting the BRIR to a possible application-specific formats. Werner Oomen, Philips, supported many formats in a single interchange format. He noted that there are two formats for binaurization information in MPEG Surround. The Chair noted that the group should consider the impact of high-resolution BRIR on a possible interchange format. Werner de Bruijn, Philips, presented
The contribution presented a revision to a previous document (m29145) on BRIR format. It notes that there is a wide variation in the ways that BRIR are measured, represented and possibly parameterized. At the most basic level, this includes e.g. representation of angles associated with individual BRIR within a set. The contribution describes a software tool that can convert from BRIR time-domain representations to other formats. Philips experts have contacted the AES SC-08-02 working group and shared with them this format proposal. The contribution asks that the format be included in 3D Audio RM0. Clemens Par, Swissaudec, noted that the he believes that AES envisions a “scalable” standard which could result a simple format aimed at industrial applications and a more complex format aimed at research applications. The Chair noted that the AES X212 work is just beginning, and MPEG is in a position to influence the AES work. What that will might be need more discussion. BRIR Study Thomas Sporer, FhG-IDMT, presented
This contribution describes a study comparing codecs in headphone listening tests using different sets of BRIRs. The results show that the influence of the BRIR set is sometimes an important factor. NHK Presentation on 2016 Broadcast Model Akio Ando, NHK, made a presentation on the NHK view on Japan on 2016 Broadcast Model on 22.2 channel broadcast. The timeline is:
The presenter highlighted what is missing in MPEG-4:
The presenter described the 2016 Broadcast Model MPEG-4 AAC decoder
MPEG-H 3D Audio Renderer/Downmix
The Chair asked what is 2015 experimental model? The presenter stated that the near-term work would focus on 22.2 channel encoder/decoder, and downmix may not be considered. The Chair asked about DRC – is it required that MPEG produce standardize DRC technology on a certain timeline? The presenter stated that DRC is not essential in the 2016 broadcast model. If it is available it may be used. Thomas Sporer, FhG-IDMT, noted that NHK asks for default and artistic downmix coefficients. The Chair noted that this could be re-expressed as:
There are two possibilities moving forward: Model 1
Model 2
Model 1 is possible if MPEG-H 3D Audio is DIS (i.e. frozen technology) in July 2015. Thursday Discussion The Chair noted that the IIS submission is based on a USAC core coder with extensions, and asked for comments on whether Audio experts want:
After discussion it was the consensus of the Audio subgroup to:
ETRI requested to conduct a CE using only the CfP decoded waveforms. It would compare the subjective quality of:
Several additional companies requested to participate in the CE. The Audio subgroup decided that the following process would be used to manage access to the RM0-CO decoded waveforms in the AhG prior to the next meeting (at which time the full RM0 deliverables will be provided by the RM proponents, i.e. bitstreams, decoded waveforms and decoder source code and WD text). Process for access to RM0 decoded waveforms:
Daniel Fischer, FhG-IIS, presented
The presentation describes a error in the software when operating in 7-2-5 mode. A fix was described. It was the consensus of the Audio Subgroup to add this correction the existing Defect Report on MPEG Surround text. Daniel Fischer, FhG-IIS, presented
The presentation describes a error in the SAOC software which contains low delay MPEG Surround. The fix aligns the software with the standard text. It was the consensus of the Audio Subgroup to put this correction into a new Defect Report on SAOC Reference Software.
Oliver Hellmuth, FhG-IIS, presented
The contribution identifies errors in the text specification and proposed corrections. It was the consensus of the Audio Subgroup to add these corrections to the existing Defect Report on SAOC and to issue a DCOR 2. Oliver Hellmuth, FhG-IIS, presented
The contribution progresses the WD on SAOC-DE to
Henney Oh, WILUS STRC, requested time to study the contribution and discuss it with the presenter. The topic will be brought up later in the week. After additional discussion, it was the consensus of the Audio Subgroup to progress this work to PDAM. The Chair noted that the presenter should prepare a Request for Amendment document. |