5th etsi speech Quality Test Event Anonymous Test Report


IP Gateway – Conversational Aspects



Download 0.63 Mb.
Page6/13
Date06.08.2017
Size0.63 Mb.
#27239
1   2   3   4   5   6   7   8   9   ...   13

IP Gateway – Conversational Aspects


The following analyses of transmission parameters address the one way transmission and conversational aspects. The main focus is on echo performance and echo canceller implementation, the double talk performance and the various aspects concerning quality of background noise transmission. In order to most efficiently evaluate the implemented signal processing like echo canceller, NLP and echo suppression with its acoustical CN injection, VAD and CN generation these tests were carried out using the G.711 speech codec (a-law).

The same tests as carried out for the VoIP gateway have been carried out over the reference connection without any gateway or echo canceller involved (echo path with infinite ERL or 40 dB ERL). These results demonstrate the test conditions during the test event and can be seen as references. For better comparison some of these results are also shown below.

The analyses shown in this report below are subdivided in


  • One-way parameter analyses evaluating the implemented signal processing like VAD, packet loss concealment, background noise transmission and speech quality in the presence of background noise (chapter 5.1)

  • Echo performance tests with infinite ERL in the echo path covering echo attenuation, echo attenuation vs. time, double talk performance and modulation of background noise caused by NLP and CN injection (chapter 5.2)

  • Echo performance tests simulating different cordless telephones in the echo path (DECT phones, chapter 5.3)

  • Echo performance tests with a low ERL of 6 dB in the echo path (echo attenuation, echo attenuation vs. time, double talk performance and modulation of background noise caused by NLP and CN injection, chapter 5.4).

During the 5th SQTE further tests were carried out using a simulation of a 40 dB ERL echo path. The results are all documented in the individual test reports. However, in order to limit the volume of the report the analyses are skipped in this Anonymous Test Report.

3.5One-way Parameter Analysis

3.5.1Gateway No.1


“Evaluation Module”






JLR

Delay

Sending Direction

-0.2 dB

42.5 ms

Receiving Direction

-0.2 dB

43.5 ms

The measured delay is low and both transmission directions are transparent (JLR close to 0 dB).

The sending direction is transparently activated, the initial clipping introduced by VAD is negligible (fig. 5.2).



Packet loss concealment works reliable and leads to low residual disturbances. The phase is properly interpolated (fig. 5.3). The listening speech quality scores are high (see Quality Pie).







Fig. 5.1: Activation in sending direction (VAD off)

Fig. 5.2: Activation in sending direction (VAD on)

Fig. 5.3: 5% PL, 0ms jitter







Fig. 5.4: Café (VAD on)

Fig. 5.5: Δ Relative Approach Café (VAD on)

Fig. 5.6: Organ (VAD on)



The activation threshold of VAD is low. Background noises are transmitted without noticeable degradation. The Δ RA analysis in fig. 5.5 does not detect unexpected artifacts

The organ test signal is also transparently transmitted. The test signal level exceeds the VAD activation threshold. Comfort noise is not generated.

The speech quality scores in the presence of background noise (S-, N- and G-MOS according to ETSI EG 202 396-3) are in the expected range for gateways providing a transparent connection (see Quality Pie).

3.5.2Gateway No.2


“Complete System”






JLR

Delay

Sending Direction

0.0 dB

71.3 ms

Receiving Direction

0.2 dB

76.4 ms

The measured delay is in the expected range. The digital connection between the two gateways is transparent in both directions (JLR close to 0 dB).

VAD does not introduce clipping during the tests. The short term gaps analyzed in fig. 5.8 are caused by a delay drift during the test and practically negligible.



The listening speech quality is high (see Quality Pie). Packet loss is reliably considered by the implemented PLC The residual disturbances are low (fig. 5.9). The phase is properly interpolated.







Fig. 5.7: Activation in sending direction (VAD off)

Fig. 5.8: Activation in sending direction (VAD on)

Fig. 5.9: 5% PL, 0ms jitter







Fig. 5.10: Café (VAD on)

Fig. 5.11: Δ Relative Approach Café (VAD on)

Fig. 5.12: Organ (VAD on)



The gateway transmitted all background noise signals completely during testing. The homogeneous color in the Δ RA analysis in fig. 5.11 indicates an undisturbed connection.

The organ test signal is also transparently transmitted. The test signal level exceeds the VAD activation threshold. Comfort noise is not generated.

The Quality Pie representing the speech quality scores in the presence of background noise (S-, N- and G-MOS according to ETSI EG 202 396-3) indicates an expected high quality for a transparent connection.

3.5.3Gateway No.3


“Evaluation Module”






JLR

Delay

Sending Direction

-0.2 dB

42.5 ms

Receiving Direction

-0.2 dB

43.5 ms

The measured delay is low. The JLR indicate a transparent connection in both directions.

The signal bursts activate VAD during the tests and are nearly completely transmitted except a very short (and therefore unnoticeable) initial clipping (fig. 5.14).



Packet loss is considered by PLC. The implementation works reliable, the phase shifts are properly interpolated. The listening speech quality scores are high (see Quality Pie).







Fig. 5.13: Activation in sending direction (VAD off)

Fig. 5.14: Activation in sending direction (VAD on)

Fig. 5.15: 5% PL, 0ms jitter







Fig. 5.16: Café (VAD on)

Fig. 5.17: Δ Relative Approach Café (VAD on)

Fig. 5.18: Organ (VAD on)



The background noise levels exceed the VAD threshold. The test signals are transmitted without noticeable degradation. The Δ RA analysis in fig. 5.17 does not detect unexpected disturbances

The organ test signal is also completely transmitted without disturbances (see fig. 5.18).

The S-, N- and G-MOS results according to ETSI EG 202 396-3 are in the expected range for a transparent gateway connection. (see Quality Pie).



3.5.4 Gateway No.4


“Complete System”






JLR

Delay

Sending Direction

0.1 dB

70.9 ms

Receiving Direction

0.2 dB

70.7 ms

The gateway connection is transparent. The measured delays are in the expected range for a complete system.

The VAD tests required more detailed investigations during the test day in order to verify VAD settings. Some tests were therefore skipped.



The quality of implemented packet loss concealment is high. The RA analysis in the middle window of fig. 5.21 as well as the cross correlation analysis indicates a very good performance.



Test skipped



Fig. 5.19: Activation in sending direction (VAD off)

Fig. 5.20: Activation in sending direction (VAD on)

Fig. 5.21: 5% PL, 0ms jitter







Fig. 5.22: Café (VAD on)

Fig. 5.23: Δ Relative Approach Café (VAD on)

Fig. 5.24: Organ (VAD on)



The different background scenarios used for testing indicate a transparent connection between the gateways. The Δ RA analysis in fig. 5.23 does not detect unexpected or missing components. The spectrogram of the organ signal also indicates some noisy components when the signal characteristic changes from the real organ to the tone sequence.

The Quality Pie indicating the speech quality scores in the presence of background noise (S-, N- and G-MOS according to ETSI EG 202 396-3) are in the expected range for a transparent gateway connection (see Quality Pie).




3.5.5Gateway No.5


“Complete System”






JLR

Delay

Sending Direction

0.2 dB

89.9 ms

Receiving Direction

0.0 dB

101.5 ms

The measured delay is relatively high in both directions. The digital connection between the two gateways is transparent (JLR close to 0 dB). The G.729 speech coder was not tested.

VAD introduces slight initial clipping during the tests. The gaps a short and practically negligible.



The listening speech quality is for the G.711 speech coder is lower than the average scores from previous events (see Quality Pie). The detailed analysis in fig. 5.27 indicates that packet loss leads to strong disturbances and phase shifts in the transmitted signal.

Test skipped





Fig. 5.25: Activation in sending direction (VAD off)

Fig. 5.26: Activation in sending direction (VAD on)

Fig. 5.27: 5% PL, 0ms jitter







Fig. 5.28: Café (VAD on)

Fig. 5.29: Δ Relative Approach Café (VAD on)

Fig. 5.30: Organ (VAD on)



The activation threshold of VAD is low. Background noises are transmitted without some slight degradation. The peaks in the Δ RA analysis in fig. 5.29 indicate short term artifacts.

The organ test signal level is higher than any activation threshold. The signal is completely transmitted.



The speech quality scores in the presence of background noise (S-, N- and G-MOS according to ETSI EG 202 396-3) are in the expected range for an electrical connection between two gateways (see Quality Pie).

3.5.6Gateway No.6


“Complete System”






JLR

Delay

Sending Direction

6.1 dB

80.5 ms

Receiving Direction

8.7 dB

65.9 ms

The system was tested via analogue POTS interface. The delay is in the expected range. The sending sensitivity is slightly lower than required for many European countries (acc. to ETS 300 439), the receiving sensitivity is in the expected range. VAD functionality is not provided.

The listening speech quality does not exceed the average scores. Packet loss concealment leads to strong residual disturbances in the low frequency range (fig. 5.33).



No VAD
functionality available




Fig. 5.31: Activation in sending direction (VAD off)

Fig. 5.32: Activation in sending direction (VAD on)

Fig. 5.33: 5% PL, 0ms jitter







Fig. 5.34: Café (VAD off)

Fig. 5.35: Δ Relative Approach Café (VAD off)

Fig. 5.36: Organ (VAD off)



The different background scenarios signals are completely transmitted. The offset in fig. 5.34 is caused by the analoge sensitivities. The Δ RA analysis in fig. 5.35 does not detect unexpected disturbances. The organ signal is also completely transmitted.

The Quality Pie showing the S-, N- and G-MOS according to ETSI EG 202 396-3 also reflects the transparent transmission. Note that the level offset caused by the analogue sensitivities does not influence the quality of transmitted signals.




3.5.7Gateway No.7


“Complete System”






JLR

Delay

Sending Direction

0.0 dB

77.8 ms

Receiving Direction

0.8 dB

67.6 ms

The system was tested via analogue POTS interface. The delay is in the expected range. The sensitivities for the analogue connection are free to configure and were adjusted close to 0 dB. VAD threshold is low, transmitted test signals are not clipped (fig. 5.38).

The listening speech quality is very low under jitter influence. A jitter buffer was not activated. Packet loss concealment is based on a repetition of previous frames without phase interpolation (fig. 5.39). The low frequency disturbances are detected by the RA analysis in fig. 5.39.







Fig. 5.37: Activation in sending direction (VAD off)

Fig. 5.38: Activation in sending direction (VAD on)

Fig. 5.39: 5% PL, 0ms jitter







Fig. 5.40: Café (VAD on)

Fig. 5.41: Δ Relative Approach Café (VAD on)

Fig. 5.42: Organ (VAD on)



The background noise levels exceed the VAD threshold. The test signals are completely transmitted. The Δ RA analysis in fig. 5.17 detects very short peaks which are practically not noticeable.

The organ test signal is also completely transmitted without disturbances (see fig. 5.42).



The S-, N- and G-MOS results according to ETSI EG 202 396-3 are in the expected range for a transparent gateway connection (see Quality Pie). Packet or jitter is not inserted during these tests.



Download 0.63 Mb.

Share with your friends:
1   2   3   4   5   6   7   8   9   ...   13




The database is protected by copyright ©ininet.org 2024
send message

    Main page