Conversion of a structured annotation to a Praat TextGrid

Although tiers whose times are predictable are structurally and implicitly coded in Emu, it is nevertheless possible to convert most Emu annotations into a Praat TextGrid in which times are explicitly represented for every annotation and for every tier. For example, the equivalent TextGrid representation for the utterance that has just been annotated is shown in Fig. 4.9: in this TextGrid, the annotations of the tiers that were declared to stand in a hierarchical relationship to each other in Emu have identical times at boundaries, as Fig. 4.9 shows. But not every Emu annotation structure can have a corresponding representation as a Praat TextGrid. In particular, it is possible to have annotations in Emu that are completely timeless even after annotations have percolated up through the tree, in the manner described earlier. An example of this occurs in the downloadable kielread corpus in Fig. 4.10 in which the Kanonic tier (used for citation-form, dictionary pronunciations) stands in a hierarchical relationship to the segment tier Phonetic from which it therefore inherits its times. However, since some annotations at the Kanonic tier are not linked to the Phonetic tier, then the times cannot be passed up the tree to them and so they remain timeless.

This purpose of these unlinked annotations is to express segment deletion. Thus the citation-form, isolated word production of und (and) has a final /t/, but in the read speech form that actually occurred in this utterance, the /t/ appears to have been deleted, as both the spectrogram Fig. 4.10 and listening to the utterance suggest. This mismatch between the citation-form representation and what was actually spoken is given expression by representing this final /t/ at the Kanonic tier as being unassociated with any time. The advantage of this representation is that the user can subsequently compare analogous contexts that differ according to whether or not segment deletion has taken place. For example, although based on a similar spectrographic/auditory impression the final /t/ in past six may appear to have been deleted (thereby rendering the first word apparently homophonous with pass), a more rigorous follow-up analysis may well show that there were, after all, fine phonetic cues that distinguished the long [s] across the word boundary in past six from its occurrence in pass six. But in order to carry out such an analysis, it would be necessary to find in the corpus examples of [s] that do and do not precede a deleted /t/ and this will only be possible if the deletion is explicitly encoded at this more abstract level of representation, as has been done for the /t/ of und, the /ə/ of schreiben (to write) and the final /ən/ of lernen (to learn) in this utterance in the kielread corpus. Now since annotations in Praat must be explicitly associated with times, then these kinds of timeless segments that are used to express the possibility of segment deletion will not appear in the Praat TextGrid, when the conversion is carried out in the manner shown earlier in Fig. 4.9 (and a warning message is given to this effect).
Fig. 4.9 about here

Fig. 4.10 about here

The types of queries that have been discussed so far can be combined for more complex queries of structured annotations. A full review of the query syntax is beyond the scope of this Chapter but is summarized in Cassidy & Harrington (2001) and set out in further detail in appendix A on the website associated with this book. An example is given here of a complex query that makes use of the additional functionality for searching by position and number: 'find all L*+H pitch-accented words that are in intermediate-phrase-final position and in H% intonational phrases of at least 5 words and such that the target words precede another word with a break index of 0 in an L- intermediate phrase'. Such a query should find the word Thorsten in the gt database (Fig. 4.3) because Thorsten:

• 
  is the last word in an intermediate phrase
  
• 
  is associated with an L*+H pitch-accent
  
• 
  occurs in an H% intonational phrase of at least five words
  
• 
  precedes another word (ja) whose break index is 0 and which occurs in an L- intermediate phrase
  

The graphical user interface to the Emu query language, which was written by Tina John, can be of great assistance in calculating complex queries such as these. This GUI (Fig. 4.11) is opened by clicking on the graphical query button in the Query Tool window: for the gt database, this action brings up a form of spreadsheet with the tiers arranged from top to bottom according to the database's template file. It is a comparatively straightforward matter to enter the search criteria into this window in the manner shown in Fig. 4.11. The search instruction is automatically copied into the Query Tool window after clicking the Query button. You could also copy the calculated search instruction and then enter it as the third argument of the emu.query() function in R thus:

labels start end utts

1 Thorsten 1573.73 2086.38 thorsten
Fig. 4.11 about here
4.7 Re-querying segment lists

The emu.requery() function in the Emu-R library allows existing segment lists to be queried for position or non-linearly. This is useful if, say, you have made a segment list of words and subsequently want to find out the type of intermediate phrase in which they occurred, or the annotations that followed them, or the pitch-accents with which they were associated. The emu.requery() function can be used for this purpose and it takes the following three mandatory arguments:

• 
  a segment list
  
• 
  the tier of the segments in the segment list
  
• 
  the tier of the desired segment list
  

and the following two optional arguments:

• 
  the position (as an integer) relative to the input segment list
  
• 
  a specification for whether a new segment list or just its labels should be returned
  

For example, suppose you have made a segment list of all words in the utterance thorsten:

"1" "3" "1" "1" "1" "3" "0" "1" "1" "3" "1" "1" "1" "1" "1" "4"

"no-segment" "L*" "no-segment" ...

If you want to find the preceding or following segments relative to a segment then use the seq argument. Thus:

segment list from database: gt

query was: requery

labels start end utts

1 no-segment 0.000 0.000 thorsten

2 jeden 47.689 250.366 thorsten

3 morgen 250.366 608.032 thorsten
The reason why the first segment has a no-segment entry is because there can be no segment that precedes the first word of the utterance.

If you want to re-query for both a different tier and a different position, then two queries are needed. Firstly, a segment list is made of the preceding word and secondly these preceding words are queried for pitch-accents:

prec.tone = emu.requery(prec.word, "Word", "Tone", j=T)

Putting together annotation structures such as the one discussed so far can be useful for subsequently searching the database, but the data entry can be cumbersome, especially for a large and multi-tiered database. For this reason, there is the facility in Emu to automate various stages of tree-building via an interface to the Tcl/Tk programming language and there are some existing programs in the Emu-Tcl library for doing so. Since an introduction to the Tcl/Tk language is beyond the scope of this book, some examples will be given here of using very simple programs for building annotation structures automatically. Some further examples of using existing Emu-Tcl scripts are given in Appendix B of the website associated with this book.

For the present example, the task will be to link annotations between the Tone and Word tiers in the aetobi database in order to be able to make similar kinds of queries that were applied to the gt database above. The annotations in aetobi are arranged in three time tiers, Word, Tone, and Break that have the same interpretation as they did for the gt database considered earlier. Since in contrast to the gt database, none of aetobi's annotations are linked, then inter-tier queries are not possible. The first task will be to use an Emu-Tcl function LinkFromTimes to link the Word and Tone tiers based on their times in order to allow queries such as: 'find all pitch-accented words'.
Fig. 4.12 about here
The Emu-Tcl function LinkFromTimes, causes annotations at two tiers T and U to be linked whenever the time(s) of the annotations at tier U are within those of tier T. Therefore, LinkFromTimes should link the H*, L*, and H* pitch-accents to Anna, married, and Lenny respectively in Fig. 4.12 because the times of the pitch-accents all fall within the times of these word boundaries. It is not clear what this function will do to the annotations L-H% and L-L%: in annotating these utterances, which was done in the early 1990s as part of the American English ToBI database, the task was to align all phrase and boundary tones like these with the right word boundary (so the right boundary of Anna and L-H% should have the same time) but it would have needed very fine mouse control indeed to make the boundaries coincide precisely. So it is more than likely that these boundary times are either fractionally before or after the word boundary and for this reason it is difficult to predict whether they will be linked with the phrase-final or phrase-initial word.

Three steps are needed to run LinkFromTimes function and these are:

• 
  write an Emu-Tcl script than includes this function
  
• 
  load the script into the template file
  
• 
  modify the template file if need be
  

For the first of these, the syntax is:

proc AutoBuildInit {template} {}

proc AutoBuild {template utt} {LinkFromTimes $utt Word Tone}
Save this plain text file as aetobi.txt somewhere on your system. Now edit the template file of the aetobi database in the usual way in order to make two changes: firstly, Word must be made a parent of Tone because Emu will only link annotations between two tiers non-linearly if one tier is declared to be a parent of the other; and secondly you will have to tell Emu where to find the plain text file, aetobi.txt, that you have just created, (Fig. 4.13).
Fig. 4.13 about here
Then save the template file, reload the database and open any utterance. There will now be a new button Build Hierarchy which is used to run your Tcl-script over the annotations of the utterance that you have just opened. Switch to the hierarchy view, then select Build Hierarchy and finally Redraw to see the result which should be linked annotations between the Word and Tone tiers (Fig. 4.14). The same figure also shows that the L-H% phrase-boundary tone has been linked with the second word (so the labeler evidently positioned this annotation fractionally beyond the offset of the first word Anna).
Fig. 4.14 about here
You could now save the utterance and then work through the entire database one utterance at a time in order to link the annotations in this way, but even for a handful of utterances this becomes tedious. Your Tcl script can instead be run over the entire database with Database Operations → AutoBuildExtern which will bring up the Emu AutoBuild Tool with your Tcl-script ready to be run (Fig. 4.15). If you follow through the instructions in this window, the script will be applied to all of the utterances and the corresponding hlb files saved to the directory specified in the Emu template file. If you open any utterance, you should find that the annotations at the Word and Tone tiers are linked.
Fig. 4.15 about here
Once the tiers are linked, then a query to find e.g. all L* accented words is possible:
emu.query("aetobi", "*", "[Word != x ^ Tone = L*]")
Read 13 records

segment list from database: aetobi

query was: [Word!=x ^ Tone=L*]

labels start end utts

1 married 529.945 897.625 anna1

2 can 1063.330 1380.750 argument

3 can 3441.810 3723.970 argument

4 Atlanta 8483.270 9214.580 atlanta

5 audience 1070.865 1473.785 audience1

6 bananas 10.000 655.665 bananas

7 poisonous 880.825 1786.855 bananas

8 ma'am 1012.895 1276.395 beef

9 don't 2201.715 2603.025 beef

10 eat 2603.025 2867.945 beef

11 beef 2867.945 3441.255 beef

12 and 3510.095 3622.955 blond-baby1

13 pink 3785.905 4032.715 blond-baby1
In Cassidy, Welby, McGory and Beckman (2000), a more complicated Tcl-program was written for converting the flat annotations of the aetobi database into a hierarchical form similar to that of the gt database considered earlier, in which intonational phrases dominate intermediate phrases which dominate words and in which words are associated with pitch-accents. In addition, and as for the gt database, the boundary and phrase tones are marked at the intonational and intermediate tiers and the script ensures only pitch-accents from the Tone tier are associated with words. The script for carrying out these operations is tobi2hier.txt and it is located in the top level folder of the aetobi database when it is downloaded.
Fig. 4.16 about here
As described above, the parent-child tier relationships need to be defined and the script has to be loaded into the template file. For the first of these, the template file has to be changed to encode the path in Fig. 4.16. This path has to be entered as always in the Levels pane of the template file and the simplest way to do this, when there are already existing tiers, is in text mode, as shown in Fig. 4.17. Having edited the tiers in the manner of Fig. 4.17, you must also load the new Emu-Tcl program (in the Variables pane of the template file, as already described in the right panel of Fig. 4.13). If you downloaded the aetobi database to the directory path, then the file that you need to load is path/aetobi/tobi2hier.txt. While you are editing the template, you should also select the new tiers Intonational and Intermediate to be displayed in the View pane of the template file.
Fig. 4.17 about here
Now call up the program for running this script over the entire database as in Fig. 4.15 which should now load the new Emu-Tcl script tobi2hier.txt. After you have run this script, then the hierarchy for each utterance should be built and visible, in the manner of Fig. 4.18. You will now be able to query the database for all the linked tiers. For example, the words associated with H* in L- intermediate and H% intonational phrases is given by:

emu.query("aetobi", "*", "[ [ [ Word != x ^ Tone = H* ] ^ Intermediate = L- ] ^ Intonational = H% ]")

1 Anna 9.995 529.945 anna1

2 yes 10.000 2119.570 atlanta

3 uh 2642.280 3112.730 atlanta

4 like 3112.730 3360.330 atlanta

5 here's 10.000 206.830 beef

Fig. 4.18 about here

In all of the tier relationships considered in the various corpora so far, the tiers have been stacked up on top of each other in a single vertical path – which means that a tier has at most one parent and at most one child. However, there are some kinds of annotation structure that cannot be represented in this way. Consider as an example of this the relationship between various tiers below the level of the word in German. It seems reasonable to suggest that words, morphemes, and phonemes stand in a hierarchical relationship to each other because a word is composed of one or more morphemes, which is composed of one or more phonemes: thus kindisch (childish) unequivocally consists of a sequence of two morphemes kind and -isch which each map onto their constituent phonemes. Analogously, words, syllables, and phonemes also stand in hierarchical relationship to each other. Now there is a well-known phonological process (sometimes called final devoicing) by which obstruents in German are voiceless in prosodically-final position (e.g., Wiese, 1996): thus although the final consonant in Rad (wheel) may have underlying voicing by analogy to e.g., the genitive form, Rades which surfaces (is actually produced) with a voiced /d/, the final consonant of Rad is phonetically voiceless i.e., produced as /ra:t/ and possibly homophonous with Rat (advice). Therefore, since kindisch is produced with a voiced medial obstruent, i.e., /kɪndɪʃ/, then the /d/ cannot be in a prosodically final position, because if it were, then kindisch should be produced with a medial voiceless consonant as /kɪntɪʃ/ as a consequence of final devoicing. In summary it seems, then, that there are two quite different ways of parsing the same phoneme string into words: either as morphemes kind+isch or as syllables kin.disch. But what is the structural relationship between morphemes and syllables in this case? It cannot be hierarchical in the sense of the term used in this Chapter, because a morpheme is evidently not made up of one or more syllables (the first morpheme Kind is made up of the first syllable but only a fragment of the second) while a syllable is also evidently not made up of one or more morphemes (the second syllable consists of the second morpheme preceded by only a fragment of the first). It seems instead that because phonemes can be parsed into words in two different ways, then there must be two different hierarchical relationships between Phoneme and Word organised into two separate paths: via Morpheme along one path, but via Syllable along another (Fig. 4.19).

Morpheme Word

Phoneme Morpheme

Syllable Word

Phoneme Syllable
Fig. 4.20 about here
This type of dual path structure also occurs in the downloadable ae database of read sentences of Australian English in which an Abercrombian stress-foot has been fused with the type of ToBI prosodic hierarchy discussed earlier. A stress-foot according to Abercrombie (1967) is a sequence of a stressed syllable followed by any number of unstressed syllables and it can occur across word-boundaries. For example, a parsing into stress-feet for the utterance msajc010 of the ae database might be:
It is | fu | tile to | offer | any | further re|sistance |

w w s s w s w s w s w w s w

emu.query("ae", "msajc010", "[Foot=F & End(Intonational, Foot)=1]")

emu.query("ae", "msajc010", "[Word=C & End(Intonational, Word)=1]")

emu.query("ae", "msajc010", "[Foot=F & End(Intermediate, Foot)=1]")