MediaLisp a system Design for Multimedia Augmented Transition Network in Lisp

Download 0.63 Mb.

Page	7/7
Date	02.05.2018
Size	0.63 Mb.
	#47217

1 2 3 4 5 6 7

Buckets

Contents	Multimedia String of MATN
3	x1, x2, x3, x4, x5	x1&x2&x3&x4&x5
5	x1, x3, x4, x5	x1&x3&x4&x5
7	x3, x5	x3&x5

The corresponding MATN is drawn as follows:

x3&x5

x1&x3&x4&x5

x1&x2&x3&x4&x5

3.2. Related API Functions:
In order to manipulate multimedia object, we define a small set of API functions described as follows:
3.2.1. Query Name by Reference:

?
	Calling Convention:
		(?stream_ref)
	Parameters:
		stream_ref	The reference of the media stream
	Return Value:
		The name of the media stream.

3.2.2. Getting Reference by Name:

?
	Calling Convention:
		(? stream_name)
	Parameters:
		Stream_name	The name of the media stream
	Return Value:
		The reference of the media stream. Nil means no media stream exists under this name.

3.2.3. Property Query:

GetProperty
	Calling Convention:
		(GetProperty stream_ref property_name)
		(GetProperty stream_name property_name)
	Parameters:
		Stream_name Or Stream_ref	The name or reference of the underlying media stream
		Property_name	The name of the property
	Return Value:
		If the property exists, then return the corresponding value, otherwise return Nil.

3.2.4. Property Update:

SetProperty
	Calling Convention:
		(SetProperty stream_name property_name property_value …) or (SetProperty stream_ref property_name property_value …)
	Parameters:
		Stream_name Or Stream_ref	The name or reference of the underlying media stream
		Property_name	The name of the property
		Property_value	The value of the underlying property
	Return Value:
		None.
	Description:
		For convenience, this function supports variable input. The input must contain exact integer number of "name-value" pairs.
		For example:
		(SetProperty stream_name name1 value1 name2 value2 name3 value3)

Multimedia Representation Control

Except for functionality supported in general Lisp language, MediaLisp also provides support for media stream branching and composing, as well as precise control for multimedia presentation.

The MediaLisp maintains a set of system properties. Those properties can be used to configure the behavior of the overall environment for the multimedia presentation. Dor the sake of flexibility, a concept of Phase for fine-tuning of run-time behavior of one or a group of media streams:

As in the MATN model, media stream branching occurs whenever user choice or automatic choice based on condition table occurs. Media stream choice provides great flexibility to deal with unpredictable situations in real life. We define three branching functions for multimedia presentations, they are Case and Select.

4.1. System Property:
The are two types of system properties. They are defined by MediaLisp (thye are recognizable by system) so as to describe and the control the runtime behavior of the whole presentation. The set of system properties should contain at lease:

Name	Type	Description
Physical_Interval	Number	The physical time duration of unit time interval.
Error_Handling	Integer	The predefined error handling behavior
		1	Ignore error
		2	Prompt message
		3	End presentation
		4	…
…

Accordingly, we define can use functions GetSProperty and SetSProperty to access system properties:

4.1.1. System Property Query:

GetSProperty
	Calling Convention:
		(GetSProperty property_name)

	Parameters:
		Property_name	The name of the underlying system property
	Return Value:
		If the property exists, then return the corresponding value, otherwise return Nil.

4.1.2. System Property Update:

SetSProperty
	Calling Convention:
		(SetSProperty property_name property_value …) or (SetSProperty stream_ref property_name property_value …)
	Parameters:
		Property_name	The name of the system property
		Property_value	The value of the underlying system property
	Return Value:
		None.
	Description:
		For convenience, this function supports variable input. The input must contain exact integer number of "name-value" pairs.
		For example:
		(SetSProperty name1 value1 name2 value2 name3 value3)

4.2. Phase:
The introduction of concept Phase provides detailed control over multimedia presentation. A phase is a sub runtime environment under environment MediaLisp. Similarly, phase maintains a set of properties similar to system properties. Actually, the runtime system can be treated as an implicit phase. If there is no phase specified, MediaLisp uses system properties. Also, used can use any user defined phase properties as long as there is no conflict with the preserved phase properties.

Whenever a phase is specified, MediaLisp uses the properties of the phase instead. When a phase finishes, MediaLisp switches back to the default system properties to control remaining presentation. The following table lists some basic phase properties:

Name	Type	Description
Error_Handling	Integer	The predefined error handling behavior
		1	Ignore error
		2	Prompt message
		3	End current phase
		4	…
…

4.2.1. Semantic of Phase Creation:

Phase
	Calling Convention:
		(Phase phase_name)
	Parameters:
		Phase_name	Optional. If it's not specified, the system will automatically assign an unique one.
	Return Value:
		The reference of the phase

4.2.2. Phase Property Query:

GetPProperty
	Calling Convention:
		(GetPProperty phase_name Property_name) or (GetPProperty phase_ref Property_name)

	Parameters:
		phase_name	Name of the specific phase
		phase_ref	Reference of the specific phase
		Property_name	The name of the underlying system property

	Return Value:
		If the property exists, then return the corresponding value, otherwise return Nil.

4.2.3. Phase Property Update:

SetPProperty
	Calling Convention:
		(SetPProperty phase_name property_name property_value …) or (SetPProperty phase_ref property_name property_value …)
	Parameters:
		phase_name	Name of the specific phase
		phase_ref	Reference of the specific phase
		Property_name	The name of the phase property
		Property_value	The value of the underlying phase property
	Return Value:
		None.
	Description:
		For convenience, this function supports variable input. The input must contain exact integer number of "name-value" pairs.

4.3. Media Stream Branching
4.3.1. Case:
The Case function is similar to "switch" function in C++, its decisions are based on integer value. Usually, the integer value should be the choice sequence number from a user interactive choice. For this purpose, we also define ChoiceList data type to describe user prompts.

4.3.1.1. Semantic of Case:

Case
	Calling Convention:
		(Case num stream1 stream2 …)
	Parameters:
		Num	The integer on which the decision should be based at runtime
		Stream_i	The corresponding media stream reference
	Return Value:
		The reference of media stream (unit stream, parallel or sequence) that is determined by num at runtime.
	Description:
		If there is no stream defined corresponding to the choice number, an exception will be passed to the phase or system.

4.3.2. ChoiceList:
A ChoiceList object is simply an array of prompt string. The sequence number starts from 1. In MediaLisp, ChoiceList object triggers a dialog that shows all prompts in order to user and returns the choice number. The semantics of ChoiceList is shown as follows:

ChoiceList
	Calling Convention:
		(ChoiceList prompt1 prompt2 prompt3 …)
	Parameters:
		prompt_i	The corresponding prompt string reference
	Return Value:
		The reference of choice object

4.3.3. Select
An alternative of media stream branching is Select. Compared with Case, Select provides much more flexible and detailed detail. This function can be used to serve automatic decision-maker depending on various system or phase properties at run time. The semantics of Select is shown as follows:

Select
	Calling Convention:
		(Select condition1 stream1 condition2 stream2 …)
	Parameters:
		Condition_i	The condition which should be of boolean data type
		Stream_I	The media stream reference corresponding the underlying condition
	Return Value:
		The reference of choice object

The input parameter consists of a list of "boolean-stream" pairs. This function is executed as follows: The MediaLisp evaluates the input list pair by pair from left to right. If the condition is true, then return the corresponding media stream reference. If no condition is true, then raises an exception and passes it the underlying phase of system. To make sense, we can always put an explicit "true" condition (value T in Lisp) paired with a stream reference to specify a default value.

4.3.4. Temporal Relationship in Media Branching
All media branching functions can be regarded as a specific media stream (can be simple media stream, parallel or sequence of media streams) whose actual time duration need to be determined at runtime. To keep track their actual time, we need to utilize MATN model to describe and control the branching. The detailed technique concerning this issue will be discussed in later section.
4.4. Presentation
After all media streams and their temporal and behavioral properties has been defined, the work left is to bind all those information together and add them to the MATN based execution tree implemented within MediaLisp. This job can be done by the Present function call. The semantics of Present is listed as below:

Present
	Calling Convention:
		(Present stream) or (Present phase stream)
	Parameters:
		stream	The reference of media stream
		phase	The reference of underlying phase
	Return Value:
		None.

Comprehensive Examples

So far we have briefly discussed all necessary conventions about MediaLisp. In this section, we will demonstrate MediaLisp to by an example.

5.1. Example1:
In this example, a bunch of multimedia objects of different types need to be displayed time by time. They are listed in following table:

Object	Type	Start Time	Duration	Location
V1	Video	0	5	http://www.fiu.edu/vedio1.mpg
T1	Text	0	5	C:\temp\text1.txt
I1	Image	3	7	D:\iamges\image1.jpg
A1	Audio	3	8	http://www.fiu.edu/a.au
T2	Text	5	2	C:\temp\text2.txt

The corresponding MediaLisp script:

(Define "v1")

(setproperty "v1" "Duration" 5

"Mediatype" "video"

"location" " http://www.fiu.edu/vedio1.mpg")

(Define "T1")

(setproperty "T1" "Duration" 5

"Mediatype" "text"

"location" "C:\temp\text1.txt")

(Define "I1")

(setproperty "I1" "Duration" 7

"Mediatype" "image"

"location" "D:\imges\image1.jpg")

(Define "A1")

(setproperty "A1" "Duration" 8

"Mediatype" "audio"

"location" "http://www.fiu.edu/a.au ")

(Define "T2")

(setproperty "T2" "Duration" 2

"Mediatype" "text"

"location" "C:\temp\text2.txt")

(setf x1 (||| (? "v1") (? "T1")))

(setf x2 (<<< (delay 3) (||| ( (? "I1") (? "A1")) )

(setf x3 (<<< (delay 5) (? "T2") ))

(present (||| x1 x2 x3) )

5.2. Example2:
In this example, we use a sequence of images to simulate an animation. For each image there are two versions of source files: compressed version and uncompressed version. Suppose for j^thimage

Compressed version:	http://www.fiu.edu/compress_i.jpg
Uncompressed version:	http://www.fiu.edu/regular_i.jpg

(setf ANI NIL)

(setf i 1)

(while (le N)

(setf x1 (define NIL))

(setf x2 (define NIL))

(setproperty x1 "duration" 1, "Mediatype" "image")

(setproperty x2 "duration" 1, "Mediatype" "image")

(setf s1 (+ "http://www.fiu.edu/" "compressed_" i "jpg"))

(setf s1 (+ "http://www.fiu.edu/" "uncompressed_" i "jpg"))

(setf unit (select (getSproperty "Bandwidth") x1 true x2))

(setf ANI (if (eq ANI NIL) unit (<<< ANI unit)))

)

(present ANI)

hether to use the compressed or uncompressed version, depends on the bandwith of the system. Let N be the total number of images included in this animation, then the script is:

Issues Concerning MediaLisp Interpreter

To support multimedia related functionality, MediaLisp must provide:

Multi-threaded execution and management support.
Network communication ability
Real-time management and synchronization.
Intelligence to handle the semantic heterogeneity of multimedia representation.
The ability to display various multimedia objects via various devices.

So the in internal implementation of MediaLisp interpreter is much more complex than common Lisp interpreters, it has four sub-systems that work together to process a multimedia presentation. They are:

General Lisp interpreter
MediaLisp interpreter
Real-time media presentation control sub-system
Real-time media retrieval management sub-system

6.1. General Lisp Interpreter:
This part severs as common list interpreter that evaluates (interprets) instructions in MediaLisp program except for those in MediaLisp extension. It manages all variables and control flows. It also passes the MediaLisp instruction to MediaLisp Interpreter.

6.2. MediaLisp Interpreter:
This interpreter is designed to evaluate MediaLisp instructions. It creates some extra data structures will be created to hold all the media object's internal properties as well as their mutual temporal and behavioral relationship, such as:

Vector of unit media objects:
Vector of phase objects (the outermost phase is the
A MATN graph data structure whose edges are multimedia strings and nodes are time stamps and choice markers.

The pointers (reference relationship) among those data structures will alos be built up within this part.

6.3. Real-time media presentation control sub-system:

This sub-system plays a crucial role in multimedia representation. It serves as a central coordinator to control (track and synchronize) the media retrieval, display and error handling during realtime presentation based upon the MATN graph,

Starts from the start node of the MATN graph, the coordinator periodically (every unit time interval) looks one more steps ahead to predict what to do next time interavl.

Before a multimedia object starts to display, it needs to initiates a pre-retrieval thread to build up the network transmission channel, caches the whole object or its header depending on the properties of the object. Whenever a choice is made, it can correctly lead the media stream to the certain branch. It also manages media displayers and assign different media objects to certain players.

A small knowledge needs to be involved to help making decision.

7. Conclusion and Future Works
From the proposed draft design of MediaLisp in this paper, we have presented a feasible approach to describe and control multimedia representations. By adopting the MATN semantic model, this approach is expressive, flexible and convenient enough to handle multimedia presentations.

Though Lisp was developed in 1950s, it has gained extensive interests in recent 10 years since its simplicity and flexibility. The semantics is very suitable to define tree-like data structures. By extending Lisp with multimedia support, this approach can easily describe all static and dynamic properties and conditions with respect to multimedia presentation.

By adopting the MATN semantic model, MediaLisp can precisely control and keep tracking of both static and dynamic temporal relationship among all multimedia objects in a presentation. The ability to model dynamic temporal relationship as well as the flexibility of scripting technique, MediaLisp can handle any complex situations concerning multimedia presentation.
8. Reference

[1].	Shu-Ching Chen, Mei-Ling Shyu, and R. L. Kashyap, "Augmented Transition Network as a Semantic Model for Video Data," accepted for publication, International Journal of Networking and Information Systems, Special Issue on Video Data.
[2].	Sheng-Tun Li, Shu-Ching Chen, and Mei-Ling Shyu, "A Snapshot Browsing Model for Distributed Surveillance Systems," accepted for publication, Journal of Applied Systems Studies, Special Issue on Distributed Multimedia Systems with Applications.
[3].	Sheng-Tun Li, Shu-Ching Chen, and Mei-Ling Shyu, "A Live TV-Quality Distant learning Multimedia Presentation System for Education," the 34th Hawaii International Conference on System Sciences (HICSS-34), CD-ROM, IEEE Press, 9 pages, January 3-6, 2001, Maui, Hawaii.
[4].	Sheng-Tun Li, Shu-Ching Chen, and Mei-Ling Shyu, "A Presentation Semantic Model for Asynchronous Distance Learning Paradigm," ACM Multimedia 2000 Conference, pp. 404-406, October 30 - November 3, 2000, Los Angeles, California.
[5].	Mei-Ling Shyu and Shu-Ching Chen, "A Bayesian Network-Based Expert Query System for a Distributed Database System," IEEE International Conference on Systems, Man, and Cybernetics, Nashville, Tennessee, USA, pp. 2074-2079, October 8-11, 2000.
[6].	Shu-Ching Chen, Mei-Ling Shyu, and Naphtali Rishe, "Modeling Interactive Multimedia Presentation Systems Using Augmented Transition Networks," First International Workshop on Intelligent Multimedia Computing and Networking (IMMCN'2000), pp. 643-646, February 27-March 3, 2000, Atlantic City, NJ, U.S.A.
[7].	Shu-Ching Chen, Srinivas Sista, Mei-Ling Shyu, and R. L. Kashyap, "Augmented Transition Networks as Video Browsing Models for Multimedia Databases and Multimedia Information Systems," 11th IEEE International Conference on Tools with Artificial Intelligence (ICTAI'99), pp. 175-182, November 9-11, 1999, Chicago, IL, U.S.A.
[8].	Guy L. Steele, "Common Lisp the Language, 2nd Editio", Digital Press 1990 paperbound ISBN 1-55558-041-6
[9].	Sean Luke, "Java for Lisp Programmers",at AAAI99 (the 1999 conference of the American Association for Artificial Intelligence).
[10].	Heiko Kirschke, "PERSISTENT LISP OBJECTS!", Version 2.09 of May 22, 2000. "http://www.lisp.de/software/plob/Welcome.html"
[11].	http://www.elwoodcorp.com/alu/table/lisp.htm (March 2001).

Download 0.63 Mb.

Share with your friends:

1 2 3 4 5 6 7