Model-driven development of interactive multimedia applications [Elektronische Ressource] : towards better integration of software engineering and creative design / vorgelegt von Andreas Pleuß

Andreas Pleuss

Sujets

• Mathematics

Informations

• Publié par: ludwig-maximilians-universitat_munchen
• Publié le: 01 janvier 2009
• Nombre de lectures: 37
• Poids de l'ouvrage: 15 Mo

Extrait

Model-Driven Development of
Interactive Multimedia Applications
Towards Better Integration of
Software Engineering and Creative Design
Andreas Pleuß
München 2009Model-Driven Development of
Interactive Multimedia Applications
Towards Better Integration of
Software Engineering and Creative Design
Andreas Pleuß
Dissertation
an der Fakultät für Mathematik, Informatik und Statistik
der Ludwig–Maximilians–Universität
München
vorgelegt von
Andreas Pleuß
geboren in Würzburg
München, den 25. Februar 2009Erstgutachter: Prof. Dr. Heinrich Hußmann
Zweitgutachter: Prof. Dr. Peter Forbrig (Universität Rostock)
Prof. Dr. Jean Vanderdonckt (Université Catholique de Louvain, Belgien)
Tag der mündlichen Prüfung: 27. Mai 2009Contents
Abstract xv
1 Introduction 1
2 Interactive Multimedia Applications 5
2.1 Basic Terms and Definitions .............................. 5
2.1.1 Multimedia ................................... 5
2.1.2 Multimedia vs. Multimodality ......................... 7
2.1.3 Applications ............................ 7
2.1.4 Multimedia vs. Web and Hypermedia Applications ....... 8
2.2 Multimedia Application Development 9
2.3 Implementation Technologies and Tools ........................ 10
2.3.1 Frameworks and APIs ............................. 11
2.3.2 Declarative Languages 12
2.3.3 Authoring Tools ................................ 13
2.4 Classification ...................................... 19
2.4.1 Existing Classifications 19
2.4.2 A Classification for this Thesis......................... 20
2.5 Conclusions for this Thesis ............................... 22
3 Problem Statement and Proposed Solution 25
3.1 Current Problems in Multimedia Application Development .............. 25
3.1.1 Interdisciplinary Roles ............................. 28
3.1.2 Authoring Tools 30
3.2 Analyzing the Spectrum of Possible Solutions ..................... 30
3.3 Proposed Solution: A Model-Driven Approach .................... 33
3.4 A Short Introduction into Model-Driven Engineering ................. 34
3.4.1 Models, Metamodels, and Modeling Languages................ 34
3.4.2 Model-Driven Development .......................... 37
3.4.3 Practical Application and Profiles ....................... 40
3.5 An Illustrating Example Scenario............................ 43
4 Related Modeling Approaches 47
4.1 User Interface Modeling ................................ 47
4.1.1 General Concepts 48
4.1.2 Concrete Examples ............................... 51
4.1.3 User Interface Modeling and Software Engineering Concepts . . . ...... 57vi CONTENTS
4.2 Web Engineering .................................... 61
4.3 Multimedia Modelling ................................. 63
4.3.1 HyDev ..................................... 64
4.3.2 OMMMA 68
4.3.3 Code Generation ................................ 74
4.4 Discussion........................................ 75
5 Modeling Multimedia – MML Core Principles 79
5.1 Basic Decisions and Language Engineering ...................... 79
5.1.1 Scope ...................................... 80
5.1.2 Language Definition .............................. 80
5.1.3 Notation .................................... 81
5.1.4 Custom Icon Design 82
5.1.5 Basic Technical Metamodeling Principles ................... 84
5.2 Modeling Media Components ............................. 86
5.2.1 Rationale 86
5.2.2 Media Types and Media Components ..................... 87
5.2.3 Interfaces 89
5.2.4 Media Representations 91
5.2.5 Artifacts and Instances 92
5.2.6 Keywords for Media Representations 94
5.2.7 Media Parts ................................... 95
5.2.8 Inner Properties................................. 98
5.2.9 Part Artifacts ..................................100
5.2.10 Variations ....................................101
6 MML – A Modeling Language for Interactive Multimedia Applications 103
6.1 MML – Task Model103
6.2 MML – Structural Model ................................105
6.3 MML – Scene Model107
6.4 MML – Presentation Model...............................113
6.4.1 Abstract User Interface.............................114
6.4.2 UI Realizations .................................118
6.4.3 Sensors .....................................120
6.5 MML – Interaction Model123
6.5.1 Rationale ....................................123
6.5.2 MML Interaction Model based on UML Activity Diagrams . . . .......126
6.5.3 Temporal Synchronization ...........................129
6.6 Model Interrelations and Modeling Process ......................132
6.6.1 Modeling Process................................132
6.6.2 Conceptual Interrelations............................135
6.7 Modeling Tool Support .................................136
7 Integrating Creative Design: From MML to Flash 141
7.1 A Metamodel for Flash and ActionScript........................141
7.1.1 Flash Metamodel Basics142
7.1.2 Flash Documents144Contents vii
7.1.3 ActionScript ..................................147
7.2 A General Structure for Flash Applications ......................152
7.2.1 Principles....................................152
7.2.2 Scenes .....................................153
7.2.3 Complete Structure ...............................156
7.3 Transformations158
7.3.1 Round-Trip-Engineering ............................161
7.3.2 A Tool for Extracting Flash Models from Flash Applications . . .......164
7.4 Working with the Code Skeletons in the Authoring Tool................164
8 Validation 173
8.1 Platform Independence: Transformations to other Platforms..............174
8.1.1 Code Generation for Java175
8.1.2 Code for SVG/JavaScript .....................176
8.1.3 Code Generation for FlashLite .........................176
8.2 External Validation ...................................178
8.2.1 MML in the Course “Multimedia-Programming” ...............179
8.2.2 MML in Project Theses ............................183
8.2.3 Conclusions187
8.3 Internal Validation187
8.3.1 Comparison to General Criteria for Modeling Languages ...........188
8.3.2 to the Goals of This Thesis ....................191
8.3.3 Comparison to Other Approaches .......................193
9 Outlook: Towards Better Integration of Software Engineering and Creative Design 197
9.1 Creative Design in Software Development197
9.2 Vision: Models as Central Hub .............................198
9.3 First Steps ........................................199
10 Summary and Conclusions 203
10.1 Summary203
10.2 Contributions ......................................204
10.3 Future Work.......................................204
10.4 Conclusions206
A Multimedia Taxonomy by Hannington and Reed 207
B ActionScript Class MovieClip – Documentation 209
B.1 Properties ........................................209
B.2 Inherited from Class Object .........................211
B.3 Operations211
B.4 Event Handling Operations ...............................214
C Transformation from MML to Flash Model 217
D Questionnaire 221
Bibliography 227viii ContentsList of Figures
1.1 Examples for multimedia applications today. (From left to right, starting in the top
row: Rich Internet Application [Goo], E-Learning [Jungwirth and Stadler03], Enter-
tainment [Hilliges et al.06], Entertainment on mobile devices [Tavares], Infotainment
in cars [Inf], Instrumented Environments [Bra]) .................... 2
2.1 Technical media: Classes and examples based on [Hoogeveen97] and MHEG [ISO97b]. 6
2.2 Classification of media according to [Fetterman and Gupta93]. ............ 6
2.3 Multimedia applications in the value chain. ...................... 9
2.4 Structure in the Piccolo framework........................... 12
2.5 MHEG-5 class hierarchy from [MHE].......................... 13
2.6 A screenshot of Adobe Authorware. .......................... 15
2.7 A of Director. ............................ 16
3.1 RepresentationOf relationship according to [Favre04c] ................ 35
3.2 ConformsTo relationship according to [Kurtev et al.06] 35
3.3 The basic idea of MDA ................................. 37
3.4 OMG Metadata Architecture according to [Obj05]................... 39
3.5 Technical Spaces according to [Kurtev et al.06] .................... 40
3.6 Notation options for UML stereotypes.......................... 41
3.7 Screenshot from the example Racing Game application................. 44
3.8 Example: Wheels of a racing car defined as independent inner parts. ......... 45
3.9 The generated skeletons including placeholders can be directly opened and edited in
the Flash authoring tool. ................................ 46
4.1 Model-based User Interface Development Environments according to [Szekely96,
da Silva00] ....................................... 48
4.2 The CAMELEON reference framework according to [Calvary et al.02, Calvary et al.03] 50
4.3 Example ConcurTaskTree Model from [Wu06b].................... 52
4.4 Extract from the UsiXML metamodel [Usi06] showing the UsiXML models. .... 53
4.5 Relationships between different models in UsiXML .................. 54
4.6 Example for a Context-sensitive task model in Dynamo-Aid. ............. 55
4.7 for a Context-specific dialog model calculated from task models in Dynamo-
Aid