Methodology: How to apply experimentally driven research

Draft agenda (Latest agenda)

23 November 2009, 11.15 – 12.45

Intro on Experimetation methodology and speakers	Nancy Alonistioti	10 min
Necessity for experimentation from the PPP point of view	Didier Bourse	15 min
Presentation on the methodology issues and how to experiment	Martin May	15 min
How the methodology could facilitate the shorter time to product development	Vania Conan	15 min
Experimentation as a methodology to achieve concrete results: where, how, when	Dimitri Papadimitriou	15 min
Panel discussion a structure of FI research which may be too strict and inhibit research (forced division in many areas). Need for a frequent interaction between the areas and harmonization and cross-validation of activities/ideas. Experimentation is a mandatory step in the validation of new ideas. FIRE facilities offer the bridge between tests in a lab and in a very large scale Migration strategies and coexistence testing are as much important as models validation. FIRE facilities offer also such capabilities. International collaboration and concertation, also at the technical level is of utmost importance. In particular standardization.

Background and Rationale

The Internet has been the ICT success story for many decades, having great impact in everyday life as well. Economical as well as societal aspects have been greatly affected by the emergence and proliferation of the Internet technologies. The support of Internet communication and the great number of novel services over heterogeneous systems has boosted new forms of communication schemes, collaboration possibilities, services, education, knowledge and information propagation as well as innovation. The fast grow of the Internet, points to the need for broader experimentation capabilities for future technologies.

Theoretical models in the context of future internet, emerging communications and services, are usually validated through dedicated prototyping activities. The test beds used in these activities are usually focused in terms of testing capabilities and fragmented in terms of validation capabilities. Therefore, the need to evolve the test beds into coherent experimentation facilities, enabling broader scope experimentation and validation of theoretical approaches, becomes apparent. This will engage the federation of multi scope platforms under a common framework.

Scope

The experimentally driven research, based on large scale federated experimental infrastructures is quite a challenge. The mere benefits appear to be the engagement in a validation and assessment loop early in the R&D process. In this way in complex systems, experimentation could be the key for discovery and validation throughout the research process. There is a lot of value in putting end users participating as active testers at an early stage of the R&D cycle. They could become key indicators of several metrics related to the realization of theoretical approaches in tangible real life scenarios. Indicative metrics that could be assessed would be the scalability of solutions, performance, usability, robustness and security of the tested functionality. Reduced time to market of tested products could be the imminent outcome of the whole process.

Relevant topics to be addressed:

Experimentation as a methodology to achieve concrete results: where, how, when?
The experimentation facilities as a service offered to R&D
What are the metrics relevant to experimentations?
The impact to standardization
Large scale experimentation: requirements and limitations

Participation

The targeted audience would be organizations involved in R&D projects with prototyping and proof of concept work items, the FIRE projects, the PPP representatives, engineers and application providers, as well as the EC members.