D5.1 - Algorithms Performance and Reliability Assessment Report
This report aims at evaluating the impact of key design decisions on the performance and reliability of database replication protocols being considered in the GORDA project. Namely, we are interested in the detailed evaluation of concurrency control mechanims and in the interaction with group communication protocols with realistic workloads and faultloads.
However, current methodologies of evaluation of replication protocols reveal themselves limited. The existent studies report to analysis conducted in local area networks or in small scale scenarios. The fact is that there are few chances to put together a wide area testing environment because that is not costeffective, does not provide full control over the environment variables as it does not enable deterministic testing. Furthermore, although in local area environments there is more control over the environment, there is still no deterministic execution and, as in the wide area, a full deployment of the software is also required. These issues must be addressed when researching, developing, evaluating distributed replication protocols. Realistic and controlled performance evaluation of distributed and replicated applications is still very hard to achieve, specially if one considers large scale scenarios and fault injection. Although there are few testbeds [7, 24, 47] that enable the evaluation of distributed systems, they all focus a small portion of the spectrum of possible problems. On the other hand, pure simulation models may be a solution to this problem, but creating abstract models from real implementations is not always possible or even desirable.
The problem extends itself when there is the need for testing early and often during the development cycle of a new replication protocol. The researcher/developer ﬁnds himself without the possibility to abstract the irrelevant parts of the system, focussing on the protocol and still conduct a realistic evaluation. For instance, when evaluating a database replication protocol, the database engine and the network could be abstracted as realistic simulation models, since they are not really part of the problem, and the protocol should be the only real implementation. This methodology is known as incremental development. In this context, the development process is backed by an abstract simulation model comprised of multiple sub-models that get replaced by real implementations as these become available. The problem here is having the simulation execution to play nicely with the real execution. Whenever real code execution takes place, the time spent on it must be accounted in the simulation time line. Doing this in an ad hoc fashion is time consuming and error prone. Hence the approach of centralized simulation proposed in CESIUM , must be taken into account. One cannot reuse CESIUM implementation because it deﬁnes speciﬁc models that are not completely suitable for evaluating database replication protocols.
Last modified 2007-06-25 04:18 PM