Speaker: prof. dr. Benoît Depaire
The design and evaluation of algorithms have been a major concern of computer science since its founding days and is still a matter of discussion, as the recent debate for more and less experimental evaluations of algorithms shows. In essence, tension exists between different views of how much research on algorithms is a matter of formal science or empirical science. This talk presents ongoing research with Jan Mendling and Henrik Leopold which follows up on this debate and is connected to the following two observations:
- First, we observe that a survey on the practice of algorithm engineering is missing. Even though much of the methodological considerations seem to be implicitly clear in various domains dealing with the design of algorithms, they do vary from one domain to the other. While there are good reasons for these differences, this raises the question of whether there exists a common scientific ground?
- Second, we observe that many areas of algorithm engineering have turned to competitive testing and iterating over benchmark data sets with established quality metrics to tease out the last epsilon of improvement. Such competitive testing provides knowledge about which algorithm is better with respect to accuracy metrics, but limited understanding of why. Since benchmarking has become such an established approach, it can be hard to consider alternative avenues of scientific knowledge creation if authors do not have the meta arguments at hand to express that their deviation provides sound insights. It is not our aim to engage in a normative discussion to which extent algorithms should be evaluated theoretically or empirically. We believe both positions have their merit and research in both areas provides valuable insights. Instead, we develop a theoretical framework that integrates different perspectives on algorithms and their evaluation from computer science, operations research, design science, and the philosophy of engineering. This framework describes an ontological perspective of what algorithm engineering is concerned with, an epistemological perspective of what knowledge algorithm engineering aims to formulate, and a methodological perspective of how algorithm engineering advances our understanding of algorithms.