Loading...
Please wait, while we are loading the content...
Similar Documents
Evaluation as a Service architecture and crowdsourced problems solving implemented in Optil.io platform
| Content Provider | arXiv |
|---|---|
| Author | Wasik, Szymon Antczak, Maciej Badura, Jan Laskowski, Artur |
| Date of Submission | 2018-07-14 |
| Abstract | Reliable and trustworthy evaluation of algorithms is a challenging process. Firstly, each algorithm has its strengths and weaknesses, and the selection of test instances can significantly influence the assessment process. Secondly, the measured performance of the algorithm highly depends on the test environment architecture, i.e., CPU model, available memory, cache configuration, operating system's kernel, and even compilation flags. Finally, it is often difficult to compare algorithm with software prepared by other researchers. Evaluation as a Service (EaaS) is a cloud computing architecture that tries to make assessment process more reliable by providing online tools and test instances dedicated to the evaluation of algorithms. One of such platforms is Optil.io which gives the possibility to define problems, store evaluation data and evaluate solutions submitted by researchers in almost real time. In this paper, we briefly present this platform together with four challenges that were organized with its support. |
| Related Links | https://arxiv.org/pdf/1807.06002.pdf |
| arXiv | 1807.06002 |
| Language | English |
| Access Restriction | Open |
| Subject Keyword | Computer Science - Computers and Society Computer Science - Performance Computer Science |
| Content Type | Text |
| Resource Type | Article |
| Subject | Computer Science |
