I have evaluated a software project which has been created by a consortium of top scientists in the field. However, the project itself it doesn't really work, and has only been developed as a proof-of-concept rather than an end product (i.e., it works only with 2-3 scenarios).
This software application should perform 4 steps in order to successfully execute. Each step takes an input file, and produces an output file. The output file from the previous step is used as input in the current step. Initially, you start with 1 file. This file is used as input for step 1. After step 1, another file is produced. Let’s call the input file general_input_file, and the output file general_output_file. When general_input_file is loaded into the application, general_output_file should be produced. Now, I have an input file which I will call my_input_file. I expect the application to produce my_output_file. However, the application only accepts specific_input_file and will produce a specific_output_file. This means that it only works with 2 files which has been previously generated. Both files exist in the project. When I look at the part of the project which should process the general_input_file, there is a statement which looks like this: if the name of the given input file equals specific_input_file, then return specific_output_file. This is a file inside the project. Otherwise, try to process the generate_input_file and generate general_output_file. At this point the software breaks. A number of exceptions are thrown, and debugging and fixing this is beyond of the work that I’m doing.
The question is: in the paper, how do you approach this issue? And how do you argue, in the paper, that the reason you're unable to evaluate the software in a different scenario is due to the limitations of the software? What is the best wording to be used, without being offensive to the authors?
No comments:
Post a Comment