Review summary

I think the key improvement that is needed is to not automatically equate "open" with "better". I know that the goal here is open science, and I am very much for open. But arguing that X is better than Y because X is open or public, is not a very convincing argument on its own. For example, it is stated as an advantage for reviewers that their reviews will be public; I'm pretty sure that if you ask reviewers, many would prefer to provide their input in a private manner.

The current proposal is a bit heavy on justifying why doing what ThinkLab does/wants to do is good. In my opinion it falls short on concretely stating what you want to do next if you get funded. The fundamental idea of the proposal is to be open and share ideas before you carry them out. Leaving out your own novel ideas that you have not been implemented yet, could thus be seen by some as a bit hypocritical.

Thanks for reviewing our proposal Lars! I really appreciate it.

I would definitely like to correct all cases where the proposal argues X is better than Y because of "open". I see we did a poor job of this in the section you noted.

I wrote the part about reviewer benefit with a fundamental assumption that if a scientist thinks they have something valuable to say, they'd like to be recognized by their peers for it. This would lead me to believe that if a reviewer prefers to provide input in a private manner, it's likely because they don't have much confidence their contribution improve their reputation or be seen as adding value. Do you think that is wrong? Can you shed more light on why many might prefer to provide their input in a private manner?

In my opinion it falls short on concretely stating what you want to do next if you get funded.

Yes, we will try to fill this out. It's not something we really know that concretely. Basically, we've already got a working prototype — but it needs a lot of refinement that will largely depend on user feedback and user testing. We're not hiding some additional novel idea. I think we've already got a huge amount on our plate.

Why would scientists not like to post reviews/comments on proposals in public?

The fundamental problem is that we typically can only allocate very short time to go through a proposal and provide input on it. Consequently, some comments will be poorly thought through, be silly mistakes due to reading too fast, be poorly phrased, and possibly full of typos. This is all fine when you send the input in an email to the person writing the proposal; they take the input that makes sense and ignores the silly mistakes and typos. But we would not feel comfortable about putting such hasty, half-baked comments online for everyone to read forever after.

Doing public open review will thus likely take more time than providing private input, due to self-imposed quality requirements.

Okay great points. I will update the language to take all this into account.