How to address the risk of stealing ideas

After a first quick read through the proposal, I think the main weakness is the issue of stealing ideas. Currently, I do not feel this is addressed in a good way.

The proposal acknowledges that this will be a concern of many researchers, and that this indeed is inherent to the way the system currently works. I fully agree with that. The problem is the proposed solution: "In our view, the solution to this predicament is clear: we need to change the rules of the game!" In effect, what you write is that the ThinkLab approach won't work within the current system.

As a reviewer, I would tend to say that this implies that you're putting the cart before the horse. You are trying to develop a very specific tool that won't work until the whole system has been changed, which is likely to take a long time. On other words, why develop the system now? Why not focus on changing the system and worry about developing the tool, once the system makes it feasible?

(I am obviously playing devil's advocate here.)

Thanks for playing devil's advocate. I think there's a few things we need to clarify.

By "changing the rules of the game", we don't mean to imply that the entire system must change at once. Funders can change the rules of the game just for those researchers that they fund. They can require that proposals be submitted openly in the same way that they require that results must be published openly.

Basically, the idea here is that funders have a lot of power because demand for funding is so much higher than supply. Now it's possible requiring an openly submitted proposal could result in lower quality researchers applying initially — but you could also look at it another way: the result may be that the only teams that apply are those that are confident that they are the best team for the job. An open process could save funders a huge amount of time filtering through poor proposals.

We should also clarify that we see "changing the rules of the game" as the ideal solution. Basically, we need to use language that leaves room for the possibility that we can start to persuade some grant writers to share their proposals without funders changing anything at all.

I agree that researchers will largely tend to follow the money, so if funders start demanding that proposals are shared in a public manner, people would do so. There are some pretty big caveats to that statement, though.

Firstly, the chance of getting promoted and getting grants also depends a lot on having good publications. There would thus till be a very strong force pulling in the opposite direction, unless both universities and funders stop looking so much at people's publication records.

Secondly, it will enforce the already a very common practice that grant proposals in fact "plan" to do what has already been done (but not published quite yet), and that the money are subsequently channelled into other projects, for which the PI will apply for funding later. PHD Comics explains this well: http://www.phdcomics.com/comics/archive.php?comicid=1431

It will be very hard to convince scientists to share ideas they have not even started to work on yet, in a scientific world where being the first to do something is key.

Firstly, the chance of getting promoted and getting grants also depends a lot on having good publications. There would thus till be a very strong force pulling in the opposite direction

Yes, that's true but not sure it's a problem. Surely one good incentive and one bad one is better than 2 bad ones. Also consider that scientists are already pulled in two directions. For example they know that sharing data is good for science (pulling them to share data), but at the same time they think it may be risky for their career to do so (pulling them to hoard data).

Secondly, it will enforce the already a very common practice that grant proposals in fact "plan" to do what has already been done (but not published quite yet), and that the money are subsequently channelled into other projects, for which the PI will apply for funding later. PHD Comics explains this well: http://www.phdcomics.com/comics/archive.php?comicid=1431

This is really interesting. I have heard this before but I don't think I properly appreciated the implications. If researchers have already conducted most of the research they propose to do, it takes away a significant portion of the benefit we've highlighted in having grant proposals openly reviewed. What do you think @daniel_mietchen?

It will be very hard to convince scientists to share ideas they have not even started to work on yet, in a scientific world where being the first to do something is key.

There is one model that should overcome this problem. The only problem is it may be even harder to get funders to go for this one. Basically, funders would put up a pool of money to reward research idea generation in a particular area of interest. Scientists would contribute ideas (that they were not working on themselves), and there would be some kind of collaborative process to find the best ideas and then develop them into full research plans. There would then be some kind of bidding process where research groups could bid to actually conduct the research. This was actually one of my original ideas but it seemed to radical as a starting point!

Incentives for open proposals

In most cases posting an open proposal will prevent others from scooping you:

1. Posting the proposal establishes precedent. You will likely get cited by highly similar works.
2. Other researches are dissuaded from pursuing your idea because you could beat them to the finish line.
3. Other researchers who were considering the idea will contribute to your project rather than taking the risk of performing the project in parallel.
4. Similar concerns regarding preprints haven't played out. Preprints often bring considerable attention to a project with few disadvantages.

And if others are really better suited to perform your proposal and are willing to invest the time, then you should consider a different project because you're at high risk for being scooped regardless of whether you proposal is public.

The internet and the visibility of your research

However, one incentive trumps all prior considerations. It's an incentive based on a mechanism few scientists understand: the nature of information propagation on the internet.

First, page visits, attention, and incoming hyperlinks are cumulative over time: the older a piece of content, the more it will have been consumed. Once your proposal is posted, you start becoming recognized by the internet as an expert on the topic. It's a highly time dependent process — people stumble upon content in unpredictable ways over a prolonged period of exposure.

Second, activity brings webpages to the top of feeds, increases search engine rankings, and gives consumers a dose of fresh content. When you post a proposal and continue the project openly, you harness increased online activity for your research.

Open science allows you to passively build a brand by letting your content propagate throughout the internet. And from my experience, your personal brand is far more important than journal publications for career advancement.

Already-been-done proposals

Great point @larsjuhljensen that many grant proposals are already partially completed upon submission. In these instances, Thinklab's proposal review features lose much of their appeal.

However, the Thinklab model is the long term solution to already-been-done proposals. By creating a home for open proposals, Thinklab creates the infrastructure to begin rewarding the posting of proposals. The incentives for open proposals will lead to a race to propose first. The proposal curve will shift to more nascent ideas, as proposal creators rush to establish precedence.

Regarding the already-been-done proposals, it is important to appreciate why researchers behave like that. In my experience, there are two major reasons:

1) The grant system is too slow. If you get an idea and apply to get funding for it, someone else will have gotten the same idea, done the job, and published the result by the time you get the money start. You thus need to have some "money in the bank" that allows you to start immediately when you have a great idea.

2) Rigid funding rules. The European Commission in particular has been very bad at making grants that are essentially contracts. If you don't deliver the promised deliverables, you will be in all kinds of trouble and even may have to pay back part of the money that you've already spent! Of course, if what you do is actual research, you cannot know with certainty that things will work. Unless, of course, you have already done them before you make promises.

I am not convinced that Thinklab would make people would rush to establish precedence but putting their idea out in public. It again all depends on how researchers are evaluated by e.g. universities: as long as the evaluators care about who publish first, that is what researchers will optimize for doing. This is also why researchers in most fields are still hesitant to use prepring repositories; what counts is who publishes first, not who deposits first.

@larsjuhljensen consider this:

If funders start requiring openly posted proposals, at first this may enforce the practice of researchers "planning" to do what has already been done. Because if they're going to have their research ideas published on the internet they want to make sure they have a head start in getting to publish the results. I agree with this logic.

However there are a few things that would work against this I think:

1. We're asking researchers to publish their proposals openly and participate in a collaborative process where the community helps them improve the plan. Because this is in public it will significantly increase the awkwardness of pretending they haven't already done what they propose to do.

2. The open collaborative process I just described actually has significant benefit to the researcher. It helps them improve their research plan! And it may even identify fundamental flaws in their proposal. If the researcher has to publish their research plan anyway, they might as well do it before they start the research so they can benefit from the feedback.

3. If researcher A has published a research plan proposing to do X, this will prevent researcher B from submitting the same idea for funding. This creates an extremely strong incentive for researchers to publish their research ideas as soon as possible. If they don't, it means another researcher might publish the idea first and thereby prevent them from obtaining funding based on the merits of that idea.

Ad 1: I think people usually apply for funding for work that they have already done, but not published yet. Pretending to not have done it already is thus not quite so awkward as if it was published already.

Ad 2: I completely see the advantages of getting input to improve a research plan. Question is if it is sufficient to outweigh the disadvantage of risking that someone steals your idea at the stage when it is still only an idea. I would expect people to mostly share proposals that represent incremental work on their current line of research, not proposals that are based on truly novel ideas.

Ad 3: I sure hope you are wrong! If someone proposing an idea and failing to get funding were to prevent anyone else from trying to get funding for the idea, that would be a major roadblock for science. It would effectively grant monopoly on ideas.

Ad 1: I think people usually apply for funding for work that they have already done, but not published yet. Pretending to not have done it already is thus not quite so awkward as if it was published already.

Fair point, but we could simply add a checkbox that says "I certify I have not already completed anything I am proposing to do". The parts that they have already done can be stated as such. I would imagine such a checkbox would work for 90% of people — especially when they have to carry on conversations as if it is true.

Ad 2: I completely see the advantages of getting input to improve a research plan. Question is if it is sufficient to outweigh the disadvantage of risking that someone steals your idea at the stage when it is still only an idea. I would expect people to mostly share proposals that represent incremental work on their current line of research, not proposals that are based on truly novel ideas.

Just to be clear: getting feedback was only one of the incentives. The primary one being that sharing ideas at an early stage would be a requirement of getting funded. Several other meaningful incentives were described by @dhimmel above.

If a research idea is truly novel or brilliant, I could quite easily imagine a future where a large portion of respect and credit is bestowed on whoever shared it first. With that said, I do see that at the present moment a novel idea will give researchers more incentive to keep it secret. But that's okay. I believe Thinklab can be successful even with a small percentage of proposals starting to come online initially.

Also just real quick: it's kind of become common knowledge in the startup community that people have overblown fears of their ideas being stolen, while simultaneously undervaluing the importance of sharing their ideas to get feedback. I suspect the same is true in research.

Ad 3: I sure hope you are wrong! If someone proposing an idea and failing to get funding were to prevent anyone else from trying to get funding for the idea, that would be a major roadblock for science. It would effectively grant monopoly on ideas.

Keep in mind there is not really going to be a hard and fast rule. I'm just speculating about what will happen. If qualified research group A submits a proposal before qualified research group B, I imagine a culture norm will develop where group A gets some kind of priority. (It seems only natural.) However, if group A is not adequately qualified, it also seems natural that funders could choose another research group to do it. As you said, if they couldn't this would be a major problem. This would ultimately be up to funders to decide — but they are, of course, sensitive to researcher opinion.

I think we've covered a lot of ground. One of my big takeaways is that funders really ought to take a moment and think about what they're doing. If researchers have already conducted the work they propose to do, it reveals the entire logic of their funding mechanism as a sham. If Thinklab can start working with science funders as clients, we'll be in a position to push them towards more sensible models. It's clear to me that researchers need to be in a position to share ideas when they have them, get feedback from the community immediately, and start working immediately thereafter.

Sorry for making people read so much!! Good conversation though.