Technopolis Group has completed the independent evaluation of the Austrian Science Fund’s (FWF) 1000 Ideas Programme, a pioneering funding instrument designed to support highly original, high-risk, high-reward (HRHR) research. The evaluation findings have now been published alongside FWF’s response and are directly informing the programme’s future development and its innovative funding processes.
The 1000 Ideas Programme was established to support unconventional, exploratory ideas that may struggle to secure support through conventional funding mechanisms and therefore combines several innovative design features, including anonymised applications, assessment by an international jury, and partial randomisation.
Evaluating high-risk research requires more than conventional metrics
A central challenge for the evaluation was that transformative research is inherently difficult to assess. High-risk projects are characterised by uncertainty, non-linear trajectories, and the possibility of failure. Conventional evaluation approaches focused primarily on publications or short-term scientific outputs may therefore fail to capture important aspects of programme performance.
Technopolis applied a mixed-methods evaluation approach that combined surveys of awardees and unsuccessful applicants, interviews with researchers and reviewers, bibliometric analysis, econometric methods, and analysis of project outputs and follow-on funding.
Particular attention was paid to developing robust quantitative evidence on programme outcomes. The evaluation examined publication profiles, interdisciplinarity, novelty, and disruption of research outputs using bibliometric indicators, and explored whether the programme generated a higher-variance outcome profile consistent with a high-risk, high-reward funding logic. Econometric analyses further assessed differences in citation performance and outcome dispersion between funded and unfunded applicants and between the 1000 Ideas Programme and conventional FWF funding instruments.
Interviews with researchers, reviewers, and programme stakeholders provided essential insights into how high-risk research unfolds in practice, how apparent failure is transformed into learning, and how exploratory projects contribute to new methods, collaborations, and research directions that may not yet be visible through traditional metrics.
Key findings
The evaluation found that the 1000 Ideas Programme successfully attracts and supports highly novel and exploratory research ideas. Awardees reported high levels of methodological experimentation, conceptual innovation, and interdisciplinary exploration. Many projects generated valuable scientific insights even when original hypotheses or planned approaches did not succeed as expected.
The findings also indicate that the programme’s innovative selection mechanisms are broadly supported by applicants and stakeholders. Anonymisation and partial randomisation were generally perceived as enhancing fairness and helping to reduce biases associated with reputation and institutional prestige.
Importantly, the evaluation found evidence consistent with the programme’s intended high-risk profile. Funded projects show greater variation in outcomes than projects funded through more conventional schemes, suggesting that the programme succeeds in supporting a portfolio that includes both uncertain and potentially transformative research trajectories. The evaluation also found strong evidence of follow-on effects, with funded researchers securing substantial additional research funding and developing new research directions beyond the original projects.
Informing the future development of the programme
The evaluation was commissioned not only to assess programme performance but also to inform future decisions about the design of the 1000 Ideas Programme. The findings were used to examine whether the programme’s objectives could be achieved within existing funding instruments or whether a dedicated high-risk funding stream remains necessary.
The evaluation concluded that a stand-alone programme continues to play an important role within the FWF funding portfolio. While selected elements of the programme’s innovative assessment approach could potentially be transferred to other funding schemes, the combination of anonymisation, acceptance of failure, partial randomisation, and a strong focus on exploratory ideas is most effective when embedded within a dedicated high-risk funding instrument with clearly differentiated objectives.
The evaluation has already informed concrete decisions by FWF regarding the future development of the programme. In its response to the evaluation, FWF announced several measures aimed at strengthening the programme further.
Further Information
Evaluation report
The final evaluation report of the FWF 1000 Ideas Programme is available on Zenodo:
FWF response
FWF has published a statement outlining how the evaluation findings will inform the future development of the programme:
