Abstract
Producing an overview of innovative companies in a country is a challenging task. Traditionally, this is done by sending a questionnaire to a sample of companies. This approach, however, usually only focuses on large companies. We therefore investigated an alternative approach: determining if a company is innovative by studying the text on its website. For this task a model was developed based on the texts of the websites of companies included in the Community Innovation Survey of the Netherlands. The latter is a survey carried out every two years that focusses on the detection of innovative companies with 10 or more working persons. We found that the text-based model developed was able to reproduce the result from the Community Innovation Survey and was also able to detect innovative companies with less than 10 employees, such as startups. Model stability, model bias, the minimal number of words extracted from a website and companies without a website were found to be important issues in producing high quality results. How these issues were dealt with and the findings on the number of innovative companies with large and small numbers of employees are discussed in the paper.
Original language | English |
---|---|
Pages (from-to) | 1239-1251 |
Number of pages | 13 |
Journal | Statistical Journal of the IAOS |
Volume | 36 |
Issue number | 4 |
DOIs | |
Publication status | Published - 2020 |
Bibliographical note
Funding Information:Part of this research was supported by the Dutch Ministry of Economic Affairs and Climate Policy. We thank the many colleagues at Statistics Netherlands and students that contributed to the work on which this pa- per is based, provided feedback and especially thank those that assisted us in our search for company URLs. We also thank the employees of the Dutch company InnovatieSpotter for their valuable insights and expertise provided. The anonymous reviewers and editor are gratefully acknowledged for their insightful comments and suggestions that have greatly improved the paper.
Publisher Copyright:
© 2020 - IOS Press and the authors. All rights reserved.
Keywords
- Big data
- concept drift
- Innovation
- text analysis
- webscraping