Partnership with customers in product improvement: Testing embedded software products in the field

The features of electrotechnical products are increasingly provided by software instead of hardware. Software is the keyfactor in creating products that are ‘better’ than those from competitors, with respect to quality, price, functionality, reliability and performance. During embedded software development, testing products is a major task, like it is for non-embedded software development. Real-life is the best testing environment. Therefore a testing approach widely applied for embedded products is a method termed ‘Field Testing’. This is the release of a small number of products to customers. The objective of a field test is to make sure that faults and failures are reported and taken out of the product, before normal distribution can start. Schlumberger RPS applies field tests to its products for that purpose.

The actual number of failures reported from field tests is lower than expected. However, failures are found later that should have been reported from field test. Therefore, the effectiveness of the field tests must be improved. Our research concluded the need to emphasise two aspects. Firstly, the need to motivate people to detect and report failures by assigning responsibility for field test results. Secondly, the need to include functionality in the product to enable automatic detection, recording, and communication of failures.