The Internet of Things (IoT) is no longer a fiction. It is quickly becoming a reality. By the year 2020, an estimated 20 billion devices will be connected and be communicating with each other, whether they are smart home devices, appliances, wearables, connected healthcare devices, or connected cars.

The IoT is going to bring tremendous conveniences, efficiencies, and business opportunities. However, to ensure the success of the devices - including the tiny sensor - need to be highly reliable, low in cost and high in manufacturing throughput.

Traditional test methods are simply too expensive and time-consuming to meet these conflicting requirements.

Why Test IoT Devices?

Testing is crucial for IoT applications. Many IoT use cases involve mission-critical applications such as monitoring of critical processes in industrial applications or tracking of a patient’s vital signs in medical applications.

Without proper testing, devices can fail or operate in real-world with a performance that does not match the value measured in a laboratory environment. In mission-critical applications, such failures or unexpected performance can be life-threatening.

On the other end of the spectrum, proper testing of IoT devices enables:

  1. Filtering of faulty, marginal components or devices
  2. Assembly issues:
    Wrongly installed components, component placement misalignment, improper orientation, cold solder joints and incorrect PCBs.
  3. Early identification of performance drift:
    So that corrective action can be taken before it is too late, to avoid major failures and potential product recalls after a product ships.

Ultimately, the goal of IoT device test is to improve the quality and reliability of a product. It helps protect a company’s reputation and ensure customer confidence.

Imagine for a moment what would happen is that testing was not carried out properly. The consequences of a product failure could be significant. It might entail enormous amounts of time to troubleshoot or repair the device. And, if the product was already shipped to customers before the problem was found, the impact could be even worse, involving a product recall and negatively impacting the company’s brand.

Three Key Parameters For Testing IoT Devices

3 key parameters for testing IoT devices

IoT device test doesn’t need to be complex or expensive, assuming you know the critical parameters that will impact your device performance. IoT devices typically use a pre-tested RF module and the performance of that module is guaranteed by the module vendor. Comprehensive parametric testing has already been performed at the chipset or module level. However, the device’s design and how the chipset or module will be used in the actual design may alter the performance somewhat.

As an example, when many components are squeezed into a small space due to board space limitations, crosstalk and mismatch can occur that causes a marginally passed design to fail during volume manufacturing. Mechanical housing dimension tolerance can also be problematic.

Testing the device with or without a cover can change the electromagnetic wave propagation pattern and cause a performance change when tested in different configurations. Assembly issues such as the amount of solder applied, component value tolerance and placement misalignment, and missing components can also result in device failure.

Understanding the key test parameters and how they impact your product performance is critical to helping you to decide the test coverage.

1. Transmitter Power and Receiver Sensitivity Tests

To ensure your product meets the specified transmission range performance, you will need to verify both transmitter power and receiver sensitivity. The transmitter power measurement ensures the device is outputting sufficient power to reach the receiver, but not so high that it violates regulatory requirements or consumes unnecessary power that drains the battery. The receiver sensitivity test checks the level of signal strength that must be present at the receiver in order for it to decode the data correctly at a specific packet error rate.

2. Packet Error Rate (PER)

The PER measurement is commonly used to measure receiver performance. Many wireless systems have an error detection and correction technique that checks if the data sent by the transmitter is correctly received by the receiver. If the data is received correctly, the receiver sends an acknowledge signal to the transmitter so the next data packet is sent. If the data is corrupted, the transmitter must resend the data, resulting in a slower data transfer rate.

For instance, in a Bluetooth transmission, a failure to receive an acknowledgment from the receiver will cause the transmitter to resend the data, in turn, impacting the data throughput of the system. Hence, it is important to check the PER performance of the receiver to ensure a good data transfer rate.

PER is performed by counting the number of packets received by the device out of a series of consecutive test packets transmitted by the tester. This test is normally performed at the most sensitive level.

3. Sensitivity Search

Another related receiver test is sensitivity search. This test is performed by adjusting the input power to the receiver over the entire power range, to find the power level at which the PER starts to fail. This test is commonly used to characterize the performance of the receiver.

Because this test can be very time-consuming, having to step through many power levels to find the most sensitive one, it is typically not part of the standard production test. However, it is useful to randomly check a few devices in production to detect performance drift or catch any potential quality issue.

Warning: Test Challenges Ahead

Now that you better understand the critical parameters that can impact your IoT device’s performance, the next step is to define your test strategies. That test strategy will vary, depending on the wireless standard being utilized. IoT device manufacturers dealing with the Bluetooth Low Energy (BLE) and Wireless Local Area Network (WLAN) standards, for example, face a number of critical challenges when developing manufacturing test strategies.

Recommended for you: Bluetooth Mesh - IoT will never be the same again >>


The top four challenges include:

1. Lack of a wired connection

Many IoT devices are low cost and compact in size. They have integrated antennas and the whole device is encapsulated in a small housing with no access to a wired connection for conducted measurements. With no connection, designers and manufacturers are challenged to figure out how to perform transmitter and receiver measurements.

2. A chipset specific driver is required

Most BLE and WLAN testing methods require a chipset specific driver to put the device into test mode. This is tedious and time-consuming as different chipsets require different drivers. Developing and managing many versions of drivers is a nightmare for most test engineers. It takes months to develop a driver and requires close collaboration with the chipset vendor to develop the test method. Additional manufacturing steps are then required to load and unload the test firmware.

3. Long test time

Existing test methods that use test mode (also known as a non-signaling test) incur a long test time. This method typically requires the operator to flash in test firmware, run through a long list of transmitter and receiver tests, remove the test firmware, and load the final shippable firmware. This results in a long test time that adversely impacts the cost of test and manufacturing throughput.

4. Inability to capture defects that occurred after final assembly

Because you are testing the product when it is not in its final form and not with the final shippable firmware, you run the risk of not capturing defects that happened during the final assembly process. You also run the risk of shipping a defective product to your customers.

Solving IoT Device Test Challenges

Fortunately, these challenges can be easily solved with a new way of testing that utilizes an innovative over-the-air (OTA) signaling tester (Figure 1). Using the tester, both transmitter and receiver measurements can be performed over the air, leveraging the standard waveform packets as defined by the Bluetooth or 802.11 specifications.

X8711A IoT tester

Figure 1. The Keysight X8711A IoT device functional tester enables quick, easy, and cost-effective over-the-air transmitter and receiver measurements on IoT devices.

As an example, BLE devices will transmit at the 3 advertising channels. By measuring these advertising channels, an OTA signaling tester can obtain the needed transmit power measurements. For PER measurements, the tester sends messages to the device to stimulate it to respond. By counting the successfully received response signals, the tester can then obtain the PER of the device (Figure 2).

Using this method, a check of the transmitter and receiver performance can be completed in less than 30 seconds, offering a significant improvement in both the overall test time and the cost of test. Moreover, these tests can be carried out without the need for a proprietary chipset driver or access to the digital communication ports (UART or USB) to control the device.

Transmitter and receiver tests can be performed using final production firmware, thereby removing the additional steps of flashing in and removing the test firmware. The result is faster overall test time. The device is also being tested in its final packaging and firmware, which helps ensure that the majority of manufacturing defects are captured and guaranteeing that the product’s performance will be as expected in the real world.


The number of IoT applications being utilized worldwide are rapidly expanding. This growth brings tremendous opportunities to IoT device manufacturers, but it also creates a number of huge challenges. The key to success for these manufacturers lies in their ability to bring their products to market fast, cost effectively, and with excellent quality.

A new category of functional test solution that carries out performance testing over-the-air now offers device manufacturers an ideal way to meet these challenges head-on. With this type of solution, both designers and manufacturers can perform accurate transmitter and receiver tests quickly and cost-effectively to guarantee a high-quality product.

Click on the image and download the IoT App Note "A Flexible Test Solution for Internet of things (IoT) devices with ASK/FSK Modulation" and get on top the IoT overview poster for free. 

Keysight IoT Solutions AppNote

About the Author

Sook-Hua Wong is the Industry Segment Manager in Keysight Technologies’ General Electronics Measurement Solutions group. Prior to this role, she was a product planner, responsible for strategic planning and product portfolio development for RF/Microwave power meters and sensors. Sook-Hua holds a B.S. in Electrical Engineering from the University of Technologies Malaysia (1999) and an M.S. in Electronic System Design Engineering from the University of Science Malaysia (2003).