Success of large distributed IoT implementation depends on the foresight in careful selection of the infrastructure (tools) to design and support a robust and reliable system. Together with trusted data source(s) and strong analytics, there should also be IoT infrastructure tools for reduction of downtime of edge devices/sensors/sensor connectivity.
However, not many PoCs progress to a feasible production system, as the proponents fail to see the big picture. There is a tendency to connect few sensors and devices and use the raw data to show some useful analytics. The availability of Azure IoT PaaS cloud or AWS helps create a secured and connected IoT. It may be a good beginning but lack of foresight in the application and thus, selection of supporting tools or infrastructure, invariably result in challenges which become difficult to overcome in later stages.
One notable challenge is scalability of the product or system. Limitations become apparent as the number of sensors and devices exceed, (say > 250) in the IoT system. Connectivity becomes a major issue as every device has to be brought up to same firmware/ middleware version. The challenges and limitations compound as edge computing is becoming ubiquitous. One has to update the algorithms in the device as well, using Over the Top Architecture (OTA).