whitepaper
Accelerating IoT Scalability and Interoperability with MQTT
Overview
IoT has reached a tipping point. It is no longer the new kid on the block with everyone trying to understand the use case and approach. Organizations are aggressively pushing for connected devices in the millions, with minimal time to connect. Device data is sent to the IoT platform for storage and initial analysis to enable market differentiation, reduction in cost and compliance with market vertical legislation. This creates pressure on IoT platforms to provide secure, scalable and innovative solutions that allow organizations to meet their business needs.
MQTT has emerged as the protocol of choice due to its ease of adoption for both device manufacturer and solution provider, reducing cost and time to market. This paper focuses on MQTT and how Cumulocity; trusted by enterprises globally and managing tens of millions of connected devices worldwide; has leveraged over a decade of IIoT experience to create a third-generation Managed MQTT Service, validated for 60 million devices and one million messages per second.
Ready to start building IoT solutions that change the game?
Start fast.
Scale easily.
Evolve with the market.
Why MQTT?
MQTT is a lightweight, efficient and open protocol that is easily implemented by small devices, and does not impose any restrictions on message payloads. MQTT clients can be written to a very small footprint, making the protocol ideal for resource constrained devices; the additional data in the message header, enabling the MQTT Broker to understand what to do with the message, is likewise kept to an absolute minimum.The protocol is lightweight and designed for efficiency, performing reliably even on low-bandwidth or unstable networks. It supports real-time communication, scales easily to large numbers of devices, and tolerates clients that connect and disconnect frequently, making it ideal for distributed and challenging environments.. These factors reduce the software development overhead and have driven the use of MQTT, since its inception in 1999, to dominate the IoT communications space to the point where it is now the IoT communications global standard.
IoT Challenges
One of the biggest headaches for IoT solutions is to solve the problems of connecting millions of devices, securely and with a minimum of complexity. Why is this a headache? Let’s look at each of these separately as they relate to IoT.
- Scale. The number of devices has grown from an estimated 3.8 billion in 2015 to an estimate of 19.8 billion today, with a forecast of 40.6 billion devices in 2034. This growth is driven by businesses realising that by connecting their assets they are able to realise significant benefits such as a reduction in the amount of materials used, real time performance monitoring and remote asset management which in turn reduces overall costs. The result of this growth is that we see a single IoT tenant connecting millions of devices, each of which might have hundreds of sensors, all sending data to the IoT platform where it needs to be stored and analyzed. This puts pressure on the scaling of the IoT solution across all aspects of the infrastructure from the underlying hardware to the hosting software and the solution itself.
- Security. Device manufacturers are grappling with multiple problems that need to be addressed when connecting devices to an IoT solution, both on premise or in the Cloud. The most pressing of these is ‘how do I know you are who you say you are?’ A simple enough question you might think, but should a compromised device gain access to the IoT solution undetected then the bad actor has the potential to wreck havoc. And let us not forget the requirements of the Cyber Resiliency Act which also requires comprehensive device software lifecycle management.
- Simplicity.Many devices are held in locations where the people responsible for their installation and management may not have IT skills as their primary skill set. In these instances, minimal to no on-site setup is a must to ensure adoption. Added to this is the scale requirement, as discussed above, for connection and management this means the minimum of manual intervention with the ability to on-board devices in bulk, with remote management and diagnostics.
We will discuss these areas in more detail later in the paper as we look at what it takes to make MQTT and your IoT solution scaleable, secure and fit for the future. You will discover:
- How to navigate the extravagant claims of scaling and performance to understand what is achievable and how that relates to your needs.
- Key challenges around security and how to solve them.
- How to focus on simplicity whilst dealing with the tension of the complexity of the device landscape.
- How Cumulocity’s 3rd generation Managed MQTT Service delivers a proven, IoT-first architecture—validated for 60 million devices and one million messages per second.
If you are serious about building IoT systems that go beyond pilots to scale globally, this is the paper you need. It doesn’t just explain why MQTT is the standard—it shows you how to implement it in a way that drives resilience, efficiency, and long-term competitive advantage.
MQTT in IoT
Based on our experience with hundreds of IoT projects, IoT places specific requirements on MQTT that require attention and a slightly different approach compared to the generic message broker used in a standard messaging environment.
Figure 1 below shows the high-level architecture of a traditional ESB solution.
Figure 1. Traditional ESB message broker architecture
In this architecture, a relatively small number of applications are treated equally by the broker (“a symmetrical pattern”). Peer-to-peer communication between the applications is common, and all components are typically deployed on secure, managed enterprise-grade infrastructure.
Conversely, Figure 2 below shows the relevant components of an industrial IoT platform architecture, ingesting data from a large number of geographically dispersed devices.
Figure 2. IoT MQTT endpoint architecture
In this architecture, the message broker must handle each side in a different way (“an asymmetrical pattern”). On one side, there may be millions, or even hundreds of millions, of connected devices. Peer-to-peer communication between devices is uncommon, and in some cases can be detrimental to security, where devices are deployed in environments where they cannot be completely secured. The MQTT protocol engine is shown logically separated from the message broker, because the platform will also support devices using other IoT device protocols, with all device data ultimately published onto a shared messaging backbone.
On the other side, there are a smaller number of applications, which need to process data in a highly scalable and reliable way. Communication wise, the IoT broker needs to optimize the device-to-platform and platform-to-device traffic, while device-to-device traffic is not a standard requirement.
In summary, both sides of the MQTT broker have very different requirements, requiring features in the IoT MQTT broker that go beyond traditional MQTT brokers. Table 1 below shows some of the specific differences in scale and robustness that must be taken into account.
Table 1. Comparison of application and IoT device connections
In the rest of this section we will look at the various aspects of Scale, Security and Simplicity in more detail.
Scale
How to scale without spending a fortune on the infrastructure.
As previously noted, Industrial IoT is expected to handle very large numbers for both connections and throughput.
In order to achieve these requirements, an IoT-focused MQTT endpoint needs to focus on some specific patterns:
The primary use case is a “fan-in” scenario, where we see millions of devices pushing data into the platform that will be consumed by relatively few applications.
Message traffic in this scenario is highly asymmetric; in a typical IoT deployment, device-to-cloud communication outweighs cloud-to-device traffic by 100 to 1,000 times — and in some cases even more. This means that an IoT MQTT broker must make different architectural decisions around how to handle the data in each direction to support its scalability requirements.
To scale applications for handling millions of messages per second, an IoT broker needs to provide two features: partitioned subscriptions and application instance management. These approaches to pub/sub enable scaling without undue pressure on the underlying infrastructure.
- Partition subscription: the incoming device data is divided into separate logical partitions; typically this is done using an MQTT Broker as the front end with the persistence handed off to an underlying messaging bus. From the perspective of the device MQTT client there is nothing unusual, it publishes to a topic in the standard way; underneath the covers the handling of the topic structure leverages the technology of the messaging bus to minimise resource usage and manage the message life cycle.
- Application Management: each logical partition must have an associated application instance that is responsible for processing the messages. The application is responsible for the stateful logic; achievable by the one to one relationship of the partition and the application.
Linked to scaling is the question of performance which is often conflated with scaling. Performance is the responsiveness of a system under a given workload; typically the information provided for MQTT brokers is the throughput. It is this figure that can be difficult to understand precisely what is being measured; in many instances the number of devices concurrently connected is used as the scaling figure but with little information provided on how this is achieved.
Things to look out for:
Scenario: Standard MQTT scenarios are not where most IoT solutions will sit, so you need to make sure you look for the appropriate scenario. You want to see large numbers of devices publishing to a considerably smaller number of applications (fan-in), with much less traffic flowing from the applications back to the devices.
Marketing counting: Many web sites will use “marketing maths”, for instance publishing 100 messages per second from 20 publishers which are fanned out to 1,000 subscribers and claimed as a throughput of 2 million messages per second. While it is true that 2 million messages are delivered to subscribers, there are only 2,000 unique messages per second in this use case. Consider a more IoT-focused use case where 1,000 devices each publish 2,000 messages per second, with each message delivered to one of 20 subscribers using a partitioned subscription. The headline throughput is exactly the same in both cases — 2 million messages per second are delivered to subscribers — but in the fan-in scenario all of those messages are unique and the total amount of work done by the system is much larger.
Quality of Service: The QoS level used will also influence the performance and scaling that can be achieved. QoS 0 (fire and forget) can deliver significantly better throughput with the risk of message loss, which may be completely acceptable in some IoT use cases. QoS 1 (at least once) has more overhead from acknowledgement processing and message persistence and will have a lower maximum performance for a given use case. QoS 2 (exactly once) is not a quality of service seen in IoT applications.
The bottom line here is to check what is important to you and your use case and see if information on the infrastructure used to achieve the figures being claimed is available.
Security
How to make yourself secure without too much complication
Security in the IoT space is critical. Many of the market verticals enthusiastically embracing the transformation that near real-time analysis can bring are also in heavily regulated markets; for instance healthcare, utilities and critical national infrastructure.
An IoT MQTT broker requires three security features: multi-tenancy, device isolation and PKI-based authentication, preferably with a built-in Certificate Authority (CA). Cumulocity’s MQTT Service incorporates all three, delivering compliance with GDPR, ISO 27001 and SOC 2 standards**.**
Devices are typically the weak link in the security chain: In contrast to applications running in tier 4 data centers, devices are deployed anywhere and can be compromised more easily. It is therefore vital to ensure that if a device should be compromised, the damage is limited and does not provide the bad actor a gateway to run amok in the wider IoT solution.
This requires a feature which we call device isolation. At a high level this means that a device can only send data to its own topic space which cannot be subscribed or published to by another device. From the device’s perspective, it is the only client connected and it has exclusive access to the entire topic space. With device isolation, if a single device is compromised, the attacker cannot, for example, publish firmware update requests to other devices, because it is not possible to publish to the topic space of the other devices.
Device isolation has been proven to improve security without limiting functionality. While it is not possible to have “peer-to-peer” device communication with this model, this is not a problem in practice: First, peer-to-peer communication can be emulated by implementing it on the application side. An application can retrieve the messages from devices, and publish them to other devices. Second, peer-to-peer communication in device networks with multiple firmware versions and related MQTT payload variants would require that each device understands all payload variants (historic and future), making it difficult in practice. And thirdly, peer-to-peer communication is an exceptional requirement, in our experience it is needed in less than 5% of the IoT solutions.
Figure 3 below shows how devices and tenants can be isolated from each other through the use of per-device MQTT topic spaces. On the application side, not shown in the diagram, applications operate in the context of a single tenant and have access to all of the device message data within that tenant.
Figure 3. Device and tenant isolation
A discussion on security would not be completed without looking at certificate- based authentication. This approach to authentication is currently considered to be best practice, however in reality the uptake in the IoT space is slow due to the barriers to entry. Certificate based authentication comes with additional costs, such as purchasing the certificate from a trusted Certificate Authority and certificate lifecycle management which requires a PKI solution, which requires a license and time and effort to administer. Cumulocity provides both options; a built-in Certificate Authority for simplified certificate lifecycle management, and seamless integration with external PKI systems—ensuring secure identity management at scale.
Simplicity
How to keep it simple so users want to use your devices
With security and scalability handled by Cumulocity’s IoT platform, all that remains is connecting your devices—a process made faster and simpler through low-code configuration and out-of-the-box tools like Device Management, Cockpit and Analytics Builder. This is where the IoT solution needs to provide 80% of what you need with the customer specific requirements being addressed by configuration or coded extension to the solution itself.
Some IoT platforms provide a wide range of capabilities that need to be considered before you can start to send data; other platforms will provide a ready to go solution. The key to choosing the IoT platform that is right for your organization is understanding what you need to achieve and how much of that is provided. It is also useful to understand where you see your journey with IoT in the next 5 years as switching platforms can be an expensive project, both in terms of time and impact to your bottom line. As part of this decision making process look at how the IoT platform is architected and whether it is first and foremost an MQTT Broker or first and foremost an IoT Platform. If you are looking for a platform that will provide device management and software lifecycle management with state of the art analytics then you need to focus your search on an IoT platform first solution. If, however, your primary focus is on integration with existing solutions and applications then an MQTT Broker first solution may be sufficient for your needs.
Cumulocity MQTT Service - Next Generation
We will now look at how the Cumulocity MQTT Service, the next generation of our MQTT story, can help with your IoT journey; providing a solution to your immediate needs whilst supporting your requirements into the future.
The Cumulocity MQTT Service has been architected using our deep understanding of both MQTT (team members were part of the original team responsible for bringing MQTT to market) and the Industrial IoT market (Cumulocity was founded out of the team responsible for the Nokia IoT Platform). This expertise has allowed us to create an industry leading service that provides the best of MQTT focused on the needs of the IoT market today.
Scale
Cumulocity’s MQTT Service has been purpose-built specifically for IoT, designed to meet the demands of customers connecting millions of MQTT devices today, with the expectation of scaling to tens of millions as their IoT deployments and business operations expand. Our customers have provided input to the design, ensuring we are able to scale while limiting additional cost for infrastructure.
A generic messaging bus provides the underlying persistence and management layer for Cumulocity’s MQTT Service. This architecture means:
- We are not tied to a single protocol such as MQTT, we can use the same approach for other protocols such as LwM2M, providing a coherent behaviour for all devices connecting to Cumulocity.
- Different protocols can be supported on either side of the messaging bus, meaning MQTT can be used by devices and applications can use a different protocol to subscribe to the data.
- High volume data can be streamed to a single application efficiently.
Cumulocity has been tested with 60 million connected MQTT clients publishing an aggregate 1 million messages/second into the message broker. The same 1 million messages/second are consumed by a single application endpoint using a non-MQTT protocol. Other benchmarks might claim this as 2 million messages/second. While this can be a useful metric for a fan-out use case, it exaggerates the overall throughput of the system. Therefore, we prefer to measure the number of distinct messages successfully delivered end-to-end from the device to the IoT platform.
Security
Cumulocity is proud of the approach we have taken to security, from the multi-tenant approach built into the platform architecture to the constant innovation on meeting the latest security challenges.
The MQTT Service extends the multi-tenant architecture, ensuring devices are identified to a tenant as part of the authentication process. Cumulocity tenants are completely isolated from one another, this means that two devices in different tenants can connect using the same MQTT client identifier and remain completely isolated, with no interference from one device to the other. The MQTT Service extends this by also ensuring that each device has its own topic hierarchy within the tenant, thus ensuring that should a device be compromised it is not possible for that device to cross compromise other devices.
Certificate based authentication is the preferred method to authenticate your MQTT devices. In order to lower the barrier to entry for certificate based authentication Cumulocity provides a Certificate Authority to sign certificates and provide certificate lifecycle management. If you already have a preferred PKI in your organization, the certificates can be used as well. These features work in exactly the same manner for the new MQTT Service as they do for other protocols when connecting to the Cumulocity platform. A further enhancement when using the MQTT Service is the additional requirement to set the Common Name (CN) of the certificate so that it matches the MQTT client identifier, thus preventing a rogue device from impersonating a valid device.
In addition to device isolation the Cumulocity family of deployment options provide an end to end approach to security, allowing adoption of the one most appropriate for your business case:
ThinEdge
The MQTT service opens up a lot of exciting new use-cases for thin-edge.io users that were not possible before.
The flexibility of the MQTT service to allow clients to use custom topics and payload formats, will enable users to rethink how they handle data both on the device and in the cloud. Users will be able to reduce the consumed bandwidth by publishing telemetry data in binary formats (such as protobuf) whilst having a well-defined model of the data to assist with data normalization.
It also enables easier integration of 3rd party sensors and devices where the data can be sent transparently via thin-edge.io to the cloud, without having to decode the data on the device first.
Edge
Designed for environments that need to limit internet exposure, Cumulocity Edge delivers the same enterprise-grade functionality and security locally as in the cloud, enabling consistent operation across hybrid deployments. Nick Ponomar not clear to me if we can weave a story around MQTT but I definitely think we should have Edge as part of the overall story.
Cloud
Cumulocity has a Certificate Authority feature which allows users to take advantage of the ability to create a trust anchor, sign certificates and manage the certificate lifecycle, all within Cumulocity. This lowers the barrier to entry to using certificate based authentication for all our customers.
The Certificate Authority (enabling certificated based authentication) coupled with out of the box MQTT device isolation makes Cumulocity one of the most secure IoT Solutions on the market today. No additional configuration or specialist knowledge is required for you to be sure that your device data is secure when sending to Cumulocity, no matter the form factor.
Simplicity
A key takeaway from our discussions with our customers is that they want the minimum of decision points and configuration when connecting devices. We listened and architected the MQTT Service accordingly.
Cumulocity does not impose any restrictions on the MQTT client identifiers, topic schema, or payload format used by devices connecting to the MQTT Service. Most devices should be able to connect and start publishing after configuring the server address and authentication details, either a trusted certificate or username and password. Of course, most devices will not natively support the Cumulocity domain model, so a mapping must be provided. This can be implemented by a custom microservice, by existing tools such as the Dynamic Mapper, or by the upcoming AI-powered Cumulocity Data Preparation capability.
Data Preparation will provide a number of features out of the box that will: automatically register new devices with zero configuration to the Cumulocity platform,. Once your device has been integrated with Cumulocity, Data Preparation will also then allow device data to be mapped to the Cumulocity format with Javascript Smart Functions. The Javascript smart functions can be generated using AI, either from a description of the protocol or by dynamically inspecting messages from the device. You can also create the smart functions manually should you wish to do so.
On the other hand, devices that already support the Cumulocity domain model—using the SmartREST or JSON-over-MQTT formats—can continue to operate exactly as they do today. A device can even publish SmartREST messages to SmartREST topics and “generic” messages to other topics, within the same MQTT session. Currently, messages using the Cumulocity canonical format are bridged to the existing MQTT endpoint in the Cumulocity core. The long-term roadmap will see this endpoint decoupled from the core, and all MQTT traffic handled by the single MQTT Service endpoint.
Cumulocity’s multi-tenancy and the device isolation of the MQTT Service allow devices to be onboarded without worrying about conflicting topic schemas or potentially unwanted interactions between devices. Mutually incompatible devices can co-exist within the same tenant as long as the mapping layer knows how to handle each type of device.
Conclusion
Cumulocity’s Managed MQTT Service is purpose-built for IoT, combining proven scale (validated for 100M devices), 99.9% uptime reliability, and enterprise-grade security to simplify device onboarding and accelerate deployment.
Cumulocity’s service is not a retrofitted enterprise broker. It is purpose built for IoT, with integration to device management, analytics, and automation.
Secure by Default Device isolation prevents one device from accessing another’s data. TLS encryption protects all communication. Certificate to device binding enforces identity assurance from the moment of connection.
Proven Scale and Performance The system is validated for 100 million connected devices and one million messages per second. Horizontal scaling ensures growth without redesign.
Enterprise grade messaging backbone The messaging backbone provides a number of benefits, over and above the ability to scale without bloating the infrastructure required. Device payloads are available for processing by applications in a protocol independent way, regardless of the protocol used by the device. Applications can also use enterprise-grade messaging features that are not standard with a lightweight, device oriented protocol like MQTT, such as: key-shared subscriptions, dead-letter queues and other error-handling mechanisms, cumulative acknowledgements, tiered storage, etc.
Easy Device Onboarding Thin Edge integration simplifies onboarding, especially for certificate authenticated devices. The SmartREST bridge will further unify device management through a single MQTT endpoint.
Cumulocity Data Preparation will also provide ease of device registration with automatic data mapping.
Competitive Edge Cumulocity differentiates through IoT focus, security, scalability, ease of onboarding, and fifteen years of IoT experience. Unlike generic brokers, it is designed for millions of devices, frequent reconnections, and multi protocol integration.
Call to Action
Now is the time to prepare and position your organization for IoT leadership. Whether you are connecting thousands or millions of devices, the right architecture will determine how quickly you can innovate and how effectively you can operate at scale.
Contact the Cumulocity team today to start building your MQTT strategy. We can help you design, migrate, and deploy a world class IoT messaging solution that meets your needs today and adapts to your future ambitions.