Platform IoT and Big Data based on technology Docker
Novelti is a company that offers the latest generation of IoT intelligence, enabling its customers to leverage their intelligent devices to deliver interactive, information-based services.
What was the challenge?
Novelti develops a platform based on Big Data technologies for the analysis of data in IoT environments. It is a real-time intelligence system which, through data obtained through intelligent sensors. It is able to perform an automatic learning for the early detection of anomalies and the recognition of patterns.
The platform is based on high availability clustering technology from which all resources are controlled. The platform required certain features such as high availability and the ability to automatically deploy to have the necessary infrastructure based on the requirements of different analyzes.
Another peculiarity was to have a continuous integration environment and QA for the compilation and execution of automatic tests with version control. It was also necessary to have an advanced monitoring system to analyze the behavior of the platform.
One of the great challenges for Novelti was that its system was able to work from any cloud provider or even from client servers in case it was required.
All this was a great challenge for Nubersia's technical team. But every challenge comes with a great motivation to carry it out.
This was our solution
At the outset, we plan to take the project under our DevOps service, as Novelti was in the development phase of its project. In this way, Nubersia would be a member of Novelti's recognized technical team.
After a study of the needs of Novelti, it was agreed to approach the project through the use of Docker technology for the encapsulation of micro-services autonomously.
The project has been a great challenge due to the use of new technologies. Technology that implies a constant change and improvement. It also implies an adaptation of the systems.
The design has been developed on the Amazon Web Services platform, but as requested by the client, AWS managed services have not been used for agile migration to another public cloud platform. The entire architecture is based on Open Source software.
For cluster management Apache Mesos has been used which provides efficient isolation of resources and exchange through distributed applications. In terms of Big Data analysis and processes, are used technologies such as Apache Cassandra and Kafka.
For automated deployments, solution such as AWS CloudFormation have been used to deploy infrastructure and Marathon framework. All this with the use of Scripting for the automated deployment of services.
For advanced monitoring, the option of Nagios in combination with Carbon, Graphite and Grafana has been chosen. And finally, continuous integration and QA decided the implementation of Jenkins with the support of GitHub for version control.
This project continues under the service of DevOps, since it is necessary a constant adaptation and improvement, because of the growth that Novelti is experiencing, which is not surprising given the vision and professionalism of the team that understands it.
Álvaro Agea - CTO Novelti
"The Nubersia team is exceptional. Creative, ambitious and always ready to collaborate with their clients."