Rawpixel (pixabay.com)This multi-cloud project was created to put into practice SMPC (Secure Multiparty Computation) techniques, TOR (Onion Routing) network as well as privacy preservation mechanisms (Zero-trust) to test the hypotheses of the thesis. The application was used during the 2016 U.S. Presidential Debates by measuring in real time, all the tweets on the Twitter platform on the topic (analysis of topics/themes and favorable or unfavorable sentiment towards one of the candidates). During the second debate, in the space of 48 hours, the service processed 1.78 billion tweets, representing more than 15GB, spread across AWS/GCP/Azure/Heroku clouds. To handle this volume with little latency, an auto-scaling mechanism was implemented. Finally, I have scripted the deployment and execution in order to facilitate the DevOps method.
Technical Stack Details:
- AWS, GCP, Azure and Heroku
- Vert.X 3.3.3, with Hazelcast 3.7.2
- Maven 3.5.1 and SLJF4j 1.7.21
- JUnit 4.12, REST-assured 3.0, Assertj 3.5.1
- JClouds 1.9.2
- Kubernetes 1.14 (May 2019) and the first tests of Ubernetes (K8s Federation)
- MongoDB 1.50.5
- Stanford NLP 3.6
David Aparicio
DataOps
I fell into it when I was 👨👦👦, so I can define my start in IT. Since then, new tech have been developed..
