InfoQ Homepage Big Data Content on InfoQ
-
Increasing Users' Data Agency: from BlueSky's AT Protocol to the Local-First Software Movement
Martin Kleppmann, an associate professor at Cambridge and author of Designing Data-Intensive Applications, discusses the evolution of data systems over the last decade, mainly the shift from monolithic databases to modular building blocks. Kleppmann underlines the importance of moving from cloud-centric data storage systems to decentralised data storage similar to Bluesky’s AT protocol.
-
How to Use Apache Spark to Craft a Multi-Year Data Regression Testing and Simulations Framework
Vivek Yadav, an engineering manager from Stripe, shares his experience in building a testing system based on multi-year worth of data. He shares insights into why Apache Spark was the choice for creating such a system and how it fits in the "traditional" engineering practices.
-
Sovereign Clouds, Hyperscalers and European Alternatives: InfoQ Dev Summit Munich 2025 Preview
In this podcast episode, speakers from the upcoming InfoQ Dev Summit Munich 2025 tackle the practical challenges facing European developers caught between regulatory pressures and technological realities. The panel discusses trade-offs between using US cloud providers versus emerging European alternatives, exploring cloud-agnostic architecture strategies and the implications of data sovereignty.
-
Lucas Cavalcanti on Using Clojure, Microservices, Hexagonal Architecture and Public Cloud at Nubank
In this episode of the InfoQ podcast, Charles Humble talks to Lucas Cavalcanti about Nubank’s early architectural choices, including starting with Clojure and microservices. They discuss the challenges of using public cloud for financial services in Brazil, Nubank’s use of Alistair Cockburn's Hexagonal Architecture and immutable architecture, and lessons learnt as the startup scaled.
-
Josh Wills on Building Resilient Data Engineering and Machine Learning Products at Slack
Josh Wills, a software engineer working on data engineering problems at Slack, discusses the Slack data architecture and how they build and observe their pipelines.
-
Ayana Miller on Privacy & Data Governance and Julia Nguyen on Mental Health Tech for Good
In this podcast recorded at QCon San Francisco 2019, Shane Hastie, lead editor for culture & methods, spoke to Ayana Miller of Pinterest on privacy & data governance and Julia Nguyen about her open-source project if-me.org for supporting mental health.
-
Hoang Bao on Ethics, Privacy and Regulation in Software Engineering
In this podcast recorded at QCon San Francisco 2019, Shane Hastie, Lead Editor for Culture & Methods, spoke to Hoang Bao who was the track chair for the Ethics, Regulation, Risk, and Compliance track.