Simplifying the Lambda Architecture

The Lambda Architecture is designed to handle massive quantities of data combining batch- and stream-processing technologies, along with separate analysis and aggregation layers. It proposes that both streaming and batch workloads be run in parallel on the same incoming data, enlisting solutions like Kafka for ingestion, Storm for processing, Cassandra for state and Zookeeper for distributed agreement.

The problem? It’s complicated.

Sure, this approach can meet high-performance needs, but it comes with tremendous cost. Focusing on just the speed layer, getting Zookeeper, Kafka, Storm and Cassandra combined into a reliable fast data engine is expensive in terms of developer time, computing resources and ongoing operational support.

Removing just a single component of a typical “Rube-Goldbergian” Lambda implementation can reduce complexity and cost, but also will make it easier to change the application as business needs change. Look to replace all or part of your Lambda stack with a more integrated solution and this discussion will show you how with a clear example. See how thousands of lines of code becomes 30 when you collapse disparate systems with those that integrate ingestion, state, agreement and processing.

About the Speaker:

Ning Shi is the team lead of one of the Scrum teams at VoltDB (http://www.voltdb.com). His interest in system programming led him to join VoltDB, where he has worked in all areas of software development. Ning studied at Brown University and worked on distributed systems and databases with Andy Pavlo. He joined VoltDB in 2010.

Be sure to bring your business cards for the door prize - a $100 gift card to Amazon courtesy of VoltDB partner IIS Technologies.

IIS Technologies (http://www.iisl.com/) provides advanced IT solutions and systems integration that enable optimal business performance. Recognized for technical innovation and excellence since 1980, IIS Technologies designs and implements big data and cloud solutions utilizing best in class products from the industry’s best strategic business partners.