addressalign-toparrow-leftarrow-rightbackbellblockcalendarcameraccwcheckchevron-downchevron-leftchevron-rightchevron-small-downchevron-small-leftchevron-small-rightchevron-small-upchevron-upcircle-with-checkcircle-with-crosscircle-with-pluscontroller-playcrossdots-three-verticaleditemptyheartexporteye-with-lineeyefacebookfolderfullheartglobegmailgooglegroupshelp-with-circleimageimagesinstagramFill 1light-bulblinklocation-pinm-swarmSearchmailmessagesminusmoremuplabelShape 3 + Rectangle 1ShapeoutlookpersonJoin Group on CardStartprice-ribbonprintShapeShapeShapeShapeImported LayersImported LayersImported Layersshieldstartickettrashtriangle-downtriangle-uptwitteruserwarningyahoo

MongoDB + ElasticSearch + Redis + Kafka: NoSQL for a Scalable Content Repository

Free beer, swag, edible-delights, fun drinks and lots more!

Thierry Delprat, Chief Technical Officer, Nuxeo

A Database Month event http://www.DBMonth.com/database/mongodb-redis

Developers need to be able to handle an always-increasing amount of data inside the content repository. To meet their requirements, we created a storage adapter architecture that allows us to leverage technologies like MongoDB, Elasticsearch, Redis and Kafka.

This talk will explain the challenges we faced, and how we used opensource NoSQL technologies to address them. The focus will be to share the design and implementation choices and give our feedback on what was good and what did not work that well.

• Context, and expose the challenges

    • large volume with consistency

    • complex and dynamic

    • mass imports

    • ensure scalability

• Scaling the content repository

    • storage adapter principle

    • MongoDB connector

        • integration challenges

        • gains

    • Elasticsearch indexing

        • integration challenges

        • gains

• Side effects of a content repository at scale

    • more data => more events => more jobs

    • leading to more pressure on events and job management infrastructure

• Using Redis as a shared job queuing system

    • explain how we use Redis

    • use case involving locking

    • limits of the Redis approach and impact on queuing

• Using Kafka to scale events queuing

    • explain event processing challenge

    • how we leverage Kafka

• Benchmarks

    • some numbers

    • some tricks learned from the benchmarks

        • UUIDs, round-trips, bulk, etc.

        • consistency over performance (especially in 2.0)

        • feedback on mongoDB sharding

• Opensource cocktail - a real deployment mess!

    • Container based deployment architecture and Rancher

    • Looking for a multi model db, maybe ArangoDB?

Thierry Delprat, Chief Technical Officer, Nuxeo

Thierry Delprat joined Nuxeo in 2005, just in time to handle the migration of the platform from Python to Java. Now as CTO, he guides the architectural development of the Nuxeo Platform.

Prior to joining Nuxeo, Thierry worked for over 7 years at Unilog/Logica, with progressively senior experience across different branches of the consulting company with a focus on Collaboration, Content Management and open source solutions.

He was also a technical architect at Cryo-Networks (infrastructure for online games). Thierry graduated from the Ecole Centrale de Nantes and holds a Master of Telecommunications.

Swag giveaway + food/drinks at 6:30pm 
Power-Networking at 6:35pm 
Presentation starts at 6:40pm

Did you know that innovative use of technology-education is going to end this extreme homeless-epidemic in America in the next few years?

The Techie Youth charity provides free computer-career training to severely-at-risk youth who are statistically-likely to become homeless or incarcerated within the next year or two. Most of the Techie Youth kids are in foster care (children without parents, removed from extremely-abusive/neglectful homes) and approaching the age of emancipation where they will loose the housing provided by foster care. Many of our other kids are living in homeless shelters or youth in juvenile-justice "2nd chance prison-alternative" programs. Techie Youth pivots the life-direction for these kids and helps them begin a career as an IT-professional, and a life as a responsible self-sustaining young-adult.

Please learn more now at https://www.TechieYouth.org

Join or login to comment.

Our Sponsors

People in this
Meetup are also in:

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy