How we build phpBB: The Next Generation & The Evolution of Rabbit

How we build phpBB: The Next Generation
phpBB is a forum software that has been around since 2000 and is possibly one of the oldest still developed and widely used FOSS [Free & Open Source Software] projects around. In 2007, phpBB 3.0 ‘Olympus’ was released, and since then we have embarked on an adventure for the next major release of phpBB – 3.1 (also called 3.2 and 4.0 in the past).

This talk goes over where phpBB has come from since the Olympus release, the mistakes we’ve made, and finally how we made it to making 3.1 a finished product, built using Symfony Components. As a part of this, I will outline the different development workflows & release cycles we’ve tried and go over what worked and what didn’t in order to help prevent you from making the same mistakes.

Michael Cullum (https://twitter.com/michaelcullumuk) has been an open source contributor and PHP developer for the last nine years. He is a manager at phpBB (https://twitter.com/phpBB) and works a lot with Symfony, Composer and PHP FIG being an advocate of modern web development and FOSS [Free and Open Source Software] project co-operation.

The evolution of a rabbit
In our company we make thousands of calls per day to the major social networks’ APIs. Over the years we’ve implemented various different solutions to the problem of managing which API calls to make in what order, and most recently we had a case where we needed greater control in re-queueing and processing the calls we made to the Twitter API when fetching full advertising account details (campaigns, tweets, etc).

Our MySQL & PHP cron based system was becoming difficult to manage and lacked the flexibility we needed to make changes in future. More worryingly, the database spike and CPU spike every minute when all crons fired at once was taking a major toll on our server performance. The fact that we could only fetch and process a limited number of records from the database each time the cron ran was also a problem and enforced a speed limit on how quickly we could synchronise an account (taking ~30 minutes on average).

We decided to test RabbitMQ as a manageable queue solution as Kacper’s Symfony meetup presentation (https://www.meetup.com/symfony/events/162112862/) had given us some insight into what Rabbit could do for us in our situation. We found ways of communicating with Rabbit from our Symfony application as the producer and chose what we thought was the most lightweight and fastest consumer platform available to us: NodeJS.

As the consumer runs as a lightweight NodeJS service on a separate machine we are able to fetch messages from a dedicated queue server continuously instead of each minute and the queue processes as fast as the API will allow us to make calls, instead of being limited by our own cron schedule. Account synchronisation has been sped up by about 6 times (now takes ~4-5 minutes). The next step is to scale the Rabbit implementation and migrate all API call processing to using Rabbit instead of using Symfony tasks on cron.

This presentation runs through our specific use case; the decisions we made; the results we had and the problems we faced.

Neal Brooks (https://twitter.com/nealio82) is the Twitter product lead for Upcast Social; a Twitter, Facebook and iAds advertising API partner. He's been using Symfony for about 4 years and have been to all three Symfony Live London conferences. Neal's studying Physics with the Open University in his spare time and tweets about science and development stuff as @nealio82 (https://twitter.com/nealio82).

Meet other Symfony developers, join us for discussions, socialising and presentations.

If you'd like to share your Symfony experience, no matter how advanced you are, contact Jakub (https://www.meetup.com/symfony/members/20982981/) and speak at the next meetup!

Drinks and location are kindly sponsored by SensioLabs UK.