add-memberalign-toparrow-leftarrow-rightbellblockcalendarcamerachatchevron-leftchevron-rightchevron-small-downchevron-upcircle-with-crosscomposecrossfacebookflagfolderglobegoogleimagesinstagramkeylocation-pinmedalmoremuplabelShape 3 + Rectangle 1pagepersonpluspollsImported LayersImported LayersImported LayersshieldstartwitterwinbackClosewinbackCompletewinbackDiscountyahoo

Quarterly Large Scale Production Engineering (LSPE) meet up

Event Agenda and Speaker info

10:30 - Registration

10:45 - Tea

11:00 - Why LSPE-IN

- Unni Arvindakshan, Senior Director, Yahoo!

11:15 -  Layer 3 Load Balancers and Flattened Networks in large Datacenters

- Damodharan Rajalingam, Principal Engineer, Yahoo!

Direct Server Return (DSR) load balancing is a common way to distribute networktraffic using an approach that currently requires the load balancer and all hosts behind the Virtual IP (VIP) to be within the same Layer 2 broadcast domain. This is a severe limitation that hinders scaling VIPs beyond a single contiguous subnet. To overcome this limitation, Yahoo developed a method to perform DSR load balancing across Layer 3 boundaries (``L3DSR''), a solution that allows Yahoo! to serve up to ten times as many VIPs on a single hardware Load Balancer compared to other Layer 3 load balancing methods. The code is open sourced. Damu will guide us through this innovation.

12:00- Playing with IPv6

- Bhaskar Sengupta, Principal Engineer , Yahoo ! & Debansu Saha, Principal Engineer, Yahoo!

In this session we would demonstrate how to play with IPv6 using your dial up Internet connection and GNU/Linux , Mac Box. This would be a hands on session . You are free to experiment with us using your laptop and datacard .

12:45 - Lunch

1:30 - Performance Engineering of Large Scale Applications

- Sathya Narayanan Nagarajan, Principal Engineer, Tyco Retail Solutions

This talk is about taking an engineering approach to proactively identifying and solving performance problems of large scale applications. It provides a holistic view of Workload modeling and Workload characterization, capacity assessment for the planned workload model, identifying bottlenecks through profiling. It will also talk about the challenges we encounter based on real life case studies.

2:15 - Monitoring Large Scale Systems

- Aveek Misra, Principal Engineer, Yahoo!

Monitoring in large Internet companies needs to cover many types of systems: serving systems, databases, application, caching tiers, batch processing systems, network, revenue, abuse and targeting algorithm behavior and so on. The common solution is to develop dedicated systems, either built in house or assembled from open-source components. Such system then need to be independently scaled, operated and customized to interact with other in-house systems (like node databases, incident management etc.). All monitoring systems need to solve the same common problems: large scale geographically distributed data collection, real-time processing and aggregation, storage, display (combination of built-in reports and ad-hoc querying and presentation). Tightly integrating these layers reduces the management overhead but makes the solution harder to reuse for another type of system monitoring. At Yahoo we have developed a monitoring platform as a service on top of which domain specific extensions can be built. This talk would cover the why and how of monitoring for large scale systems. Additionally it is very important to understand how to structure and look at your monitoring data to get the correct insights. It would also talk about how to correctly do "eyes-on-glass" monitoring.

3:00 - Open forum

3:15 - onwards - Tea, snacks and networking.

Parking directions

Enter into the EGL park from Koramangala Indiranagar intermediate ring road. The first building on your left is the venue, Yahoo!. This is the Sunriver building. Take the first left to get into the parking lot. Ask the security for help and tell them that you have come for the LSPE event at Yahoo and they will help you park.

Join or login to comment.

  • anirban

    Load balancer doubt: Damu's design of load balancer is definitely great. I had a doubt regarding its capability to cater sticky sessions. And undoubtedly handling sticky session is difficult in that design. But, my question is are all the Yahoo apps are designed in a stateless manner which are brought under this load balancing

    December 11, 2012

    • Damodharan R.

      We strive to make the sessions stateless in our applications as that makes the application simple and scalable in many cases. That said there are applications which work better with session stickiness using Layer7 features and they choose Inline configuration.

      December 14, 2012

  • Bhaskar S.

    during the demo, the curl requests were failing:

    # nc -zv fd0b:fe61:d131::27b7:ea69 80
    nc: connect to fd0b:fe61:d131::27b7:ea69 port 80 (tcp) failed: Permission denied

    it was blocked by ip6tables

    # ip6tables -L INPUT --line-numbers
    Chain INPUT (policy ACCEPT)
    num target prot opt source destination
    1 ACCEPT all anywhere anywhere state RELATED,ESTABLISHED
    2 ACCEPT ipv6-icmp anywhere anywhere
    3 ACCEPT all anywhere anywhere
    4 ACCEPT tcp anywhere anywhere state NEW tcp dpt:ssh
    5 REJECT all anywhere anywhere reject-with icmp6-adm-prohibited

    after adding the following rule:

    # ip6tables -I INPUT 5 -p tcp --dport http -j ACCEPT

    it starts to work

    # nc -zv fd0b:fe61:d131::27b7:ea69 80
    Connection to fd0b:fe61:d131::27b7:ea69 80 port [tcp/http] succeeded!

    December 11, 2012

  • Shivraj

    Post-lunch sessions were of certain use to me as they belong to the realm of my domain. Unni's talk was excellant. I would love to be part of more such meetups and yes, I like the mug ! My tea tastes good in that :)

    December 10, 2012

  • balu

    this is my first session and I find it useful. I realized the importance of meeting similar minded people.. All the speakers are wonderful and informative. It helped me understand the design principles of few concepts. Unni's speech is brief and inspiring.. "The law of the farm"..yes... and the l3dsr, monitoring philosophies, ipv6 play, yamas are all crisp, compact and opening a gateway to a bigger world.

    I am surely benefited.. by the technology and even more by the spirit of this meet.

    December 9, 2012

Sign up

Meetup members, Log in

By clicking "Sign up" or "Sign up using Facebook", you confirm that you accept our Terms of Service & Privacy Policy