Main Content

Disco: Reflections on Building a Distributed Data Processing Platform with Erlang

About the Talk

March 29, 2012 11:20 AM

Marines’ Memorial Club & Hotel 609 Sutter Street San Francisco

Marines’ Memorial Club & Hotel 609 Sutter Street San Francisco

Disco is a 5-year old framework for distributing and processing data, complete with its own tag-based filesystem, and a protocol for writing workers to process data in any language. The Disco backend is written in Erlang, which provides a fault-tolerant backbone for coordinating tasks between nodes.

This talk will reflect on the evolution of Disco, and the lessons learned in developing a distributed data processing platform with Erlang. We'll talk about what we got right in Disco and where there is still room for improvement, and we'll give an update about the Disco roadmap.

Talk objectives: Summarize the Disco Project in terms of past, present and future status. Provide lessons learned in developing Disco (specifically using Erlang and Python). Reflect on the big data ecosystem and how it relates to the Erlang community. Discuss problems in distributed computing, and the relative strengths and weaknesses of Erlang in this domain

Target audience: Anyone interested in building or using a distributed computing platform. Open-source developers looking to glean insight from the Disco experience.

Ratings and Recommendations

This Talk hasn't been rated yet. Sign In to rate Talks.

comments powered by Disqus