Even though the reading club on distributed graph data bases stopped I never really lost interest in management of big data and graph data. Due to the development of research grants and some new workers in our group I decided to create a new reading club. (The next and first meeting will be Thursday September 12th 15:30 central European time.) The reading club won’t be on a weekly basis but rather something like once a month. Tell me if you want to join via hangout or something similar! But I would like to be clear: If you didn’t carefully prepare the reading assignments by bringing questions and points for discussion to the meeting then don’t join the meeting. I don’t consider skimming a paper as a careful preparation.

The road map for the reading club on big data is quite clear: We will read again some papers that we read before but we will also look deeper and check out some existing technologies. So the reading will not only consist of scientific work (though this will build up the basis) but it will also consist of hand on and practical sessions which we obtain from reading blogs, tutorials, documentation and hand books.

Here will be the preliminary structure and road map for the reading club on big data which of course could easily vary over time!

Along these lines we want to understand

  • Why do these technologies scale? 
  • How do they handle concurrent traffic (especially write requests)?
  • How performance can be increased if there is another way of building up such highly scalable systems?
  • What kind of applications (like titan or mahout) are build on top of these systems?
At some point I would also love to do some side reading on distributed algorithms and distributed and parallel algorithm and data structure design. 

As stated above the reading club will be much more hand on in future than before I expect us to also deliver tutorials like that one on getting Nutch running on top of HBase and Solr

Even though we want to get hands on in current technologies the goal is rather to understand the principles behind them and find ways of improving them instead of just applying them to various problems.

I am considering to start a wikipage on wikiversity to create something like a course on big data management but I would only do this if I find a couple of people who would actively help to contribute to such a course. So please contact me if you are interested!

So to sum up the reading assignment for the first meeting are the Google file system and the map reduce paper.

If you like this post, you might like these related posts:

  1. Reading Club on distributed graph db returns with a new Format on April 4th 2012 The reading club was quite inactive due to traveling and...
  2. Reading club on Graph databases and distributed systems Update: find a summary of last meeting and the current...
  3. Related work of the Reading club on distributed graph data bases (Beehive, Scalable SPARQL Querying of Large RDF Graphs, memcached) Today we finally had our reading club and discussed several...
  4. Some thoughts on Google Mapeduce and Google Pregel after our discussions in the Reading Club The first meeting of our reading club was quite a success....
  5. From Graph (batch) processing towards a distributed graph data base Yesterdays meeting of the reading club was quite nice. We...

Sharing:

Tags: , , , , , ,

1 Comment on Reading Club Management of Big Data

  1. Xinruo says:

    Looks very interesting! I can suggest two additional relevant papers:

    MillWheel: Fault-Tolerant Stream Processing at Internet Scale
    http://db.disi.unitn.eu/pages/.....akidau.pdf
    Spanner: Google’s Globally-Distributed Database
    http://static.googleuserconten.....di2012.pdf

Leave a Reply

*

Close

Subscribe to my newsletter

You don't like mail?