René Pickhardt

Stealing music is bullcrap and not acceptable!

Rene — Tue, 15 May 2012 13:30:50 +0000

After a series of successes of the German pirate party in elections there came up a big discussion on the German web about copyright in the publishing industry. see for example:

we are the creators
we are the citizens
and I am the bad girl
even new collecting companies (as competitor to GEMA) are about to be established

Being myself a member of the pirate party people often assume that I want to legalize file sharing and make music free of charge for anybody. Even though I think this can be a smart move in online music marketing I don’t follow these goals. After a huge discussion with two musicians I am befriend with I have the feeling that I should make this public clear statement once and for all:

I don’t support people steeling music in p2p file sharing services. This does harm the musician!

What I support is the idea of a modern copyright law that respects modern technology and also the behavior of the market and our society. This means in particular the establishment of modern business models for musicians.

I encourage any German person reading this blog to participate in the Working group copyright of the pirate party. You can participate and follow the mailinglist and join the discussion without being a member of the party. Most pirates don’t want to get rid of the copyright (as stated in media) but they want to modernize it and they enjoy music and are on the side of the musicians. Especially musicians should join the discussion since it is a topic that is of their concern!

Smartphones of Policemen could give criminals a competitive advantage

Rene — Fri, 27 Apr 2012 09:31:43 +0000

If I were a criminal I would create a smart phone app which would give me the possability to geographically and socially track policemen. Here some background on this thought.

Yesterday I was sitting in the German summit on “Facebook Goolgle & Co – Chances and Risks” (which I will blog about soon) But today during my train trip to the second day of the summit I was sitting in the train talking to a very friendly police officer. He agreed with what was said on the summit. The police is using social networks to find potential criminals. They also use cellphone tracking together with mobile providers to find people they are looking for. Nothing new and special so far. But now my interesting observation.

The police officer proudly told me that he is not using any social networking service because he enjoys his life in privacy. I understood that he believed this to be necessary in his job. By telling me this he was holding his iPhone in his hand. Again this shows one of the most crucial parts in this entire privacy discussion. Even highly educated people often lack an understanding of how much private information they implicitly give to third parties.

So I asked him if he used it during work times and he told me that he did since only mobile providers would know where he is and they could not give away data that easily. I was amazed! A policeman using an iPhone during work. That is such a security lack. If I were a terrorist organization I would create an iPhone and android app (or if possible an open mobile html5 app like Tim Berners Lee suggests <– you see the ethics overwhelm I am just not a criminal (-:). I would design this app in a way to support policemen. Help them communicate or have a cool map integration anything that was useful for the police. In this way I would create a database with real movement data of policemen. This data I could use for a different service similar to http://girlsaround.me/ displaying the current position and face of policemen (including if requested a list of people they recently communicated with including their phone numbers) on a map to anyone of my terrorist organization. The police just could never catch me since I would always know where they are (without asking any mobile provider!) I could even give them fake phone calls pretending I am one of the people they recently communicated with inputting them false information or just distracting them.

Of course this setting is only half realistic:

Every policeman would have to have a smartphone and use it during work time
Every policeman would have to install the app of the criminal
The criminal can distinguish between policemen and other people using the app (should be possible with data mining)
The criminal can decide weather the policeman is currently working or in leisure time

But it should show and demonstrate the dangers…

To conclude:

We have to disallow policemen to use private smart phones during work! Or if they do so they must not install any applications from a source they don’t trust. And here is the crucial point. Who to trust and who not? Trust usually is created through social ties. So if the app is there and some policemen like the service and recommend it to their coworkers trust is created. Who does really ask about the source of an app and about who is running/owning the data servers. A service that is well known on the web can easily run by 2 or 3 people and even if they are nice it is easy to manipulate or blackmail them in order to get access to these very sensitive data.

And on another more technical topic: We need a decentralized mobile space. There has to be a frequency on which people are able to set up their own transmitters and create decentralized mobile networks. It is a shame that those frequencies are all owned by companies creating centralized services.

By the way this would be a good solution since it would also enable the police to have their own decentralized mobile networks giving them privacy against third parties!

Disclaimer:

I never thought I would write an article in this paranoid way telling people what is possible and where the risks are. I almost feel like a member of ccc, anonymous or finally like a real pirate. But one year of PhD in a very data driven environment having social networks, information retrieval and the web as a focus really makes me understand more and more what is possible (in particular easy to achieve). Also the low awareness of society about these dangers (probably due to the complex technologies) overwhelms me and makes me feel like I have to act and at least inform people.

To bad that mostly people who are already aware of these topics read my blog. Maybe I have to go geek and create this app to demonstrate the functionality in order to really rise awareness. There are just too many interesting things to do during a PhD program so I think this time only writing about this has to be sufficient.

How Tim Berners Lee told me in front of thousand people: “Go geek and do it”

Rene — Fri, 20 Apr 2012 12:50:12 +0000

The statement already got twittered by my colleague Thomas Gottron and retweeted by many others

I am at www2012 conference and after the keynote by Neelie Kroes there was a panel discussion with her, Tim Berners Lee and Gille Babinet.
The discussion was about the question “Weather access to an open internet should be a human right?”

Clearly knowing where I am standing on this issue (yes it should be!) I was very happy that this question was discussed in front of such an audience. Tim Berners Lee obviously agreed on this point and Neelie Kroes really had some great and very diplomatic insights.

But for some reason the discussion always drifted up to the drawbacks of the web like copyright infringement. I was starting to get annoyed by this. Especially because it was always going as Free web vs copyright protection. So I decided to ask a question during Q&A which I am now about to blog.

During Q&A I also gave a litte background on the actuall question but I want to be a bit more detailed in my blog:

So yes I wish the “open web” to be a human right.
And I also think it is really important to protect the copyrights of artists, musicians and other people creating stuff. Working together with In Legend I really know how hard it is for a musician to survive and it is really important that he gets paid for what he does and shares.
BUT: the discussion is always an “eather – or” discussion and goes in the wrong direction! Bastian Emig from In Legend is very open minded about new ways to use the web working for the musician. Already in the plenary session Tim pointed out that he did not invent the Web to harm the record industry. But it is rather the record industry that refuses to think about new business models and just wishes everything to stay in the old ways which used to work quite well for them.
I made the experience that a band still needs to have a record label. You don’t get booked without the label. You don’t get articles in big print mags. The label gives you trust within the industrie and without that you are not seen by many people. And so on…
But just in my experience I see that the record label does big harm to a musician. As a member of this musicband I want to share our music on the web. Since there is piracy – which I cant change – I just have to think about a way how I could profit from it. Obviously by sharing the music myself I can increase my reach. This could significantly increase my chances for direct marketing (making the record label kind of obsolate) and this is what the labels seem to be afraid of. The web offers several huge opportunities for musicians to become recognized and an established act. But Labels own the licences and block musicians in doing smart and wise moves on the web.
I realize this problem exists due to the fact that labels have a monopoly on the product and too much power but pretending to protect the interests of the artists. Thereby hiding the fact that they are just fighting for their very own interests which do not neccessarily correlate whitch those from artists.

Here my question / point

It is not about copyright vs free / open internet. It is much more about a new model of copyright that can coexist with a free internet. In This new model licence owners (e.g. the labels) wouldn’t build those exclusive monopolies giving them such a high power. I asked what can be done to establish a new way of thinking about copyright. Since it really does not make sense that itunes gets 50% royalties for a digital distribution that is almost free of cost which I could easily run myself!

First of all – to my surprise – this won me a big applause from the audience which happened very rarely during the conference.

The full panel and discussion can be found at: http://www2012.wwwconference.org/media/videos/keynote-neelie-kroes/

Gille – to whom the question was originally directed – who is very friendly to the record industry answered some stuff I don’t even remember but he was basically stumbling around.

But then two really great answers came along:

Neelie:
“We are working on this and we see that the biggest issue is the record industrie. They pretend to protect the artists and they are not! We need legislation but maybe we need new forms of legislation. Models that worked well in the past may not serve our needs in todays world. I agree with you that you are pointing to the most cruicial point in this discussion.”

Me being totally satisfied with her answer sat down but Tim Berners Lee wanted to say something:
“You know it! Think of a world that you want. Just imagine it!

What would be the distribution?
what would be the user interface?
What would be the processes?
What third parties would be involved.

Go out and build it! Talk to the people here. Install an apache server and just go geek and make it happen!”

what a great statement!

It is always nice to have ideas and see solutions to problems. And yes you can always wine around and do nothing. But as a matter of the fact right now the web is still open an free! The technology is there. It really is just a matter of going out an building it. This is what I always said: This is why big traditional media companies didn’t built the youtube, google, facebooks, twitters, flickr,… applications in this world.

This statement gave me a lot of confidence to stronger believe in my ideas and even one day later I am really feeling that this statement will change my future life. It is really interesting that a man – who I value a lot – tells me something I always felt, hardly did and hits right a way to one of my weekest points!

After the sesion I got my copy of Tim Berners Lee’s book signed and he asked me to send him an email once my site is up. It is really amazing to receive this kind of feedback by such a great person.

That was one of the most inspiring moments in my life! So anyone who wants to join me going geek on the next generation music web app is very welcome to contact me or leave a comment! There really is a lot of stuff in my mind and I have already dreamt a lot and seen what is possible…

Tim Berners Lee signing my Copy of his book at www2012 in Lyon

Keynote on www2012 by Sir Tim Berners Lee

Rene — Wed, 18 Apr 2012 10:30:12 +0000

disclaimer: this is a very sloppy summary of the keynote speech of Sir Tim Berners Lee. It is neither spellchecked it was taking as notes while sitting inside. I hope I find the time after www2012 to go over it and improve it.

take aways

Decentralized design that is very important. and if you talk about the centralized system like DNS vs internet there are really social issiues that you are talking about.

look at the way w3c builds standards it is hard. It is a huge organizsation and they have to work together but also be split into modular groups.

IDEA 1 Mobile web apps

he gave a lot of interesting insights about the freedom of programmers and the limits of programing models. (Closely related to my thoughts on query languages for graph data bases) He says the easier the programming model the more you restrict people the more they can achieve.

But he also points out there is an ongoing battle between being universal and just being a tool like a refrigerator. he didn’t say it directly but if you listented carefully to him he says don’t support closed blocked systems like apple or even android but make it a open html5 javascript webapp. You decide weather the web stays open. I think he really sees how the mobile web is beeing more and more closed.

He warns: Make open mobile web apps. Join the working groups for setting the standards…
takeaway: VERY VERY important and interesting part of his speech! especially for typology.

My Question: “As a company building something. if it is open you don’t get as much benefit (like access to the friendship graph) as in the case of going for android / iphone systems…”

IDEA2 Standards centralized vs decentralized – power

He talked about the importance of having the same standards for the low level like html, http, and so on. He compares this to people not speeaking the same language. He says it is cool to build all the cool stuff ontop of it.
He says it is hard to build the standards but also important because it made it possible for the web to scale and win against goopher and other things from the early web.
he says the standards are great because anyone can build something on it. it contributes to be decentralized. You don’t have to ask anyone to build something.
My Question: “SPDY seems better / faster and more mature than http. but of course it is hard to change a running system. ”

Value of being completely independend (decentralized) vs the value of working together in a common way (centralized) and this is a huge fight.

IDEA 3 Trust

Ananomous are not quite sure what they are fighing for. is it complete anarchie or fight against corruption. is it the fight against certain goverments or against goverments in general. again. central vs decentral.
Very technical part of his talk that I unfortunatly could not really follow. The main idea was clear. It was about trust, the role of social networks and the methods that could build and propagate trust.
He hopes decentralized trust systems can be found but the battle is not over. (Remark from me: Of course Google+ is better than facebook but both systems are centralized I guess there needs to be low level standards like http to olve the problem but they are hard to implement.) He already said that the semantic web people tried to do it but haven’t successed yet.

IDEA4 openess especially in UK

he talks a lot about open goverment data.
“I spend a lot of time to governments talking to governments to publish data on the web. But a lot of people are pushing back trying to hold back data an maintain power.”
he says it is your responsability to ask parties before the election to have a commitment on making government data open.

open licences. Companies complain about open licenses since they want to use it but not open up their own data.

As a member of the pirate party I can only emphisize on his statements!

Idea 5 Privacy

hes says there are three different forms of privacy:
1.) Of ourse having the shop remembering me and my shoe size is common practise and nice for me. That is one way of privacy. of course those companies if they want to have a good relation with me will not give away my data.

2.) you never know what data will be out there in the future. It might be possible that anonymous data now will become transparent through other data sets coming up in the future…

3.) invasion / tracking / sells to highest bidder / or to the goverment or whoever asks. In the first days of the web this was hard routers couldn’t do this but now it is impossible. But in my oppinion tbl was talking about the facebook’s and googles of this world.
he says this is as dynamite. If someone wants to use this data against you. you will be toasted.
And you cannot stop this. because there are institutions that collet this information and once it is collected it is not save by definition.
he basically says that you cannot collect data about users by default unless you have a similar powershare of “executive legislative and judicative” if this is not happening we should not allow anyone to make such data tracking.

We have to spend 90% time to do cool stuff on the web be innovative. but 10% of the time we have to deal with theses issues otherwise the web will be locked down at some time.
It is about blocking, spamming, twitter bombing, data stealing and so own. This is a huge challange and a big risk of our open web! It is not only the open market depending on it but also democracy and human discourse that depends on it. “I call that net neutrality”

“cispn” americans do this in trying to controll the web and close communication…. go out and look for this. This things happen quick go out and defend the internet and fight for it. This is a duty we have to do in the 10 % of our time.

“I want you to see discussing these things. Think about what you are leaving for the next generation of this. i am happy to do this in the next web conferences but I really want to see ”

questions

on distributed decission making:
How do we move from hirarchical system like our goverments are to a decentralized system which is possible due to all the connections that we have. We should be able to
people naturally don’t go out an break these boundaries of locallity. (there is also a youtube paper on this) Social networking site should rather suggest to spread friends in stead of building those communities of you have 81 friends in common. go out and meet people that are far away from you. use the connections that are being made. (what is the macroscopic effect of this little change in microscoping behaviour)
I think we should do research on new democratic systems like wikipedia (or in my opinion: pirate party) there was the story that people who didn’t vote for barack obama. It turned out those who wouldn’t vote for him didn’t because they couldn’t imagine to have a black president. There was a high correlation of those people and those who have never worked together with people from different ethnical beackground. so go out and spread friendship.

Idea: liquid feedback for W3C as a working group!

question on openess an facebook and the request on tbl’s thoughts
TBL why would you build an app on facebook or a closed world if there is still the open jungle out there. Peopole already asked my that question on netscape and internet explorer. They always ask me this if monoplies rise up.

Funny/interesing Quotes by Tim Berners Lee:

“There is only one person that has been to all web conferences and that is me”

“I recieved a mail recently saying: We had to do a project on an inventor and we decided to do it on you because you are not dead!”

“Values that made the web possible: Openess, concencious about openess, transparency, privacy,”

“If you have questions it is much more interesting for me. Because I have heard myself talking before.”

“Please develop HTML5 mobile web apps rater than native mobile apps!”

Reading Club on distributed graph db returns with a new Format on April 4th 2012

Rene — Mon, 02 Apr 2012 09:37:05 +0000

The reading club was quite inactive due to traveling and also a not optimal process for the choice of literature. That is why a new format for the reading club has been discussed and agreed upon.

The new Format means that we have 4 new rules

we will only discuss up to 3 papers in 90 minutes of time. So rough speaking we have 30 minutes per paper but this does not have to be strict.
The decided papers should be read by everyone before the reading club takes place.
For every paper there is one responsible person (moderator) who did read the entire paper before he suggested it as a common reading.
Open questions to the (potential) reading assignments and ideas for reading can and should be discussed on http://related-work.rene-pickhardt.de/ (use the same template as I used for the reading assignments in this blogpost) eg:

Moderator:
Paper download:
Why to read it
topics to discuss / open questions:

For next meeting on April 4th 2 pm CET (in two days) the literature will be:

Key key value Stores for efficiently processing graph data in the cloud (Moderator: Daniel Bildhauer)
Boost graph library (Moderator: Stefan Scheglmann)
A Practical Scalable Distributed B-Tree (suggestion by Stefan Scheglmann, Moderator: Rene)

While preparing these papers we might come across some other interesting literature.

If you want to suggest some of the literature you should also read that piece of work until the reading club meeting takes place and know why you want everybody to prepare the same paper and discuss it (rule 3). Additionally you should open a topic on the paper on http://related-work.rene-pickhardt.de/ using the above template before the reading club takes place (rule 4)

I hope this is of help for the entire project and I am looking forward to the next meeting!

PhD proposal on distributed graph data bases

Rene — Tue, 27 Mar 2012 10:19:22 +0000

Over the last week we had our off campus meeting with a lot of communication training (very good and fruitful) as well as a special treatment for some PhD students called “massage your diss”. I was one of the lucky students who were able to discuss our research ideas with a post doc and other PhD candidates for more than 6 hours. This lead to the structure, todos and time table of my PhD proposal. This has to be finalized over the next couple days but I already want to share the structure in order to make it more real. You might also want to follow my article on a wish list of distributed graph data base technology

[TODO] 0. Find a template for the PhD proposal

That is straight forward. The task is just to look at other students PhD proposals also at some major conferences and see what kind of structure they use. A very common structure for papers is Jennifer Widom’s structure for writing a good research paper. This or a similar template will help to make the proposal readable in a good way. For this blog article I will follow Jennifer Widom more or less.

1. Write an Introduction

Here I will describe the use case(s) of a distributed graph data base. These could be

indexing the web graph for a general purpose search engine like Google, Bing, Baidu, Yandex…
running the backend of a social network like Facebook, Google+, Twitter, LinkedIn,…
storing web log files and click streams of users
doing information retrieval (recommender systems) in the above scenarios

There could also be very other use cases like graphs from

biology
finance
regular graphs
geographic maps like road and traffic networks

2. Discuss all the related work

This is done to name all the existing approaches and challenges that come with a distributed graph data base. It is also important to set onself apart from existing frameworks like graph processing. Here I will name the at least the related work in the following fields:

graph processing (Signal Collect, Pregel,…)
graph theory (especially data structures and algorithms)
(dynamic/adaptive) graph partitioning
distributed computing / systems (MPI, Bulk Synchronous Parallel Programming, Map Reduce, P2P, distributed hash tables, distributed file systems…)
redundancy vs fault tolerance
network programming (protocols, latency vs bandwidth)
data bases (ACID, multiple user access, …)
graph data base query languages (SPARQL, Gremlin, Cypher,…)
Social Network and graph analysis and modelling.

3. Formalize the problem of distributed graph data bases

After describing the related work and knowing the standard terminology it makes sense to really formalize the problem. Several steps have to be taken: There needs to be notation for distributed graph data bases fixed. This has to respect two things:

a) the real – so far unknown – problems that will be solved during PhD. In this way fixing the notation and formalizing the (unknown) problem will be kind of hard.

b) The use cases: For the web use case this will probably translate to scale free small world network graphs with a very small diameter. Probably in order to respect other use cases than the web it will make sense to cite different graph models e.g. mathematical models to generate graphs with certain properties from the related work.

The important step here is that fixing a use case will also fix a notation and help to formalize the problem. The crucial part is to choose the use case still so general that all special cases and boarder line cases are included. Especially the use case should be a real extension to graph processing which should of course be possible with a distributed graph data base.

One very important part of the formalization will lead to a first research question:

4. Graph Query languages – Graph Algebra

I think graph data bases are not really general purpose data bases. They exist to solve a certain class of problems in a certain range. They seem to be especially useful where information of a local neighborhood of data points is frequently needed. They also often seem to be useful when schemaless data is processed. This leads to the question of a query language. Obviously (?) the more general the query language the harder to have a very efficient solution. The model of a relational algebra was a very successful concept in relational data bases. I guess a similar graph algebra is needed as a mathmatical concept for distributed graph data bases as a foundation of their query languages.

Remark that this chapter has nothing much to do with distributed graph data bases but with graph data bases in general.

The graph algebra I have in mind so far is pretty similar to neo4j and consists of some atomic CRUD operations. Once the results are known (ether as an answer from the related work or by own research) I will be able to run my first experiments in a distributed environment.

5. Analysis of Basic graph data structures vs distribution strategies vs Basic CRUD operations

As expected the graph algebra will consist of some atomic CRUD operations those operations have to be tested against all different data structures one can think of in the different known distributed environments over several different real world data sets. This task will be rather straight forward. It will be possible to know the theoretical results of most implementations. The reason for this experiment is to collect experimental experiences in a distributed setting and to understand what is really happening and where the difficulties in a distributed setting are. Already in the evaluation of graphity I realized that there is a huge gap between theoretical predictions and the real results. In this way I am convinced that this experiment is a good step forward and the deep understanding of actually implementing all this will hopefully lead to:

6. Development of hybrid data structures (creative input)

It would be the first time in my life where I am running such an experiment without any new ideas coming up to tweak and tune. So I am expecting to have learnt a lot from the first experiment in order to have some creative ideas how to combine several data structures and distribution techniques in order to make a better (especially bigger scaling) distributed graph data base technology.

7. Analysis of multiple user access and ACID

One important fact of a distributed graph data base that was not in the focus of my research so far is the part that actually makes it a data base and sets it apart from some graph processing frame work. Even after finding a good data structure and distributed model there are new limitations coming once multiple user access and ACID are introduced. These topics are to some degree orthogonal to the CRUD operations examined in my first planned experiment. I am pretty sure that the experiments from above and more reading on ACID in distributed computing will lead to more reasearch questions and ideas how to test several standard ACID strategies for several data structures in several distributed environments. In this sense this chapter will be an extension to the 5. paragraph.

8. Again creative input for multiple user access and ACID

After heaving learnt what the best data structures for basic query operations in a distributed setting are and also what the best methods to achieve ACID are it is time for more creative input. This will have the goal to find a solution (data structure and distribution mechanism) that respects both the speed of basic query operations and the ease for ACID. Once this done everything is straight forward again.

9. Comprehensive benchmark of my solution with existing frameworks

My own solution has to be benchmarked against all the standard technologies for distributed graph data bases and graph processing frameworks.

10. Conclusion of my PhD proposal

So the goal of my PhD is to analyse different data structures and distribution techniques for a realization of distributed graph data base. This will be done with respect to a good runtime of some basic graph queries (CRUD) respecting a standardized graph query algebra as well as muli user access and the paradigms of ACID.

11 Timetable and mile stones

This is a rough schedual fixing some of the major mile stones.

2012 / 04: hand in PhD proposal
2012 / 07: graph query algebra is fixed. Maybe a paper is submitted
2012 / 10: experiments of basic CRUD operations done
2013 / 02: paper with results from basic CRUD operations done
2013 / 07: preliminary results on ACID and multi user experiments are done and submitted to a conference
2013 /08: min 3 month research internship in a company benchmarking my system on real data
end of 2013: publishing the results
2014: 9 months of writing my dissertation

For anyone who has input, knows of papers or can point me to similar research I am more than happy if you could contact me or start the discussion!

Thank you very much for reading so far!

Paul Wagner and Till Speicher won State Competition “Jugend Forscht Hessen” and best Project award using neo4j

Rene — Fri, 16 Mar 2012 11:18:38 +0000

6 months of hard coding and supervising by me are over and end with a huge success! After analyzing 80 GB of Google ngrams data Paul and Till put them to a neo4j graph data base in order to make predictions for fast scentence completion. Today was the award ceremony and the two students from Darmstadt and Saarbrücken (respectivly) won the first place. Additionally the received the “beste schöpferische Arbeit” award. Which is the award for the best project in the entire competition (over all disciplines).

With their technology and the almost finnished android app typing will be revolutionized! While typing a scentence they are able to predict the next word with a recall of 67% creating a huge additional vallue for today’s smartphones.

So stay tuned of the upcomming news and the federal competition on May in Erfurt.

Have a look at their website where you can find the (still) German Documentation. As well as the source code and a demo (which I also include here (use tab completion (-: as in unix bash)

Right now it only works for German Language – since only German data was processed – so try sentences like

“Warum ist die Banane krumm” (where the rare word krumm is correctly predicted due to the relation of the famous question why is the banana curved?
“Das kann ich doch auch” (I am also able to do that)
“geht wirklich nur deutsche Sprache ?” (Is really only German language possible?)

Why Musicians should have a Bandpage on Google Plus!

Rene — Tue, 13 Mar 2012 10:37:16 +0000

The following info graphic for businesses was released by Chris Brogan and demonstrates quite well why musicians should get on Google Plus and how to use it. It is released under a creative commons licence http://creativecommons.org/licenses/by-nc-sa/2.0/

Very good work!

Google Video on Search Quality Meeting: Spelling for Long Queries by Lars Hellsten

Rene — Mon, 12 Mar 2012 19:11:04 +0000

Amazing! Today I had a discussion with a coworker about transparency and the way companies should be more open about what they are doing! And what happens on the same day? One of my favourite webcompanies has decided to publish a short video taken from the weekly search quality meeting!

The proposed change by Lars Hellsten is that instead of only checking the first 10 words for possible spelling corrections one could predict which two words are most likely spelled wrong and add an additional window of +-5 words around them. They discuss how this change has much better scores than the old one.

The entire video is interesting because they say that semantic context is usually given by using 3 grams. My students used up to 5 grams in order to make their scentence prediction and the machine learning already told them that 4grams would be sufficient to make syntactically and semantically correct predictions.

Anyway enjoy this great video by Google and thanks to Google for sharing this:

Related-work.net – Product Requirement Document released!

Rene — Mon, 12 Mar 2012 10:26:50 +0000

Recently I visited my friend Heinrich Hartmann in Oxford. We talked about various issues how research is done in these days and how the web could theoretically help to spread information faster and more efficiently connect people interested in the same paper / topics.
The idea of http://www.related-work.net was born. A scientific platform which is open source and open data and tries to solve those problems.

But we did not want to reinvent the wheel. So we did some research on existing online solutions and also asked people from various disciplines to name their problems. Find below our product requirement document! If you like our approach you can contact us or contribute on the source code find some starting documentation!

So the plan is to fork an open source question answer system and enrich it with the features fulfilling the needs of scientists and some social aspects (hopefully using neo4j as a supporting data base technology) which will eventually help to rank related work of a paper.

Feel free to provide us with feedback and wishes and join our effort!

Beginning of our Product Requirement Document

We propose to create a new website for the scientific community which brings together people which are reading the same paper. The basic idea is to mix the functionality of a Q&A platform (like MathOverflow) with a paper database (like arXiv). We follow a strict openness principal by making available the source code and the data we collect.
We start with an analysis how the internet is currently used in different fields and explain the shortcomings. The actual product description can be found under the section “Basic idea”. At the end we present an overview over the websites which follow a similar approach.
This document – as well as the whole project – is work in progress. We are happy about any kind of comments or other contributions.

The distribution of scientific knowledge

Every scientist hast to stay up to date with the developments in his area of research. The basic sources for finding new information are:

Conferences
Research Seminars
Journals
Preprint-servers (arXiv)
Review Databases (MathSciNet, Zentralblatt, …)
Q&A Sites (MathOverflow, StackOverflow, …)
Blogs
Social Networks (Twitter, Google+)
Bibliograhpic Databases (Mendeley, nNode, Medline, etc. )

Every community has found its very own way of how to use this tools.

Mathematics by Heinrich Hartmann – Oxford:

To stay up to date with recent developments I check arxiv.org on a daily basis (RSS feed) participate in mathoverflow.net and search for papers over Google Scholar or MathSciNet. Occasionally interesting work is shared by people in my Google+ circles. In general the speed of pure mathematics is very slow. New research often builds upon work which has been out for a few years. To stay reasonably up to date it is enough to go to conferences every 3-5 months.
I read many papers on myself because I am the only one at the department who does research on that particular topic. We have a reading class where we read papers/lecture notes which are relevant for more people. Usually they are concerned with introductions to certain kinds of theory. We have weekly seminars where people talk about their recently published work. There are some very active blogs by famous mathematicians, but in my area blogs play virtually no role.

Computer Science by René Pickhardt – Uni Koblenz

In Computer Science topics are evolving but also changing very quickly. It is always important to have both an overview of upcoming technologies (which you get from tech blogs) as well as access to current research trends.
Since the speed in computer science is so fast and the review process in Journals often takes much time our main source of information and papers are conferences and twitter.

Usually conference papers are distributed digitally to participants. If one is interested in those papers google queries like “conference name year papers” are frequently used. Sites like http://www.sciweavers.org/ host and aggregate preprints of papers and organize them by conference.
The general method to follow a conference that one is not attending is to follow the hashtag of the conference on Twitter. In general Twitter is the most used tool to share distribute and find information not only for papers but also for the above mentioned news about upcoming technologies.

Another rich source for computer scientists is, of course, the related work of papers and google scholar. Especially useful is the method of finding a very influential paper with more than 1000 citations and find newer papers that quote this paper containing a certain keyword which is one of the features of google scholar.
The main problem in computer science is not to find a rare paper or idea but rather to filter the huge amount of publications and also bad publications and also keep track of trends. In this way a system that ranks and summarize papers (not only by abstract and citation counts) would help me a lot to select what related work of a paper I should read!

Psychology by Elisa Scheller – Uni Freiburg

As a psychologist/neuroscientist, I receive recommendations for scientific papers via google scholar alerts or science direct alerts (http://www.sciencedirect.com/); I receive alerts regarding keywords or regarding entire journal issues. When I search for a certain publication, I use pubmed.org or scholar.google.com. This can sometimes be kind of annoying, as I receive multiple alerts from different sources; but I guess it is the best way to stay up to date regarding recent developments. This is especially important in my field, as we feel a big amount of “publication pressure”; I work on a method which is considered as “quite fancy” at the moment, so I also use the alerts to make sure nobody has published “my” experiment yet.
Sometimes a facebook friend recommends a certain publication or a colleague points me to it. Most of the time, I read articles on my own, as I am the only person working on this specific topic at my institution. Additionally, we have a weekly journal club where everyone in turn presents work which is related to our focus of research, e.g. a certain part of the human brain. There is also a weekly seminar dedicated to presentations about ongoing projects.
Blogs (e.g. mindhacks.com, http://neuroskeptic.blogspot.com/) can be a source to get an overview about recent developments, but I have to admit I use them mainly for work-related entertainment.
All in all, it is easy to stay up to date using alerts from different platforms; the annoying part of it is the flood of emails you receive and that you are quite often alerted to articles that don’t fit your interests (no matter how exact you try to specify your keywords).

Biomedical Research by Johanna Goldmann – MIT

In the biological sciences, in research at the bench – communication is one of the most fundamental tools a scientist can have. Communication with other scientist may open up the possibilities of new collaborations, can lead to a completely new view point of a known question, the integration and expansion of methods as well as allowing a scientist to have a good understanding of what is known, what is not known and what other people have – both successfully and unsuccessfully – tried to investigate.
Yet communication is something that is currently very much lacking in academic science – lacking to the extent that most scientist will agree hinders the progress of research. Nonetheless the lack of communication and the issues it brings with it is something that most scientists will have accepted as a necessary evil – not knowing how to possibly change it.
Progress is only reported in peer-reviewed journals – many which are greatly affected not only but what is currently “sexy” in research but also by politics and connections and the “publish or perish” pressure. Due to the amount of this pressure in publishing in journals and the amount of weight the list of your publications will have upon any young scientists chances of success, scientist tend also to be very reluctant in sharing any information pre-publication.
Furthermore one of the major issues is that currently there really is no way of publishing or communicating either negative results or minor findings, which causes may questions or methods to be repeatedly investigated as well as a loss of information.
Given how much social networks and the internet has changed communication as well as the access to information over the past years – there is a need for this change to affect research and communication in the life science and transform the way we think not only about solving and approaching research questions we gather but the information and insights we gain as a whole.

Philosophy by Sascha Benjamin Fink – Uni Osnabrück

The most important source of information for philosophers is http://philpapers.org/. You can follow trends going on in your field of interest. Philpapers has a list of almost all papers together with their abstracts, keywords and categories as well as a link to the publisher. Additional information about similar papers is displayed.

Every category of papers is managed by some editor. For each category it is possible to subscribe to a newsletter. In this way once per month I will be informed about current publications in journals related to my topic of interest. Every User is able to create an account and manage his literature and the papers of his he is interested in.
Other research and information exchange methods among philosophers consist of mailing lists, reading clubs and Blogs. Have a look at David Chalmers blog list. Blogs are also becoming more and more important. Unfortunately they are usually on general topics and discussing developments of the community (e.g. Leiter’s Blog, Chalmers’ Blog and Schwitzgebel’s Blog).

But all together I still think that for me a centralized service like Philpapers is my favourite tool because it aggregates most information. If I don’t hear about it on Philpapers usually it is not that important. I think among Philosophers this platform – though incomplete – seems to be the standard for the next couple of years.

Problems

As a scientist it is crucial to be informed about the current developments in the research area. Abstracting from the reports above we divide the tasks roughly into the following stages.

1. Finding and filtering new publications:

What is happening right now? What are the current hot topics my area? What are current trends? (→ Check arXiv/Twitter)
Did a friend of mine write something? Did a “big shot” write something?
(→ Check meta information: title, authors)
Are my colleagues excited about a new development? (→ Talk to them.)

2. Getting more information about a given paper:

What is actually done in a given paper? Is it relevant for me? Is it really new? Is it a breakthrough? (→ Read abstracts. Find a good readable summary/review.)
Judge the quality of a paper: Is it correct? Is it well written?
( → Where is it published, if at all? Skim through content.)

Finally there is a fundamental decision: Shall I read the whole paper, or not? which leads us to the next task.

3. Understanding a paper: Understanding a paper in depth can be a very time consuming and tedious process. The presentation is often very short and much knowledge is assumed from the reader. The notation choices can be bad, so that even the statements are hard to understand. In effect the paper is easily readable only for a very small circle of specialist in the area. If one is not in the lucky situation to belong to that circle, one usually applies the following strategies:

Lookup references. This forces you to process a whole tree of older papers which might be hard to read, and hard to get hold of. Sometimes it is worthwhile to consult a textbook to polish up fundamentals.
Finding additional resources. Is there a review? Is there a related video lecture or slides explaining the material in more detail? Is the author going to a conference in the near future, or even giving a seminar in the area?
Join forces. Find people thinking about the same paper: Has somebody at my department already read the paper, so that I can ask some questions? Is there enough interest to make a reading group, or more formally, run a seminar about that paper.
Contact the author. This a last resort. If you have struggled with understanding the paper for a very long time and really need/want to get it, you might eventually write an email to the author – who might respond, or not. Sometimes even errors are found! – and not published! An indeed, there is no easy way to publish “errata” anywhere on the net.

In mathematics most papers are not getting read though the end. One uses strategies 1 & 2 till one gets stuck and moves on to something more exciting. The chances of survival are much better with strategy 3 where one is committed putting a lot of effort in it over weeks.

4. Finding related work. Where to go from there? Is the paper superseded by a more recent development? Which are the relevant papers which the author builds upon? What are the historic influences? What are the founding ideas of the subject? Finding related work is very time consuming. It is easy to overlook things given that the references are often vast, and sometimes hard to get hold of. Getting information over citations requires often access to commercial databases.

Basic idea:

All researchers around the world are faced with the same problems and come up with their individual solutions. There are great synergies in bringing these people together with an online platform! Most of the addressed problems are solved with a paper centric service which allows you to…

…get to know other readers of the paper.
…exchange with the other readers: ask questions, write comments, reviews.
…share the gained insights with the community.
…ask questions about the paper.
…discuss the paper.
…review the paper.

We want to do that with a new mixture of a traditional Q&A system like StackExchange or MathOverflow with a paper database and social features. The key features of this system are as follows:

Openness: We follow a strict openness principle. The software will be developed in open source. All data generated on this site will be under a creative commons license (like Wikipedia) and will be made available to the community in form of database dumps or an API (open data).

We use two different types of content sites in our system: Papers and Discussions.

Paper sites. A paper site is dedicated to a single publication. And has the following features:

Paper meta information
- show title, author, abstract, journal, tags
- leave a comment
- write a review (with wiki option)
- vote up/down
Paper resources
- show pdfs, slides, notes, video lectures, etc.
- add a resource
Related Work
- show the reference-tree and citations in an intelligent way.
Discussions:
- show related discussions
- start a new discussion
Social features
- bookmark
- share on G+, twitter

The point “Related Work” deserves some further explanation. The citation graph offers a great deal more information than just a list of references. Together with the user generated content like votes and the individual paper bookmarks and social graph one has a very interesting data set which can be harvested. We want this point at least view with respect to: Popularity/Topics/Read by Friends. Later on one could add more sophisticated, even graphical views on this graph.

Discussion sites. A discussion looks more like a traditional QA-question, with the difference, that each discussion may have related (many) papers. A discussion site contains:

Discussion meta information (title, author, body)
Discussion content
Related papers
Voting
Follow/Bookmark

Besides the content sides we want to provide the following features:

News Stream. This is the start page of our website. It will be generated from the network consisting of friends, papers and authors. There should be several modes like:

hot: heavily discussed papers/discussions
new papers: list new publications (filtered by tag, like arXiv feed)
social: What did your friends do lately
default: intelligent mix of recent activity that is relevant to the logged in user

Moreover, filter by tag should be always available.

Search bar:

Searches contents of the site, but should also find papers on freely available databases (e.g. arXiv). Adding a paper should be very seamless process from there.
Search result ranking uses vote and view information.
Personalized search information. (Physicists usually do not want sociology results.)
Auto completion on paper titles, author, discussions.

Social: (hard to implement, maybe for second version!)

Easily refer to users by @-syntax familiar from Twitter/Google+
Maintain a friendship / trust graph
Friendship recommendations
Find friends from Google+ on the site

Benefits

Our proposed websites improves the above mentioned problems in the following ways.

1. Finding and filtering new publications:This step can be improved with even very little community effort:

Tell other people, that you are interested in the paper. Vote it up or leave a comment if you are very excited about it.

Point out a paper to a colleague.

2. Getting more information about a given paper:

Write a summary or review about a paper you have read or skimmed through. Maybe the introduction is hard to read or some results are not clearly stated.
Can you recommend reading this paper? Vote it up!
Ask a colleague for his opinion on the paper. Maybe he can write a summary?

Many reviews of new papers are already written. E.g. MathSciNet and Zentralblatt maintain a large database of Reviews which are provided by the community and are not freely available. Many authors would be much more happy to write them to an open system!

3. Understanding a paper:Here are the mayor synergies which we want to address with our project.

Ask a question: Why is the author using this experimental method? How does Lemma 3.4 work? Why do I need this assumption? What is the intiution behind the “virtual truncation”? What implications does this work have?
Start a discussion: (might involve more than one paper.) What is the difference of these two papers? Is there a reference explaining this more clearly? What should I read in advance to understand the theory?
Add resources. Tell the community about related videos, notes, books etc. which are available on other sites.
Share your notes. If you have discussed a paper in a reading class or seminar. Collect your notes or opinions and make them available for the community.
Restate interesting statements. Tell the community when you have found a helpful result which is buried inside the paper. In that way Google may find it!

4. Finding related work. Having a well structured and easily navigable view on related papers simplifies the search a lot. The filtering benefits from the content generated by the users (votes) and individual information, like friends who have written/bookmarked a paper.

Similar Sites on the Web

There are several discussions in QA forum which are discussing precisely this problem:

Quora.com: Where can I comment on sci papers?
MathOverflow: Good websites for discussions of mathematical papers?
MathOverflow: Is a free alternative to MathSciNet possible?
MathOverflow: Errata-Database

We found three sites on the internet which follow a similar approach which we examined more carefully.

1. There is a social network which has most of our features implemented:

researchgate.net
“Connect with researchers, make your work visible, and stay current.”

The Economist has dedicated an article to them. It is essentially a facebook clone, with special features for scientist.

Large, fast growing community. 1.4m +50.000/m. Mainly Biology and Medicine.
(As Daniel Mietchen points out, the size might be misleading due to institutional accounts)
Very professional Look and Feel. Company from Berlin, Germany, funded by VC. (48 People involved, 10 Jobs advertised)
Huge Feature set:

Profile site, Connect to friends
News Feed
Publication Database, Conference Finder, Jobmarket
Every Paper its own page: with

Voting up/down
Comments
Metadata (Title, Author, Abstract, Preveiw)
Social Media (Share, Bookmark, Follow author)

Organize Workgroups/Reading Classes.

Differences to our approach:

Closed Data / Closed Source
Very complex site which solves a lot of purposes
Only very basic features on paper site: vote/comment.
QA system is not linked well to paper database
No MathML
Mainly populated by undergraduates

2. Another website which comes reasonably close is:

http://www.sciweavers.org/

“an academic network that aggregates links to research paper preprints
then categorizes them into proceedings.”

Includes a large collection of online tools for various purposes
Have a big library of papers/software/datasets/conferences for computer science.
Paper sites have:

Meta information and preview
Vote functionality and view statistics, tags
Comments
Related work
Bookmarking
Author information

User profiles (no friendships)

Differences to our approach:

Focus on computer science community
Comment and Discussions are well hidden on paper sites
No News stream
Very spacious design

3. Another very similar site is:

journalfire.com – beta
“Share what your read – connect to colleagues – create journal clubs.”

It has the following features:

Comment on Papers. Activity feed (?). Follow articles.
Host Journal Clubs. Create Events related to papers.
Powerful search box fetching papers from Arxiv and Pubmed (slow)
Social features on site: User profiles, friend finder (no fb/g+ integration yet)
News feed – from subscribed papers and friends
Easy paper import via Bookmarklet
Good usability!! (but slow loading times)
Private reading clubs cost money!

They are very skilled: Maintained by 3 PhD students/postdocs from Caltec and MIT.

Differences to our approach:

Closed Data, Closed Source
Also this site misses (currently) misses out ranking features
Very Closed model – Signup required
Weak Crowd sourcing: Cannot add Meta information

The site is still at its very beginning with little users. The project started in 2010 and did not gain much momentum since.

The other sites are roughly classified in the following categories:
1. Single people who are following a very similar idea:

annotatr.appspot.com. Combines a metadata-base with the disqus plugin. You can comment but not rate. Good usability. Nice CSS. Good search function. No MathML. No related article suggestion. Maintained by two academics in private time. Hosted on Google Apps. Closed Source – Closed Data.
r-Forum – a resource where mathematicians can collect record reviews, corrections of a resource (e.g. paper, talk, …). A simple Vanilla-Forum/Wiki with almost no content used by maybe 12 people in US. No automated Data import. No rating system.
http://math-arch.org/ – Post comments to math papers. very bad usability – get even errors. Maintained by a group of russian programmers LogicSun. Closed Source – Closed Data.

Analysis: Although the principal idea to connect people reading papers is there. The implementation is very bad in terms of usability and even basic programming. Also the voting features are missed out.

2. (Semi) Professional sites.

Public Libary of Science very professional, huge paper data base for mainly biology, medicine. Features full text papers, lots of interesting meta information including references. Has comment features (not very visible) and news stream on the start page.
No QA features (+1, Ask question) on the site. Only published articles are on the site.
Mendeley.com – Huge Bibliographic database with bookmarking and social features. You can organize reading groups in there, with comments and notes shared among the participants. Features a news stream with papers by friends. Nice import. Impressive fulltext data and Reference features.
No QA features for paper. No comments for paper. Requires Signup to do anything useful.
papercritic.com – Open review database. Connected to Mendely bibliographic libary. You can post reviews. No rating. No comments. Not open: Mendely is commercial.
webofknowledge.com. Commercial academic citation index.
zotero.org – features programm that runs inside a browser. “easy-to-use tool to help you collect, organize, cite, and share your research sources”

Analysis: The goal of all these tools is to simplify the reference management, by providing metadata like references, citations, abstracts, author profiles. Commenting features on the paper site are not there or not promoted.

3. Vaguely related sites which solve different problems:

citeulike.org – Social bookmarking for papers. Closed Source – Open Data.
http://www.scholarpedia.org. A peer reviewed open access encyclopedia.
Philica.com Online Journal which publishes articles from any field along with its reviews.
MathSciNet/Zentralblatt – Review database for math community. Closed Source – Commercial.
http://f1000research.com/ - Online Journal with a public, post publish review process. “Open Science – Open Data – Open Review”
http://altmetrics.org/manifesto/ as an emerging trend from the web-science trust community. Their goal is to revolutionize the review process and create better filters for scientific publications making use of link structures and public discussions. (Might be interesting for us).
http://meta.wikimedia.org/wiki/WikiScholar – one of several ideas under discussion at Wikimedia as to a central repository for references (that are cited on Wikipedias and other Wikimedia projects)

Upshot of all this:

There is not a single site featuring good Q&A features for papers.

If you like our approach you can contact us or contribute on the source code find some starting documentation!

So the plan is to fork an open source question answer system and enrich it with the features fulfilling the needs of scientists and some social aspects which will eventually help to rank related work of a paper.

Feel free to provide us with feedback and wishes and join our effort!