Sunday, February 25, 2007

Yahoo Pipes

From an article on the BBC I've started playing around with what at first seems nothing more than a bit of fun from Yahoo, but it looks like it could be pretty important stuff.

Yahoo Pipes allows you to take the data from a website and pipe it through an operation, that is, to do just about whatever you want with the content.

I've just been playing around for a bit but so far I've taken the rss feeds from hep-th arxiv and passed the content of flickr through a filter searching for words in the hep-th feed. The outcome are photos with titles like:

gluon, blackhole, derivative, topological string theorists, etc.

This is the user interface and my simple pipe example (unreadable at the moment, I'll try and sort that out) which requires no programming knowledge as far as I can tell.
pipes
OK, this is a trivial example, but I presume it would be simple once they put the documentation online, to pass blog content through to find out who's writing what about which papers in the furthest corners of the blogosphere.

Examples of content manipulation are to filter estate agent's sites just for apartments near your area (here). This means that rather than other people's sites being simply static, or in the case of wiki, editable but still static to some extent, sites can be altered dynamically to give you exactly the content you want.

Once you've created the pipe you can publish it at the press of a button so that other people can use, copy and edit the operations you've implemented.

This looks like an exciting, powerful tool and relatively simple, as I say once there's more extensive documentation. I'd be really interested to see how this takes off and if anyone comes up with more useful pipes than my simple example then please let me know.

It looks like the full documentation will become available next week (see here for more)

4 comments:

Kevin S. said...

My current understanding of Pipes is that you can mix content together in the form of different items in a feed, but it does not let you create new content based on some given content. If I'm wrong about this please let me know, and give me an example so I can see how it's done.

I've been trying to build a Pipe that will analyze a given online Chinese article, compare it to an online text file of Chinese characters I know, and then create a glossary of the characters in the article that I don't already know. I've failed because I can't figure out (or it's not yet possible) how to create content using it.

Jonathan Shock said...

Hi Kevin, Sorry, in my rush of feed reading in the morning I hadn't noticed the import of your timely article. You do indeed get first prize for noticing 'the internet technology of 2007'.

I think you're right that you cannot create 'new content' but you can clearly filter through content of one site given a criterion based on the content from another.

What sort of new content would you like to be able to create?

I think until the documentation comes out and we all do a bit more playing around it will be hard to see the real possibilities. I'm also not sure how far yahoo has gone with the applicability. I guess we should know in the next few weeks. Unfortunately the pipes that people have created so far look pretty limited.

I look forward to seeing how you do with your current trials.

All the best,

J

Anonymous said...

jon,
this sounds very cool--peng

Jonathan Shock said...

Hi Peng,

Please tell me what you find, I think that this will be a collaborative learning experience until they tell us what it can really do.

J