Machine Learning in Kyoto

The usual excuses get very boring very quickly, but yes, the blog has been sadly neglected for the last couple of months while the world has spun around and I have been getting every more dizzy from trips to all corners. Since the last post I've covered around fifty thousand miles, and taken 14 flights. My carbon offset at the end of the year is going to be painful but necessary.

24 hours after landing in Munich from another long-haul flight, and a quick laundry trip later I was back on a flight to Osaka, to attend the Machine Learning Summer School in one of my favourite cities, Kyoto. I'd spent about six weeks in Kyoto before and had found it not only beautiful but also one of the most relaxing cities on Earth. I had spent an extremely constructive few weeks working late into the night in cafes and the Yukawa institute, where somehow the surrounding temples brought and amazing peacefulness to my work, and while it was feverish at times (waking through the night thinking in code) it was always in a zone which I've achieved on very few other occasions.

This time was also amazing, but not peaceful in the same sense as before. With 300 participants from 50 countries, the machine learning summer school was a boot camp in theoretical machine learning, with 7-8 hours of intense lectures every day. I was there to pick up some new skills to add to my arsenal to solve complex computer problems. It was a fascinating couple of weeks and while the lectures were far less applied than I was hoping, I still picked up a lot of useful new information, many great academic contacts and a few good friends along the way.

Kyoto was a balmy 35 degrees most days and cycling around by bike was the best way to tame the humidity and temperature with a good breeze. I was reminded once again how much I love to cycle around unknown streets and I did a good deal of sightseeing at the weekends and in the evenings, when I wasn't simply exploring the food.

Having read The Hare with Amber Eyes last year I was keen to see the Netsuke museum which was near to the flat of the couchsurfer that I stayed with for the first two days. Sadly it was closed, but nearby there was a fascinating gallery detailing the history and techniques behind the kimono,  a more intricate artistry than I had ever imagined, and something that is worth checking out if you're ever there.

The one temple I most wanted to get back to was Kinkakuji, the Golden Temple, nestled into the hills on the outskirts of the city. I'd been there twice before but the sheer ridiculousness of this place requires a good few viewing to really take it in.

I spent a couple of hours happily snapping away as tourist groups piled past. Sadly the opening times and sunsets in Kyoto didn't coincide but still, the conditions were pretty good for some HDR shots from under the trees:

A serious downpour at one of the hidden temples near the hotel:

There wasn't time to see nearly as many temples as I would have liked, but taking a little time out at the weekends gave me a good chance to try and wind down from the intense days during the week. Trying to get on with various projects throughout made it a very intensive few weeks.

The end of the summer school saw a rather fine Italian/Japanese banquet as well as a cultural performance. I was expecting to enjoy it as the traditional artforms I've seen in the past, Kabuki in particular, I've enormously enjoyed. However, this was a performance of Geiko (Geisha from the West of Japan) and Maiko (trainee Geiko). As soon as it started however I felt incredibly uncomfortable. This may partly be that I was projecting my own cultural perspective on it and not appreciating the details of this important part of Japanese culture, but the movements and looks felt so incredibly subservient from the women dancing in front of a crowd of mostly men, that I felt immediately wrong to be watching it. I went outside to think for a while as the performance continued but I haven't yet come to terms with quite what was going on. I'd be very interested in hearing the thoughts of a Japanese person with a good exposure to Western culture for them to be able to tell me how they see it. This time in Japan there were a few things that felt very strange, from the incredibly sexualised hello kitty young girls to the immediate hierarchy you feel when going into a shop where everybody greets you and bows to you. I like to be as open to cultural habits and ideas as I can be, but while I love many things about Japan, there are some things that do make me feel uncomfortable - this is a strange position for me to find myself in.

After the two week Machine Learning school it was straight into a few days at the Yukawa institute where I was giving some talks about my research (hopefully coming out this week!) and a colloquium on atmospheric optics. A good time talking with Shin Nakamura and Tadashi Takayanagi and I was back on the plane to Munich again, another couple of weeks in Munich and then off to China, but China will have to wait till next time...

Of air and water and light - solar halos over China

My flight to Japan was a remarkably comfortable one. Last week saw me in the sky for over 70 hours and thoughts of DVT played heavy on my mind as I got ready for the last leg of a rather mammoth few days. Days that I would however repeat, travel included, given the chance. Thankfully on changing at Amsterdam airport the ground staff took pity on my grizzled face and booked me into a seat with two spares next to it so I could finally stretch out.

I managed a couple of hours sleep on the 11 hour journey which is not a bad proportion for me, given that I normally manage none at all. After a recent flight where, on asking for a whiskey, I was presented with a good half pint of the stuff, I have been staying away from alcohol on flights. The whiskey did knock me out, but it also gave me a horrendous hangover for the waking hours of the journey.

Anyway, I was up to watch the sunrise over China and managed to see the faintest of green flashes, though sadly didn't get it on camera this time.

Later on in the flight however as we neared Japan the air at 30,000+ ft became filled with ice clouds and the interplay of light and ice made for a truly spectacular display.

The conditions were very interesting. Around the plane was a thin layer of cirrus clouds, with a high density of ice crystals, and below us where much thicker clouds, also packed full of ice crystals, many of them plate-like. Plate-like hexagonal ice crystals like to lie flat in the sky and act like mirrors to the sun. These are the crystals that cause sun pillars:

and sub-suns (the reflection of the sun off ice crystals in the clouds below):

This time however the display was a lot more complex. The column crystals in the clouds around us caused a 22 degree halo while the plate crystals gave a faint sundog (also known as a parhelion). In the thick clouds below however things got more interesting. The plate crystals below gave an effect that I'd never seen before. Not only did the light refract as it was passing through the sides of the crystals, but it also bounced off the bottom face, and back up, acting like a mirror to give a so called subparhelion. Moreover, because of the high density of crystals in the lower layers there was a strong 46 degree halo coming from a rather rare dynamic of light through the end of column crystals. It seems that this may not have been photographed before.

Here is the almost undoctored photo:

and here is a rather more doctored one just to enhance the effect to see the different arcs more clearly:

The reason that the circle around the sun is so distorted is because it's from the edge of a very wide-angle lens (Sigma 10-20mm at 10mm).

I sent this through to Les at Atopics who has been my source of knowledge and inspiration in the subject over the last few years and he sent me back a ray-trace computer simulation of what was going on. This is the simulation of the particular conditions so you can compare with the images above (click to view the full image):

The 46 degree arc in the clouds below seems to be an extremely rare event and the infralateral arc is also something rather special. All in all one of the best halo displays I've ever seen.

I've seen halo displays a few dozen times when flying now. Take some sunglasses with you, look out the plane and see if you can spot them next time...they're are there to be marveled at.

Now in Japan for the next 3 weeks. 2 weeks at a machine learning summer school and then a few days at the Yukawa institute where I will be giving a talk and hopefully chatting with some of the experts on entanglement entropy in holography. There are some ideas I need to talk with them about...

Reboot

I've been silent here for a long time. I've also been in a bit of a reboot phase for the last few months, probably since around the end of June. I have a confession to make, that the reason for this was that the first six months of this year I simply took on too much. I'm not very good at saying no in general and have always loved the buzz of being busy. To be bored is a waste of the incredibly short time we have here and I find myself frustrated by all the other things that I wish I could pack into the hours of the day and of the night.

A couple of posts ago I wrote about the incredible new explosion in online learning tools (take a look for links), from Khan Academy to Coursera and Udacity and now with EdX in the mix, there are more and more courses online every day, for free, for anybody connected to the web. Some of the best teachers in the world offering their services in a fascinating business model to bring highly advanced skills to the masses. On Coursera alone there are now over 100 courses, from Natural Language Processing to Sociology, from Vaccines to Computer Vision and from Automata theory to the study of Modern and Contemporary American poetry.

The opportunity was too much to miss and I jumped on board in a big way. I signed up to every course that looked interesting, and dove in, head first. I new from the start that it was going to be busy, but didn't know quite what I was taking on or how I wasn't going to be able to say no to finishing the assignments, even if it meant sleepless nights. And that it did. In fact I spent a good few months at the beginning of the year working 18 hour days. I was working in the office during the day, working on assignments and watching course videos at night and then at the weekends working on  other projects, some of which are now finished and some of which are ongoing (working on interdisciplinary areas has been enormously fun!). It was a huge buzz to be doing this and I felt in a great zone. The courses finished off one by one but as they did so a new one started.

South Africa came in the middle of all this and I was teaching a course on a subject I'd never taught before, while giving lectures about my work as well as working on the courses at night. It was all a bit too much.

From South Africa I came back and went off to Denmark, then to the UK then back to Germany, then somehow the Netherlands crept in and I spent a few days in Leiden after a trip through Köln and Neijmegen. Again, more talks, more work, more courses. I don't really remember July, it came and went in a flash with Strings at the end

The next week is going to be the most ridiculous of all time in terms of travel, the last piece of it will be a trip to Japan in about 8 days to spend two weeks at a summer school before spending a few days at the Yukawa Institute where I will give a talk and hopefully be able to discuss with some of the world experts on a problem I'm looking at at the moment.

There are also plans afoot for next year but I'm going to keep these somewhat hidden for the moment. When there is any movement I will talk about it.

So, After working for the first six months like crazy, the courses slowly trickled off and my concentration span trickled off with them. I emerged from this intense period in a completely hyperactive state and unable to concentrate on any one thing for more than a fraction of the time I'd normally be able to commit. I found myself easily distracted and even the simple pleasure of reading a book, which is my normal wind-down activity didn't seem to be happening.

A few weeks passed by like this and I knew that I really needed a break, things were not improving, I felt fully burnt out. There are friends around the world that I would love to have seen, but I knew that this time it had to be a holiday of withdrawal, a trip where I could get away from everything and reboot. I wanted to go somewhere with very little to do, with beautiful streets and cafes where I could sit and read. More than anything I wanted somewhere that I didn't have to fly to! After a little searching I settled on Ljubljana, Slovenia, a mere six hour train ride from Munich. I was recommended this by a Slovenian friend and at the same time was told of all the fantastic natural wonders to go and visit in the country. Normally I would jump at such photo opportunities, but this time I simply wanted a week or so of doing nothing!

And that's what I managed to get. I spent 9 days in Ljubljana doing nothing but sitting in cafes and reading, occasionally talking with street musicians who I would see every day as I wandered around, ate some good food and spent the afternoons exercising. It felt like a true retreat. I didn't have to speak to anyone, I didn't have to think about emails or facebook or traveling from one place to another. I read some great books, probably my favourites being The Brothers Karamazov, Religion For Atheists and Cosmic Anger, the biography of Abdus Salam. I sat in the castle at night under the stars watching the open air cinema, I sat and did nothing, it was perfect!

So, I am back now. Now I go away again and and will be gone for a few weeks, but I'm feeling a lot more ready to get my head down and concentrate, to finish the projects that are ongoing and to start some new ones, as well as to try and figure out a bit more what next year may hold.

For now I'll leave you with a photo from Ljubljana castle. The city is surrounded by mountains and as the yellow light of the sunset cut through the valleys, this castle was bathed in the glow:

The meal of a lifetime at NOMA

Things are beginning to slow down, with a couple of the major Coursera courses coming to an end. I will write up a post when possible about the Natural Language Processing course which has been spectacularly well taught/paced and executed in just about every way.

I've been wanting to write up a post also about my trip to Denmark a week or so ago, on a gastronomic pilgrimage which lead us to the best restaurant in the world (whatever that means). A trip three years or so in the making, with all the possibilities for dire disappointment, but eventually a trip which was with out a doubt the highlight of my life's food experiences so far. Much of this may seem a little Emperor's new clothes, but to be honest I don't mind. If 90% of the enjoyment was in my head, rather than in the tastes themselves, then fine, that still resulted in a great deal of enjoyment.

Noma was the destination, and being my first time in Denmark I planned on making a few days of the trip. We had booked the table at the beginning of the year, having tried and failed for the last 3 years, ever since it took El Bulli's place at the top spot of the gastronomic hierarchy.

I was lucky enough to have hosted a very friendly Dane through Couchsurfing when I was living in Spain and this gave me the perfect chance to crash his place. This worked out fantastically as well as it gave me, as couchsurfing always does, a view of the city which I almost certainly wouldn't have seen had I stayed in a hostel.

Having returned from South Africa to a very sunny Munich, the weather continued to follow me and the first day in Copenhagen was a spectacular 25 degrees. We spent the afternoon sightseeing and sitting with a drink in Christiania, catching up on the last couple of years of our lives and watching the world go by.

Konrad, who had booked the table, turned up on the Wednesday evening and we went for dinner and a drink in the centre, sampling Copenhagen's only CAMRA rated pub where the price of the pints was as much the cause of weakened knees as the strength. Retiring after a couple of pints we cycled back through the city at night, ready for the feast which was to come the next day.

Booked for lunch at one, Konrad, his friend and I headed at a leisurely pace through the city (having warmed up our stomachs with a fine breakfast), still in full spring haze to the North Atlantic House, the ground floor of which houses NOMA, done up in relatively unpretentious decor

The second you step through the door you are made to feel special. You forget about the fact that the place itself is special, but the waiters are immediately interested in you, asking questions and making you feel like you've just met a best friend of a best friend.

They introduce the concept of the restaurant: Nordic inspired cuisine, with Nordic ingredients in highly seasonal, frequently unusual combinations. This is not another El Bulli, where the food was all about the processes - a lot of incredible molecular gastronomy, but is really about the freshness and purity of flavours and the subtlety of their combinations.

I know very little about music and I wouldn't claim to know all that much about food, but, at a piano concert a few years ago, a friend who really does know a great deal about music told me that the brilliance of the particular performance was not about getting the notes in the right order, or at the right time, but was about the subtlety and exactness of the strength of those notes.

In the same way, at NOMA you are not going to be blown away by whizzes and bangs in your mouth, there is not the satisfaction of a great steak or burger, or the sugar and fat buzz of a tiramisu that may satisfy for a few moments, but there is an incredible harmony of flavours that are so finely woven together that one follows another follows another, each one creeping up as the other fades, just as the notes of a brilliant piece of music follow in perfectly weighted succession.

For me the pinnacle of this melody was a dish of scallops, sepia ink and biodynamic grains in a pea puree. The sepia ink gives a background platform which carries the first hit of the scallops, which have been processed for around a day, purifying their flavour and turning them almost into caramel wafers, and finds its way into your mouth before the scallop is even there. The flavour of the scallop slowly fades as your mouth habituates to it, at which point the grains and puree are building to a head and take over, the chemical similarities between the scallops and peas allowing this to happen seamlessly:


The very first dish was actually sitting on the table when we arrived, in the form of malted bread sticks, hidden within the flower pot and dusted with juniper (these are in the back, the strange rather furry looking things sticking through the foliage).

Along with the 23 courses came 9 incredible wines, all white, bar one rosé, all from Europe and there were some outstanding pairings with the meal, including a wine which was almost calvados in its depth.

After almost 4 hours we were left wonderfully satisfied, slightly drunk and enormously happy. Konrad and I went to a bar after this and sat in the sun, giggling like little kids about individual bites, about surprise flavours and about the experience as a whole, sipping a cold beer and lapping up the waves of emotion.

The day continued in much the same vein until around 6 o'clock the next morning, when I returned exhausted to collapse asleep for a few hours before having a very easy day the next day.

I shall simply leave you here with a slideshow of the meal. It is an expensive meal, but I can say that for anyone who loves food experiences, this is truly one of the greats, and for me, as a once every few years extravagance, I am willing to forgo a long holiday for a journey like this.

 

I should also add as an important postscipt, that I was hugely taken by Copenhagen and by the Danes in general. I would certainly love to return some time and spent a bit more time exploring that part of the world.

An attempt at a Cape Town catch up

Time simply doesn't allow to do all things I want to do, and to write about them. As of the last post my life has been taken over by the online courses I mentioned, and this means working till 2 most mornings to get through the material while having a normal time in the department during the day. I wrote the following when waiting for my plane in Cape Town, and while it's by no means a polished blog post, I want to post it before I disappear again tomorrow morning for Copenhagen. The next 3 days are part of an absurd trip which has been in the planning for several years and will come to its climactic conclusion on Thursday with lunch at NOMA.

-----------

I find myself once again sat in a cafe, in an airport, waiting for another 12+ hours of plane flights. This time I'm in Cape Town, waiting to go via Johannesburg back to Munich. Another overnight flight, another night of watching others loll in and out of sleep as I wander the aisles trying to get tired enough to ignore the discomfort of folding my frame into a spasm-inducing gap.

Anyway, the above sounds a bit negative perhaps and the rest of the post, I hope will not be. I've met enough interesting people on flights now to know that flights have the possibility to be as enjoyable as they are ghastly.

I've spent the last three weeks in Cape Town, my second time here, and each extra day I spend here makes me realise even more what a wonderful place it is. Scenically it is almost unrivalled for a city. With the sea surrounding it and the mountain defining its geography, you are never away from a spectacular view. I've been for a couple of sundowners (a phrase I'd not heard before but means, unsurprisingly, drinks drunk during sunset, generally with a majestic view to accompany them) which have been wonderful evening highlights, a time to really sit back and appreciate.

The city itself is made up of an incredible diversity of populations and cultures, the mixing of which is often not present but always available. I've met some incredible people here, in taxis, in shops, in cafes, from refugees of Rwanda to ex-militants who fought in Umkhonto we Sizwe, the spear of the nation, the group that was the force behind the ANC when talks and peaceful action only managed to tighten the grip of Apartheid. Everyone here seems to have a story to tell and I've met so many people who are actively pushing to make the situation better. In many ways South Africa has come a long way since Apartheid, but in others it has gotten nowhere. There are many embarrassments of the current social and political situation but there are people out there who are really trying to make a difference in a non political context. One of the taxi drivers I met also worked as a coast guard guarding the local ecology from abalone poachers, who often come armed with guns during the night. This is a man who puts himself in a place of enormous danger because he cares - lessons to take from this, for sure.

I am in no way qualified to make judgement calls on the current situation but I can recommend these books that I've read over the last couple of years to give some picture of South Africa's history and recent developments. Sadly time does not allow for in depth reviews of any of them but I can say that from the point of view of someone who knew very little about Africa or its politics, these are great starters (Thanks to Ben for all of these recommendations):

The State of Africa - a history of the states of Africa since independence
Country of my skull - a journalist's eye view of the truth and reconciliation commission
My traitor's heart - a white man with a difficult family pedigree coming to terms with his place in Africa
The shackled continent (don't read this without reading The State of Africa as it will give you a terribly biased view) - a view of why things have not improved since independence. As I say, this does not give anywhere near the whole story and for the reason behind these reasons you have to read more.
Long walk to Freedom - Mandela's pre-freedom autobiography.
The autobiography of Ghandi also gives some very interesting insights into the relatively recent history of South Africa.

The trip has been as eclectic a mix as the city is and I've been both giving seminars as well as starting some collaborations with the string theory group. I gave my atmospheric optics talk to the undergraduates which is always a pleasure, though I still struggle at times not to include too much material in this talk. After this I gave a series of lectures on a particular method of machine learning as applied to neuroscience. This was my first time lecturing on a non-physics subject and it was a great experience. The subject is fascinating and thankfully there were a huge number of questions which I find always make a lecture course a lot more enjoyable. It also makes me, the lecturer, understand the topic much better when you have a lot of intelligent people giving you questions that perhaps you would never have considered.

Some light photo relief:

 The outrageously beautiful University of Cape Town campus

Ivydene guest house, where I would sit and work at the weekends under the tree with the dogs and cats resting in the garden. This is without a doubt my favourite guest house having stayed at many over the last decade or so. Lucille, who runs it is absolutely fascinating and introduced me to a slew of amazing people while I was there. Her daughter Jackie and son-in-law Rob also make the place feel like family.

And finally I gave a couple of talks about one of my last string theory papers which again was a lot of fun. The group ranges from those working in emergent geometry (how space and time can emerge from some other theory which don't include space and time as input) to AdS/QGP - how we can use string theory as a tool to understand the internal constituents of the nucleus in particular conditions. It's always good to have a diverse audience like this because, again, you get a great range of questions, from the abstract to the very practical. The outcome of this was some really nice ideas for future investigations which I hope to continue remotely.

and apart from a little socialising my life has been almost entirely overtaken with the online courses that I discussed in the previous posts. As an information addict these are truly fatal. How can I turn down a perfectly good course on automata theory given by a world expert, even if it means sleeping a few hours less a week.

The first round of courses on computer science finished a couple of weeks back which I can highly recommend, CS101 for those without much experience in programming and CS373 for the more advanced and those who may be interested in control theory and dynamical programming.

The course on natural language processing (NLP) is now over half way through and is the best taught and most instructive from the point of view of programming assignments and problem sets, of the Coursera courses I've been taking. Right now I'm struggling, along with most of the rest of the class to write our own English language parser (how do you write a computer program which can work out the logical structure of a sentence - ie. which nouns do which verbs and prepositions relate to - in the sentence "fish people fish tanks" does the word people correspond to a verb or a noun - does the second 'fish' correspond to a very or a noun, etc. These are the sorts of question that an automatic parser would try to answer in a probabilistic fashion (ie. it will find various ways to break down a sentence and give probabilities for each possibility)).

The NLP course also ties in very nicely with material in the automata course and the programming languages course from Udacity. The link being finite state machines and Markov networks.

Probabilistic Graphical Models is also very good, but it really does need 10-15 hours a week to truly understand the material. This is a large investment and something that I will probably split over two runs through the material.

In other news, the publication I was working on about Malaria has now been accepted for publication which is great to have finished. More work is ahead on this front and there are some very exciting possibilities in the pipeline. Through the same link that gave me the Malaria connection (many thanks Victor!) we've started also on another investigation into an area called medical ethnography - a subject whose existence I was unaware of until I dove into some data analysis on a related project. This area is related to the interaction between society, ethnological behaviour and medical practice and is so far very interesting to work on.

So, with the above I have very little time for a normal life right now. I work until late at night including at weekends on these projects and courses, which is great in terms of learning and in terms of the excitement of pursuing new areas of investigation but I am aware that this is not a sustainable pace. I am also aware that I am out of a job in 7 months and am desperately trying to mold a future which will be interesting, sustainable, economically viable and will allow for some constancy in my life. Most of these factors are falling into place, it's just the small matter of economics which may be the biggest hurdle. It's easy to find people with whom I can work on interesting projects, it's not so easy to get them to pay me for it. The thinking cap is on and I have a few ideas as to how to construct this Rube-Goldberg machine of a life out of gaffer tape and fuzzy felt.

2012 - bring on the learning revolution

(I should note that the title is taken straight from Ken Robinson's TED talk)

I'm exhausted at the moment, but hugely excited. I've been spending evenings over the last three weeks, usually till the early hours, experimenting with a new development in online education.

The Open Courseware (OCW) movement started in 1999, in Tubingen, Germany, but quickly spread to the US. The idea is simple: Take the best educators at the top universities around the world, video their courses and put them online, for free, for anyone to watch. Over the last couple of years I've watched probably a dozen or so courses on subjects from human behavioural biology, to quantum field theory (admittedly this is a little late, being Sidney Coleman, who passed away before the videos were put online), to existentialist philosophy, from science and cooking to biology at MIT. Some of these may not fall under the umbrella of OCW officially, but the idea is the same: As of the last decade, there have been a wealth of amazing materials online for anyone to access. Open Culture keeps a pretty good track of the best of the courses online here.

These courses were often fantastic to watch and I learned a great deal, but there was always a large piece missing from these sets of audio or video materials, which is that they were very much uni-directional. You sat, as a viewer, and took in the information. It turns out that this sort of learning is pretty inefficient. Building up passive understanding of a subject is one thing, but building up a true working understanding of it takes exercise and effort. The act of letting it wash over you is not enough. In fact even going through the material yourself in 'revision' mode is pretty ineffective (there's a lovely paper here detailing precisely the effects of various types of learning methods).

So, come the end of 2011 things started to change. Two courses were offered online from Stanford whereby not only were there video lectures, but there were online exercises in the form of multiple choice problem sets and programming assignments, all of which were marked automatically. Suddenly you were forced to understand the material (though in the case of the hugely successful Machine Learning course I'd say that, given that this was a first run, the tests didn't actually examine how much you understood the material, but more the programing language - in this case Octave - this will change in future revisions I'm sure). The other course, on Artificial Intelligence, gained over 100,000 participants, and suddenly the whole thing exploded. Sebastian Thrun said that, having lectured to so many students, lecturing to a classroom full of 200 enormously lucky individuals no longer seemed terribly efficient (I'm misquoting and paraphrasing enormously) and from this vision of the future, Udacity was born. Some of the quotes can be found in the article here which gives more outline of his vision.

Udacity is one of several platforms which have started in the last few of weeks. Udacity aims, eventually, to give the material of an entire degree course in computer science. The modules are not coming online in a linear fashion but this means that whatever your knowledge of programing there will probably be something for you.

I'm taking part in the first two courses. CS101 teaches you to build your own search engine, and is really a very basic course in Python with some simple implementations of web crawling and web searching but for a beginner in Python it's perfect.

CS373 is a more advanced course and expects that you already have a good working knowledge of Python, although you could pick up what you need to know for this in a matter of a few hours. The course is based around programing a robotic car - like the car that Google used to win the Darpa Grand Challenge (a challenge whereby an automated car has to make its way through long and complex real-world road situations). In this course you learn a great deal about robotics and designing intelligent computer systems for navigating complex environments.

Udacity is going to be offering more courses starting in April. Each course, I believe, will last on the order of a couple of months and at the end you get a certificate based on the marks you get in your homework assignments. Typically there are an hour or so of lectures a week and the lectures themselves are built around solving problems for yourself - you learn as you move through the lectures by solving the puzzles as you go. Here is a screenshot from one particular programming assignment in the course - not homework, but just a question that is given as you move through the second unit. This is all done online and you program directly into the browser.

You can see above the format of the class whereby not only are there lectures and homework, but also a hugely active discussion area where students discuss the course, pose questions and build understanding together.

So, that's Udacity. At the same time, another organisation, Coursera has come online and is offering 15 or so courses on everything from Natural Language Processing to Anatomy.
Right now I'm taking part in Model Thinking, Probabilistic Graphical Models, Natural Language Processing and Design and Analysis of Algorithms 1. Each of these have several hours of lectures a week, so I'm downloading them and watching them with VLC at a faster speed, slowing down for the more complex ideas. The format is very similar to the Udacity courses, though the lectures are less active for the watcher on Coursera. Again, the emphasis is on plenty of exercises to make sure that you really build understanding as you go through the course. Here's a screenshot of the Natural Language Processing page:

Here's an introduction to the model thinking course:

and on probabilistic graphical models:

In addition to all of this, MIT, through MITx is offering their own format for an online course starting with MITx 6.002 Introduction to circuits and electronics. This really astounded me. Not only do you get the same extremely high quality video lectures, homeworks, discussion forums etc. but there is, built into the interface, a virtual electronics lab for you to experiment with and use for the assignments. For completeness I'll include a screenshot of this interface as well:

They also have an ever-evolving wiki page which will allow the students to write their own online version of the course which will evolve with ever-better explanations of the concepts discussed.

What is immensely important with respect to all of these online courses is that every one of them is completely free!

As of this year, and, I predict, in vastly increasing numbers, you can now get some of the highest quality education in the world, online, for free. Anyone, in any country, with an internet connection, can register, log in and learn with some of the best educators in the world. This is mind-blowingly powerful stuff! I'm hugely excited to see what this will do for the younger generation of students (high school included) who are dying to get immersed in all this stuff but up until now simply haven't had the resources (not only the material, but also the interaction that this will now allow).

I should add incidentally that TED is also getting in on the act, having launched TED Education this week:


A discussion of online free learning would not be complete (and this blogpost is certainly not that) without a mention of Khan Academy, another incredibly exciting venture which looks to have the potential to revolutionise highschool education:


Anyway, I highly advise delving into these courses and even if something doesn't pop out now that appeals, I think that within a very short space of time, the vast library of online, interactive information available will have something to tickle your brainbuds.

So, for now, I have a little more to watch and work through tonight (for the last weeks I've been working through from about 8-1 or 2am at home, after work). I've taken on a pretty big load with all of these courses and am not aiming at perfection but for now just exploring the landscape of possibilities, so far it's a lot of fun...

P.S. I've shared this before, but this is always worth putting out there for anyone who hasn't already seen it. Sir Ken Robinson is one of the most wonderful rhetoricists I've ever seen, and I've had the pleasure of seeing him live at a conference in London. Here describes how the current education system is badly broken and needs a massive rethink. Is education killing creativity?:


Also, find out more here at Learning Without Frontiers, about how disruptive technology may be able to shift the status quo.

Beijing talks, Munich talks

Physically and mentally pretty drained right now. The week has been crazy and I am in dire need of another weekend. Today I've had a fascinating meetup with a researcher in memory who is, himself a frequent competitor in the world memory championships and is in the top ten in the world in a number of disciplines. His work is specifically about how memory techniques actually work, so we spent a very interesting couple of hours talking about this research and some of the fMRI studies that they have been carrying out.

I don't have much energy to write a lot at the moment (mostly thanks to my first squash game in seven years yesterday, which has left me barely able to walk) but I was sent some photos recently from last year when I was in China. I combined time at a conference at the KITPC with a number of talks to research departments about my work (at the ITP and the IHEP), but I also gave talks at a number of highschools around Beijing about frontiers in modern physics, about the history of the universe and the search for the Higgs boson and about why pursuing studies in physics is a valuable thing to do. Without exception I had dozens of excellent questions and at one particular school I have now been made an official scientific consultant (voluntary) and so act as an external expert on science questions when they have particular problems that they want to find out about - I generally direct them to resources as well as providing my own explanations. This school is also trying to set up a regular visiting scholar program whereby scientists who come to Beijing to do research at one of the many departments there can also come to the highschool to talk with the students about their particular line of research. This initiative is really to get the kids interested in relevant research and to bring something more cutting edge to the curriculum.

Anyway, this week I was sent these photos from one of my talks where I was talking to about 200+ students from four schools around Beijing:

Anyway, this was a lot of fun and I hope to do the same next time I'm back in Beijing.

Two weeks ago I also gave a talk at the Munich Nerd Nite. The talk I gave was on atmospheric optics and it was the first time I'd ever given a talk in a bar. There, also was an audience of at least 200 though there was a lot more beer involved than at the highschools. This time the talk was recorded and so here is me giving what I normally give as an hour long talk in fifteen minutes, hence the rather fast pace of it. Also note that my explanation for the 46 degree halo is incorrect - it's got nothing to do with pyramidal crystals, but comes from a very particular refraction through normal column crystals. My talk starts about 3 minutes in...

Anyway, that'll do for now. I've got another hour or two of work to do tonight and then I'm going to have an early night!

The weeks fly by

It's been an immensely busy week and by yesterday afternoon I was already feeling spent. The previous blog post that I put up garnered a lot of interest and I spent a good few hours replying to emails with advice and support - many thanks for the links, the tales of other people's similar experiences and the general positive notes.


I spent the evenings before Thursday this week getting a talk ready to give at Munich's monthly Nerd Nite, held in a bar and usually with an audience of at least 200 of the cities keenest nerds. My talk, on atmospheric optics, was one I'd given a good few times before, to both physics departments and to the general public, but normally the talk lasts around an hour. Trying to condense it down to 15 minutes was not easy and I fear that my enthusiasm mixed with my naturally fast talking pace made it all a bit rushed. Still, the audience seemed to enjoy the pictures at the very least and so it was a good experience all in all. The previous talks of the evening, on polyphasic sleep and on the neuroscience of dreams were also interesting, though being in German I didn't get more than three quarters of the content.

Speaking of which, my German is coming along painfully slowly. I've upped the lessons to three a week, one with a private teacher and two in a beginner's class at a language school. The language school has a bit of a strange format however, with a very relaxed 'drop in when you want' approach which means that the class, normally of six to eight students, is made of of some who know nothing at all, and some who have been studying for a few months, so we spend a lot of the time with the teacher asking us what we'd like to do. This isn't her fault, but a problem with the format. Still, I have another couple of months there, and it's cheap and for now any input is good. I will also be starting a language exchange next week as the aspect that I have to work on now is simply getting out there and speaking. I can go for an hour or so talking in German with my private teacher, but it's ponderously slow and so I'm very self-conscious about doing this with anyone but my teacher for now - this is a natural language-learning hurdle that I need to get over.

On the physics front two of the projects are making progress and we've come up with a new way to tackle a rather interesting problem, but I'll leave the details of this for any papers which may come out of it.

On the front of various other directions of study I've been looking a little into ICA (independent component analysis), a method within machine learning, which is really a fascinating area. The idea is pretty basic, though the implementation can be complex and subtle, and can be explained with a very simple example. Say you have two microphones in a room and you have two people speaking at the same time. Each microphone will record a slightly different linear combination of the two voices. The question is then how you can take the two recordings, and knowing nothing more than the fact that the two signals were in no way correlated, disentangle them. It turns out that there is an extremely clever way to do this, and basically you can disentangle any form of uncorrelated signal, be it a visual signal, an auditory signal, or the readings from a brain scan. The latter use is the one that I'm most interested in, but I wanted to play around with doing this for mixed audio signals and images first, and with a few lines of Mathematica it turns out to be very simple. If you are interested in knowing about how this works then I can either recommend having a read through this tutorial, or ask me if you'd like more details about what I've implemented in code.

Of the online undergraduate courses that I and a number of friends are taking, three have now started: Model Thinking, plus Computer Science 101 (building a search engine) and Computer Science 373 (programing a robotic car). These are all very nicely presented but so far CS101 looks to be rather basic unless you don't know any programing at all, in which case it will probably be a great introduction. Model Thinking is actually surprisingly insightful, and just with a few examples he has shown how powerful this subject can be. The most interesting example came from looking at segregation within cities and discussing how actually you can have people with a relatively high tolerance for having others, unlike themselves living around them, but still, over time, end up with a highly segregated population. Such unintuitive results are very clear to see once you write down a simple dynamical model of such a situation.

Anyway, another busy week beckons. This is going to continue unabated for the next three weeks, with a number of couchsurfers coming to visit along the way, until I head to Spain for a long weekend in March - I'm going to need the break by then...