Personal PhD

Hard work pays off [Personal Phd update #2.4]

~ diana ~ Leave a comment

At work a teammate started a reading group to read book Deep Learning by Ian Goodfellow (https://goodfeli.github.io/dlbook/). Even though it is ahead of my curriculum I can’t miss the chance to read the book and discuss it with my coworkers. We are now at Chapter 8.

This book is definitely harder than what I have been studying so far, but there is one thing I am very glad for. The progress I have made with the Coursera’s ML course is directly relevant in better understanding the deep learning material. For example in the Coursera course I learned really good basics on regularization and now reading about fancy methods of regularization in the Deep Learning book it is easier to understand what’s what.

Other than that, I am behind the schedule on other things, but I still have 2 weeks left in this quarter 🙂 Now I am off for a week of holiday visiting family.

Happy Holidays!

Personal PhD update #2.3

~ diana ~ Leave a comment
So I am totally behind on writing these updates, the last one was almost 3  weeks ago. But hey, better later than never.

 

The Coursera course on Machine Learning is also a bit behind. Right now I am almost done with week 7. In order to finish the rest of the course by end of December I would have to go at a pace of one week of lectures in a week’s time as opposed to twice as slow. Doable, but holidays are not exactly helping as I tend to be offline during vacations.

 

Take for example Thanksgiving! We managed to do an epic road-trip from San Francisco to Grand Canyon to Phoenix to Joshua Tree National Park and then back to San Francisco.

 

On the flip-side, I got a lot of reading done. I finished reading the Programming Collective Intelligence book (as part of Computer Science curriculum) as well as Watching the English: The Hidden Rules of English Behavior book (vaguely part of Linguistics curriculum, but hey, it’s my curriculum!).

 

Fun stuff
Are these names pokemon names or big data cmpanies? https://pixelastic.github.io/pokemonorbigdata/  (Thanks Matt for the link!)

 

ML in the news
Google open sourced Tensorflow: a library for Machine Learning. https://www.tensorflow.org/
Here is Wired article about it.

 

That’s all for today!

Personal Phd update #2.2

~ diana ~ Leave a comment
Here is another update on my Personal PhD progress. This time short and sweet 🙂

 

My progress
I have finished week 6 of the Coursera’s Machine Learning course by Andrew Ng. This section of the course was a little easier than week 5. Week 5 probably was the hardest week of all (had to implement a backpropagation algorithm). Week 6 talked about training error, cross-validation and test-errors and the usefulness of plotting them. Now I really understand how to interpret learning curve graphs I have seen at work! So another proof that taking this course is helpful for my work.
Also read few more chapters  (6 and 7) of the Programming Collective Intelligence book.

 

Machine Learning in the news:

Plan for Q2 (Personal Phd update #2.1)

~ diana ~ Leave a comment
A little on the later side but here is my plan for the second quarter of my Personal Phd (Oct-Nov-Dec 2015).
Computer Science track:
1. Finish reading Programming Collective Intelligence book (Chapters 6 to 12)
   – why? First 5 chapters have been great with putting Machine Learning in the context of other ways of solving problems, I am looking forward to the rest.
2. Do rest of the Machine Learning course by Andrew Ng on Coursera (Weeks 6-11)
   – why? First 5 weeks have been super useful and has instilled good basics including the math, so I will continue with it. I also like the quiz format and programming exercises as it forces me to pay attention.
(I am pausing on NYU Automatic Speech Recognition course as it is goes over my head right now).
Linguistics track:
3. Previous quarter I focused on brushing up on German. This time I actually want to focus on English as opposed to yet another foreign language. So item for this quarter is to read book Watching the English: The Hidden Rules of English Behavior.
   – why?  Here are the reasons: 1. At least in my personal experience so much of language understanding comes from understanding the culture. 2. My boyfriend is English 🙂 3. The topic sounds fascinating and this is my curriculum so I can make it whatever I would like it to be 🙂  yay, the benefits of a “personal” learning path 🙂
Paper Reading
4. Read 10 more scientific papers on Speech Recognition.
Writing:
5. Write an update every other week.
This quarter is going to be tight with starting quite late and having Thanksgiving and Christmas holidays, but will see what I can do 🙂 (during holidays I may get more reading done while on planes, etc., but not any studying).

Reflections on first quarter and how did I do? (Personal PhD update #1.6)

~ diana ~ Leave a comment
The first quarter of Personal PhD is done! I finished right around October 1st, but took a while to put this update together.
This update contains my summary and thoughts on how the inaugural quarter of my Personal PhD went.
Observations
Overall, I am glad I started this Personal PhD project! The first quarter was interesting and rewarding experience.
I think I learned more than if I hadn’t structured it. Having a structure and goals means I had to keep doing things and you all are watching me 🙂
There were unexpected side benefits from sharing this project with other people. Turns out other people are interested in doing something like this as well! I love hearing from others that they are interested in self-education as well.
I now keep noticing how much machine learning there is around us: to select fashion models, to select resumes, price prediction, etc.
Doing this has definitely helped me in my job. I know understand more of the lunch conversations my coworkers have, which is fun!
All that Computer Science theory that I learned in college that I thought would be useless in the real world — not true. Here I am, wrangling weighted finite state transducers (extension of finite state automata) at work for speech recognition.
Slight adjustments to my Personal PhD plan
I am big fan of launch and iterate, so here is an adjustment to the original plan of how to structure the Personal PhD time-wise.
Still keep quarters (3 month long periods), but spend the first two weeks of the quarter devising a curriculum, aim to finish coursework two weeks before end of quarter (also so that it doesn’t coincide with actual work quarter), then two weeks to finish any loose ends and to write up a reflection on the quarter.
 
Goals and what actually got done
Here is a summary of my original plan and how I actually did.
Computer science track:
  • Goal: Do half (5.5 weeks) of Andrew’s Ng Machine Learning course on Coursera (https://www.coursera.org/learn/machine-learning), watch videos and do quizes and homework programming assignments (automatically graded).
    • What actually got done: I did 5 weeks of Andrew Ng Machine Learning course. I watched all videos and did all quizzes and exercises which I think was worthwhile to do. Going through the hassle of implementing backpropagration algorithm was useful. I consider this done as 5 weeks is close enough to 5.5 and stopping at middle of the week is quite awkward.
  • Goal: Go through all of Automatic Speech Recognition class at NYC (http://www.cs.nyu.edu/~eugenew/asr13/)
    • What actually got done: I went through only half of NYU ASR class videos.
  • What else got done:
    • I read 1-5 chapters of book Programming Collective Intelligence. This book is an older one but very useful in putting ML in context of other ways of solving problems.
    • I also read through Neural Networks tutorial: http://karpathy.github.io/neuralnets/
  • Score: 0.75 (out of 1)

Linguistics track:

  • Goal: Brush up on my German using similar method as I did for Spanish – study for 30 days German itself + learn more about the linguistics aspect of it.
    • Done!  I did study German for 30+ days and I did go to Germany and was able to use it to get around, buy train tickets, order food, have conversation at a store about what cell phone prepaid plan I would like 🙂 Overall I declare it a success. 🙂 I did study German in high school for 3 years, but hadn’t really used it since. This was more like a review. Yes, foreign language skills get dusty very quickly, but now I see that its possible to revive them in a month long time-frame.
  • I also learned about linguistic concepts such as homographs, homonyms and words pronounced letter by letter.
  • Score: 1.0 (out of 1)

Reading:

  • Goal: Come up with a system for myself to keep track of papers I read and start reading papers.
    • End result:  I tried out Mendeley – that didn’t work as it is a separate app and not at all integrated in my workflow and the tools I use. Also, it’s search wasn’t as good as Google Scholar search. So I ended up having Google Scholar search + labels to keep track of papers to read; I also have spreadsheet to keep track of papers I have read and for each paper: notes, things to learn, cited papers to potentially read later.
    • I read 11 papers on topics related to Speech Recognition. Most of them were hard and I didn’t understood most of it, but some were reasonable.
  • Score: 1.0

Writing:

  • Goal: send updates on my progress via email every other week.
    • I wrote 5 updates in 12 weeks, so that comes out to almost every other week.
  • Oh, I also finally migrated my blog from blogger to wordpress 🙂
  • Score: 0.83
Overall score for the Q1: 0.9 which I am quite happy about 🙂
Now onto the second quarter! 🙂

Whiiii, the first quarter is done!! (Personal PhD update #1.5)

~ diana ~ Leave a comment
Whiiii, the first quarter is done!!
I didn’t do everything as I had planned, but nonetheless, I am very happy with my progress!
I managed 5 weeks of Machine Learning course on Coursera (vs 5.5 weeks planned). So I will call this as a success!
For the Automatic Speech Recognition videos I watched first 6 lectures out of 12 vs I had planned to do all. Got half done, but still more than nothing! Next time will know better that 1.5 courses per quarter is more than I can do.
I will write a more detailed review of my quarter soon, but wanted to get this out first 🙂
Next up: I will take two weeks to plan out syllabus for the next quarter. (Such a good feeling to have the freedom to make it up myself to make it the most relevant to what’s going on at work right now!)
Machine Learning in the news:
Learning to learn, or the advent of augmented data scientists – no machine learning will not replace data scientists 🙂
New book:

A new popular science book on machine learning has just come out (on Sept 22nd), looks interesting and I plan to read it: The Master Algorithm: How the Quest for the Ultimate Learning Machine Will Remake Our World (amazon)

Chugging along (Personal PhD update #1.4)

~ diana ~ Leave a comment
Here is an update on my progress via an infographic.

​
Machine learning resources:
I have discovered more Machine Learning resources:
Other progress:
I am also reading book called Collective Intelligence. And although it is an older book it has a very intuitive intro into machine learning and it explains where it fits among all other methods of processing data and shows why is machine learning useful (complete with practical examples).
In contrast the Coursera course on Machine Learning starts off with much more math. In the end, both resources are very useful to learn from, but if I knew this from the beginning I would have read the Collective Intelligence book before starting the Coursera course.
Speech recognition though is still hard. I don’t understand most of the lectures. But that’s ok, even understanding just 10% is more helpful to doing my job better and understanding better what other people in my bigger team are doing.
I have learned a little more about linguistics. Ie what are heterographs and heteronyms.

 

 
Machine learning in the news
Fascinating, machine learning is used in all sorts of places: A machine learning algorithm picks out the fashion models most likely to succeed.

Personal PhD update #1.3

~ diana ~ Leave a comment

A few weeks ago I was in Germany. I arrived at the Frankfurt train station. I was in no rush so when I saw a big bookstore I went in. I love hanging out in bookstores. So it being a bookstore where most of the books would be in German wasn’t a deterrent. One book piqued my interest. It was titled “The devil lies in the detail”. I flipped through it, saw this chart:

So I bought the book. (Seems I can’t come back from an international trip without a book or two.) It’s a German book in German about English language. My German is not that good yet so I have to read it with dictionary but the first chapter is already hilarious. The author recounts a scene he overheard. A German lady in England asked at an ice-cream stand: “Can I please have two ice balls”.  To which the seller replied: “My ice balls are not for sale, Ma’am!” If she had said “two scoops of ice-cream” it wouldn’t be such noteworthy conversation.

Anyways, this is an example of how I follow my curiosity. One thing will lead to another, and I am reading a book in German.

It is the same with this Personal PhD project. I don’t know where it is going to get me in 5 years, but I am convinced its gonna be some place awesome.

Already it has taken me places I didn’t plan for in the beginning. For example I  explored how to visualize my progress on the curriculum (thanks Dominique for the suggestion!)

See here:

https://infogr.am/personal_phd_progress_q1

I picked infogr.am as its a startup I have heard a lot about (it is a Latvian startup). The result is best I could do with free version in half an hour. 🙂

Oh, and by the way. That brushing up on my German was very useful. While I was on vacation in Germany I was able to buy train tickets, order food, check in the hotel. Even though the total studying time didn’t add up to much, hearing it a little every day helped a lot. If anything I was able to keep the conversation going in German however broken as opposed the other person switching to English right away. So I am done with my 30 days of learning German but I will extend it to read the new book.
Another installment of machine learning in the news. AirBnB is doing machine learning to predict at what price a place is going to rent out. http://www.forbes.com/sites/ellenhuet/2015/06/05/how-airbnb-uses-big-data-and-machine-learning-to-guide-hosts-to-the-perfect-price/

I admit defeat.. but only this week (Personal PhD update #1.2)

~ diana ~ Leave a comment
I admit defeat.. but only this week.
It has been a month since I have started my Personal PhD project. I set a schedule for myself of how much course material I wanted to cover per week.
I have been on vacation for a week now. Before I went I thought I will have so much free time I will get so much done!
Alas, that didn’t happen. Visiting home in Latvia is a lot of work, a lot of people to catch up, plus jetlag doesn’t help either and I am not as productive during the few moments when I do have free time.
So for this update I didn’t get done as much as I hoped. Now, the temptation of the perfectionist was to try to finish everything and only then send out the updateafterwards.
But one of the goals for Personal PhD was writing and sending regular updates, so here I am admitting that that this week I am behind schedule. However I will have a free weekend in couple of weeks, so I will be able to catch up then.

Continue reading →

Personal PhD update #1.1

~ diana ~ Leave a comment
It has been two weeks since I have started my Personal PhD. It is still very exciting! So far I am progressing along as planned for my first quarter curriculum.
Q1 curriculum:
Computer Science track:
1. I have finished Week 1 of Andrew Ng Coursera course right on my half-speed schedule of one week of course material in two weeks.
2. Watched 2 lectures of Automatic Speech Recognition class. It is very dense. Turns out Speech Recognition is based on Finite Automata. When I studied about Finite Automata in college I never thought that theory would be ever so useful. But here I am now using them at work. 🙂 Looks like there is a Coursera course specifically on Automata here.

Continue reading →