For the last months, I have been preparing a course on natural language processing using neural networks. Time and time again I have heard that the best way to learn something is by teaching about it. In my mind, this has become a sort of cliché, one that rings true and echoes throughout my life. Many moons ago, as a teenager, I obsessively read everything I could about Richard Feynman. His brand of teaching was putting things as simply as possible. It was focusing on the kind of playful understanding that lets one do something with the knowledge and then try to build upon it while having fun. Feynman’s playful approach to problem solving feels like humble and honest teaching, first to himself, then far and wide into the world.
This goes well with anything related to programming or computers. Anything that can be tinkered with—anything where you can practice and experiment—is best taught by exercise, example and play. To learn to do something, you simply have to tickle the problem in all sorts of ways until you figure out the way to solve it. To understand why or if your solution is good, you have to understand how you are got there, starting from some most basic idea and then seeing where the little steps lead. Teaching becomes about showing the way and doing so in as simple terms as you can. And the way is only obvious if you really know where you are headed—trying to guess and wing it will just get you lost. You have to first teach yourself, play around until it becomes natural. This is what has guided the course: keeping it simple and understanding every principle.
Now, natural language has always fascinated me. As a kid, I wanted to be a writer. In a sense, perhaps I still want to be one. Perhaps I already managed to be one. It so just happens that my target audience is equal parts computers and people.
The advances in neural networks in the last decade have improved the way we deal with many complicated problems. We can now unlock phones with our pretty faces and have our computer call people up to set up appointments for us. And among these once impossible problems we have natural language as a whole. We once approached natural language with hacky hand-engineered features and complicated rules. Sure, we are still very far away from being able to capture all the nuance hidden between the lines, drawn silently in the interplay between words. One could argue that we are not being that smart about our problems, that we are simply using brute-force to tackle them. High performance computing and general purpose GPUs have really turned the world around. Fair. Nevertheless, the progress in problems such as question answering or machine translation has been nothing short of a miracle. And this miracle has been largely powered by deep neural networks. This is the pitch: program those miraculous solutions yourself using deep learning and natural language processing.
Why should anyone be interested in my course in particular? There are many courses online, with many well known instructors on their forefront. And after all, as nice as it might be that I am trying to make it as accessible as possible, the fact is that most people are not good at simplifying things. I might be oversimplifying things, in turn setting you up for failure when you try to apply the same concepts elsewhere. I might not be simplifying it at all, instead making it all confusing! And even if I am doing neither, perhaps my teaching style is just not for you…
I agree. My answer is that the contents of the course will be free and open source. By that, I mean that you will be able to freely access the lessons, as Jupyter notebooks, on Github. Inasmuch as I can, I want people to learn, and to help me learn how to better teach and understand these concepts. I will include every learning material I have produced, so that interested people can help me improve the readability, quality and content of the course.
However, it is be nice to be rewarded for effort—and an economic reward feels like proof that I did something worthy of being taught! Indeed, if people are willing to pay, there must be at least a modicum of value in what I have done and the time I’ve spent. As such, I plan on selling the course online with videos to go with every lesson. In doing so, visually or verbally minded people will also be able to get value out of my work. Additionaly, it will force me to pick up basic video editting and audio engineering skills, both of which seem quite enjoyable to me!
Although I have prepared my lessons on my spare time as a personal project, I am enormously indebted to my colleagues. My focus was to try to build something on my own, training mentally for side-hustling. However, during the day, I still work as a data engineer and general tinkerer for the data analytics team in my company. My colleagues have helped me revise the contents of the course, simplify things further and better explain the hard concepts. One of the many joys of working with experienced machine learning experts, information retrieval PhDs and top-notch engineers is that, slowly but surely, you end up picking up some of the smartness through osmosis! Lecturing in our office has greatly improved my understanding, corrected flaws in my lessons and made the course better altogether. I want to believe it has also given my team mates a new set of power-charged tools to work with, to improve any project, whether present or future. Any display of gratefulness from my part just won’t ever be enough.
I similarly owe much to my friends and to the Internet people that convinced me to go for it. To the former, for their encouragement and comments, which help me better focus on specific, purposeful things. I am prone to get lost on my thoughts and ambitions otherwise. And because they know better than me on many-so-often things, our discussions really showed me how to explain things better. The insightful Internet tribe, with @balajis, @naval, @desantis and @sknthla, made me take this leap in producing out the knowledge I have to share. I am no stranger to doing things on the internet, but this is the first time trying to earn out of it. Time will tell how that goes.
So far, four lessons are ready. By ready, I mean that I have written up four notebooks covering the basics of the underlying linear algebra, an introduction to neural network, the application of neural networks for NLP and the structured architectures of Convolutional and Recurrent networks. I have also gone through a flavor of peer-review by lecturing at my office and having my friends look at the content. The next step is to correct the flaws and mistakes and prepare the videos, programming projects and, perhaps, additional exercises. Then, the whole course will be ready to go. I am unsure if I will add new lessons. Covering more advanced models that include attention, use transformers, or relate different inputs would be great. However, they could muddle up the whole thing, and perhaps they are better suited for a sequel.
Whatever I decide, this post is my public commitment to make sure I finish, and to keep having fun in the process. Expect another post when all is said and done.