<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Vishal Maini on Medium]]></title>
        <description><![CDATA[Stories by Vishal Maini on Medium]]></description>
        <link>https://medium.com/@v_maini?source=rss-19a4c39d50a8------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*7qOP1DnjjYcPKMxUG1592g.jpeg</url>
            <title>Stories by Vishal Maini on Medium</title>
            <link>https://medium.com/@v_maini?source=rss-19a4c39d50a8------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 07:41:03 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@v_maini/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[Taking the plunge: my path to working on artificial intelligence]]></title>
            <link>https://medium.com/@v_maini/taking-the-plunge-my-path-to-working-on-artificial-intelligence-fcf4dc4314f6?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/fcf4dc4314f6</guid>
            <category><![CDATA[existential-risk]]></category>
            <category><![CDATA[autodidacticism]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deepmind]]></category>
            <category><![CDATA[strategy]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Mon, 03 Jul 2023 22:31:16 GMT</pubDate>
            <atom:updated>2023-10-12T23:37:21.391Z</atom:updated>
            <content:encoded><![CDATA[<blockquote>If one does not know to which port one is sailing, no wind is favorable. — Lucius Annaeus Seneca</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*7aoGWluCc4wYTEaIQ6phOw.jpeg" /></figure><p><em>Drafted in 2019, unpublished til 2023. Now building </em><a href="https://twitter.com/MythosVentures"><em>Mythos Ventures</em></a>, <em>investing in civilization-scale thinkers building pro-social AI startups.</em></p><p>I first started thinking seriously about artificial intelligence in late 2016, during a process of examining, at a very zoomed-out level, the trajectory of human civilization: how far we’ve come, what the future might look like, and what I could do to positively shape that trajectory.</p><p>After four years leading the growth team at <a href="https://www.upstart.com/about#who-we-are">Upstart</a>, a breakout-stage company leveraging machine learning to expand access to affordable credit, I felt I’d learned enough and built enough career capital to start thinking long-term. My approach was to optimize globally, rather than locally, and to assume that there were no hard constraints on what I would be qualified to do.</p><p>I read about the <a href="https://waitbutwhy.com/2014/05/fermi-paradox.html">Fermi Paradox</a>, and our mission to become an <a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html">interplanetary species</a>, starting with Mars colonization. At first, it sounded too futuristic to feel real in my lifetime, but after delving more deeply into the details of this and other ambitious scientific projects, I started to viscerally understand how unintuitively different the not-so-far future might look from the not-too-distant past.</p><p>I thought about large-scale <a href="https://theconversation.com/the-five-biggest-threats-to-human-existence-27053">risks</a> to <a href="https://en.wikipedia.org/wiki/Global_catastrophic_risk#Likelihood">future generations</a>: run-away climate change, engineered or natural pandemics, nuclear exchange, or naturally-occurring risks like asteroid impact, to name a few. The cumulative level of risk felt unacceptably high. I wouldn’t accept these odds if gambling all my savings at a casino, let alone all future human lives. Luckily, many risks seemed to be anthropogenic, and thus potentially more tractable given the right combination of social, political, and technical solutions — and indeed, for each category of risk, there seemed to be some promising efforts already <a href="https://waitbutwhy.com/2015/08/how-and-why-spacex-will-colonize-mars.html/2#part2">under</a> <a href="https://en.wikipedia.org/wiki/Future_of_Humanity_Institute">way</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/459/1*_fwDlBT7aj-yUcBy7b_vbA.png" /><figcaption>A 2016 report by the <a href="https://en.wikipedia.org/wiki/Global_Challenges_Foundation">Global Challenges Foundation</a> estimates that an average American is more than five times more likely to die during a human-extinction event than in a car crash (see <a href="https://api.globalchallenges.org/static/reports/Global-Catastrophic-Risk-Annual-Report-2016.pdf">report</a>). Read more on <a href="https://en.wikipedia.org/wiki/Global_catastrophic_risk#Likelihood">Wikipedia</a>. All of these estimates should be taken with many grains of salt.</figcaption></figure><p>And what about this generation — including myself, my family, my friends, and everyone I’ve ever met? I think life is beautiful. There’s something nostalgic and special about getting to experience it, and something tragic about the way it inevitably, and often abruptly or painfully, ends. I read Nick Bostrom’s <a href="https://nickbostrom.com/fable/dragon.html">Fable of the Dragon Tyrant</a> on the ethics of mortality, and wondered whether there might come a day soon when life expectancy would reach ‘<a href="https://en.wikipedia.org/wiki/Longevity_escape_velocity">longevity escape velocity</a>’, increasing by more than one year per year, leading to the first generation of people living thousands, rather than dozens, of years.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FcZYNADOHhVY%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DcZYNADOHhVY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FcZYNADOHhVY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/41461eb6f502a76fba3fcf01ec95f0ac/href">https://medium.com/media/41461eb6f502a76fba3fcf01ec95f0ac/href</a></iframe><p>I studied human tendencies like <a href="http://mindingourway.com/on-caring/">scope insensitivity</a> (it’s hard to “feel” the magnitude of large-scale problems or viscerally understand the difference between orders of magnitude) and <a href="https://en.wikipedia.org/wiki/Diffusion_of_responsibility">diffusion of responsibility</a> (a generalization of the bystander effect). I thought through the unfortunate game theory of <a href="https://en.wikipedia.org/wiki/Collective_action_problem">collective action problems</a>, and the coordination failures that inevitably result. These factors, in combination, might help explain why some of these Really Big Problems seem not always to be met with proportionally large-scale efforts to find Really Big Solutions.</p><p>These concepts, I concluded, might explain the unintuitive neglectedness of some of the world’s <a href="https://80000hours.org/articles/cause-selection/">most pressing problems</a> — but also, perhaps more excitingly, implied the existence of low-hanging fruits and tractable opportunities for individuals to make a potentially massive counterfactual impact on these problems. As an <a href="http://vishalmaini.com/entrepreneurs-are-betting-on-a-losing-strategy">entrepreneur</a> at heart and a <a href="https://medium.com/startup-frontier/marketing-shouldn-t-be-a-dirty-word-in-silicon-valley-81aba98c5066">growth marketer</a> by training, I naturally gravitated towards looking for the ‘<a href="https://genius.com/Peter-thiel-zero-to-one-chapter-8-secrets-annotated">secrets</a>’ that people weren’t seeing yet. The unseen opportunities for outsize impact. The small number of moves that would make the biggest difference.</p><p>I won’t detail the entire intellectual journey here — I’ve instead compiled a <a href="https://medium.com/@v_maini/80-20-ai-reading-list-c4753afd97a">reading list</a>, to sketch out the path — but I can share my personal conclusion. Ultimately, after hundreds of hours of investigation, I converged on the view that if I wanted to work on any of these problems, <strong>artificial intelligence </strong>seemed to be the most promising Really Big Solution — plausibly, for <em>all</em> of these Really Big Problems.</p><p>The simplest expression of this view is: <strong>AI is a meta-solution. </strong>Let’s say <strong>G</strong> is the thing we’re working towards — the ambitious long-term goal. Solving climate change; colonizing space; helping people live healthier and happier lives; creating lasting global stability and peace, addressing root causes of inequality and injustice, improving our ability to communicate and learn and understand each other; generally <a href="http://www.europarl.europa.eu/cmsdata/148934/2.2%20Brundage.pdf">scaling up humanity</a>.</p><p><strong>G</strong> could also be something deeper. Understanding the universe, and consciousness itself. Answering some of the most fundamental questions of philosophy. <em>How do I make sense of the universe, and my place in it? What does it mean to live well? What should I do?</em></p><p>If we care about<strong> G</strong>, and believe <em>human intelligence</em> can make progress towards <strong>G</strong>, and if we also believe that we can, in the coming decades, create<em> ‘general’ machine intelligence that rivals or exceeds all human cognitive abilities</em>, then for almost any sufficiently long-term <strong>G</strong>, one could arguably more effectively achieve <strong>G</strong> by developing and deploying general artificial intelligence than by working directly on <strong>G</strong>. The relevant question, then, seemed to be whether developing ‘general’ AI was actually realistic on a reasonable timescale, and whether I could personally do anything to positively shape how that played out.</p><p>I didn’t have a background in AI, and evaluating the question of whether we might actually create AI that could rival or exceed most human cognitive abilities was hard to evaluate, without stronger technical foundations. I’m not one to blindly defer to expert opinion when it comes to answering questions of fundamental importance to my life. But I had heard of some stories of other people making radical transitions into fields like AI and rocket science (like <a href="https://blog.gregbrockman.com/my-path-to-openai">Greg Brockman</a>, co-founder and CTO of OpenAI, and <a href="https://www.quora.com/How-did-Elon-Musk-learn-enough-about-rockets-to-create-and-run-SpaceX">Elon Musk</a>, founder and CEO of SpaceX), and it seemed possible to do.</p><p>So I decided to take the plunge. I quit my job. I had no guarantees of what was next: I was leaving behind a well-paying leadership role at a breakout-stage company for a new domain with no warm prospects, few connections, no expertise or well-defined target roles, self-funded, without any kind of brand or stamp of legitimacy to put on my CV for the self-education phase that followed. It was a terrifying act, reminiscent of burning many bridges behind me. One can always somehow build a boat to get back, I reasoned.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*qfSTA6fT69DJS00wVz0f-g.jpeg" /><figcaption>Saying goodbye over a catered Chipotle lunch. Jan 25, 2017.</figcaption></figure><p>I put my social life on pause, set up an office in my apartment, and began studying AI full-time, hoping to achieve something like a self-directed Master’s degree in under six months.</p><p>I started by enrolling in a machine learning <a href="https://www.galvanize.com/data-science">bootcamp</a>. It was an excellent accelerator for building coding skills and machine learning intuitions, but the curriculum was optimized for job placement in data science roles, and it didn’t cover deep learning or reinforcement learning, which are currently favored approaches for developing general AI systems. After six weeks, I decided to drop out and pursue pure self-study. I spent another fifteen weeks taking online <a href="https://medium.com/machine-learning-for-humans/how-to-learn-machine-learning-24d53bb64aa1">classes</a>, studying probability/statistics/linear algebra, writing <a href="https://github.com/vmaini">code</a>, reading <a href="https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap">papers</a>, and seeking advice from mentors and friends as often as I could.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*i1Mu-_bQ7xMsKhizbS7oOA.jpeg" /><figcaption>Apartment window as whiteboard. Trying to wrap my head around reinforcement learning. Sept 14, 2017.</figcaption></figure><p><a href="https://soundcloud.com/alesia-arkusha">House music</a>, caffeine, daily meditation, cold showers, coworking sessions, a rigorous exercise regime, a few incredibly supportive friends — along with sheer intellectual fascination for the subjects at hand — kept me sane, focused, and intensely motivated.</p><p>Each day I’d aim for entering a state of <a href="https://en.wikipedia.org/wiki/Flow_(psychology)">flow</a> for at least two of {“morning sprint”, “afternoon sprint”, “evening sprint”}, five or six days a week. The regimen, while grueling, was deeply satisfying. Personally, I find AI to be endlessly fascinating. It comprises a luscious mix of academic disciplines — computer science, philosophy, neuroscience, logic, mathematics, economics, and decision theory — each containing endless lessons that can <a href="https://hackernoon.com/algorithms-to-live-by-book-review-77f53d63fa78">improve our lives</a> and reshape our ways of thinking.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BC28UfHgrYsMhrJXw4Byug.jpeg" /><figcaption>Standard fare for noontime brunch. Brother (and flatmate for the year) not pictured. Apr 2, 2017.</figcaption></figure><p>After completing the autodidactic curriculum a few months <em>ahead</em> of schedule (I had written on my window, in dry-erase marker: “<em>F*** you, </em><a href="https://en.wikipedia.org/wiki/Hofstadter%27s_law"><em>Hofstadter’s Law</em></a><em>”</em>, and made hefty commitment bets on monthly deadlines with my brother), I spent another month traveling while writing <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a>, and launched it using the growth tactics I’d learned in my previous life. Today, it has over 1 million views, is required reading in some university courses, and shows up on the front page of search results for many key terms in artificial intelligence (try “machine learning” or “reinforcement learning”).</p><blockquote>Hofstadter’s Law: It always takes longer than you expect, even when you take into account Hofstadter’s Law.</blockquote><p>In the end, through a combination of persistence, good fortune, and fundamental mission alignment, I ultimately joined the Public Engagement &amp; Communications team at <a href="https://deepmind.com/">DeepMind</a>, one of the two leading labs undertaking the ambitious goal of developing <a href="https://medium.com/@deepmindsafetyresearch/building-safe-artificial-intelligence-52f5f75058f1">safe</a> artificial general intelligence. My start date was February 5, 2018 — one year and ten days after ‘taking the plunge’.</p><p>Throughout that process, I made sure to continuously re-evaluate whether working on beneficial AI still seemed likely to be the most promising strategy for long-term civilizational flourishing.</p><p>Now, I’m excited to be working on building and deploying safe artificial general intelligence at DeepMind, with a personal focus on strategy, communications, and coordination towards beneficial outcomes — in hopes that AI will be the meta-solution to so much else.</p><p>If you’d like to explore these ideas further, you might enjoy this <a href="https://medium.com/@v_maini/ai-reading-list-c4753afd97a">AI Reading List</a> that I put together along the way. And if you’re going through a similar exploration and would like to chat, feel free to <a href="https://twitter.com/v_maini">reach out</a>. Taking the plunge can be daunting, and if I can, I’ll be happy to help.</p><p><a href="https://medium.com/@v_maini/ai-reading-list-c4753afd97a">AI Reading List</a></p><p><em>Views expressed here are my own, and do not represent the views or opinions of any other entity.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/867/1*Zlkowd9gt3D48xVJPoqK5A.jpeg" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fcf4dc4314f6" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI Reading List]]></title>
            <link>https://medium.com/machine-learning-for-humans/ai-reading-list-c4753afd97a?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/c4753afd97a</guid>
            <category><![CDATA[ai-strategy]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[ai-governance]]></category>
            <category><![CDATA[ai-safety]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Wed, 19 Jun 2019 16:32:07 GMT</pubDate>
            <atom:updated>2019-06-19T18:56:49.619Z</atom:updated>
            <content:encoded><![CDATA[<p>For newcomers to the field of artificial intelligence, prioritizing among endless AI resources can be an overwhelming challenge. This list attempts to do exactly that: it’s a carefully curated compilation of resources for getting up to speed quickly on key topics in artificial intelligence research and its long-term implications.</p><p>The list is divided into “<a href="https://en.wikipedia.org/wiki/Pareto_principle">80/20</a>” sections with a few high-priority readings, for maximum value with minimal time investment, and “deep dive” sections for further exploration.</p><p><em>Readers need not be technical, nor have a prior background in artificial intelligence. The list may be of special interest to those considering entering the field of AI research or adjacent fields, whether in technical or non-technical roles.</em></p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*w8lAJMbsEa9A3NTiL6BPfw.png" /><figcaption><em>This list covers high-level context (“What is intelligence, and what would it mean to re-create it in machines?”) technical foundations (“How does narrow AI work today, and what are some of the favored technical approaches towards general AI?”), safety considerations (“What will it take for beyond-human-level AI to be safe and act in accordance with human preferences?”), and strategic questions (“How can we coordinate towards beneficial outcomes from advanced AI?”).</em></figcaption></figure><pre>To get the most value out of these resources, consider actively reading. Write short summaries of key concepts and takeaways that you can refer to later (see: <a href="https://mattyford.com/blog/2014/1/23/the-feynman-technique-model">Feynman technique</a>). Spend some time critically analyzing the ideas. What makes intuitive sense to you, and what doesn’t? What are your criticisms of the ideas presented? How do concepts fit together?</pre><h3>1. Intelligence</h3><p><em>What is intelligence, and how might we re-create it in machines? Why now? Three ingredients for AI progress — compute, data, and algorithms.</em></p><h4><strong>80/20</strong></h4><ul><li><a href="https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html">The AI Revolution: The Road to Superintelligence</a> — Tim Urban (Wait But Why), 2015</li><li><a href="https://www.amazon.co.uk/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111">Superintelligence: Paths, Dangers, Strategies</a>, Ch. 1–3 — Nick Bostrom, 2014</li><li><a href="https://www.youtube.com/watch?v=3N9phq_yZP0">The Power of Self-Learning Systems</a> (video) — Demis Hassabis, 2019</li><li><a href="https://www.ted.com/talks/max_tegmark_how_to_get_empowered_not_overpowered_by_ai">How to get empowered, not overpowered, by AI</a> (video) — Max Tegmark, 2018</li><li><a href="https://www.youtube.com/watch?v=YHCSNsLKHfM">Can we rule out near-term AGI?</a> (video) — Greg Brockman, 2018</li><li><a href="https://blog.openai.com/ai-and-compute/">AI and Compute</a> — OpenAI, 2019</li></ul><h4><strong>Deep dive</strong></h4><ul><li><a href="https://www.csee.umbc.edu/courses/471/papers/turing.pdf">Computing machinery and intelligence</a> — Alan Turing, 1950</li><li><a href="http://www.vetta.org/documents/Machine_Super_Intelligence.pdf">Machine Super Intelligence</a> — Shane Legg, 2008</li><li><a href="http://www.fhi.ox.ac.uk/reframing">Reframing Superintelligence</a> — Eric Drexler, 2019</li><li><a href="https://www.netflix.com/watch/80190844?source=35">AlphaGo (documentary</a>) — DeepMind, 2017</li><li><a href="https://www.amazon.co.uk/Measure-All-Minds-Evaluating-Intelligence/dp/1107153018">The Measure of All Minds: Evaluating Natural and Artificial Intelligence</a> — José Hernández-Orallo, 2017</li></ul><h3>2. Machine Learning</h3><p><em>Enabling machines to learn for themselves. Learning to make predictions and identify patterns given different kinds of data (supervised, unsupervised, and reinforcement learning). Demystifying the objective function.</em></p><h4><strong>80/20</strong></h4><ul><li><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12">Machine Learning for Humans</a>, Parts 1–3 — Vishal Maini &amp; Samer Sabri, 2017</li></ul><h4><strong>Deep dive</strong></h4><ul><li><a href="https://www-bcf.usc.edu/~gareth/ISL/ISLR%20First%20Printing.pdf">Introduction to Statistical Learning, Chapters 1–3</a> — Gareth James et al., 2013</li><li><a href="https://www.coursera.org/learn/ai-for-everyone">AI for Everyone</a> (nine-hour Coursera course) — Andrew Ng, 2019</li><li><a href="https://medium.com/machine-learning-for-humans/how-to-learn-machine-learning-24d53bb64aa1">The Best Machine Learning Resources </a>— Vishal Maini &amp; Samer Sabri, 2017</li></ul><h3>3. Deep Learning</h3><p><em>Learning to predict an output given an input; drawing inspiration from the brain. Looking inside the black box of deep neural networks. What is a cat, and why? How do we represent the world around us numerically (as sensory “input” or data) and use math to make sense of it? Common architectures.</em></p><h4><strong>80/20</strong></h4><ul><li><a href="http://neuralnetworksanddeeplearning.com/chap1.html">Neural Networks and Deep Learning, Chapter 1</a> — Michael Nielsen, 2015</li><li><a href="https://medium.com/machine-learning-for-humans/neural-networks-deep-learning-cdad8aeae49b">Machine Learning for Humans, Part 4: Neural Networks &amp; Deep Learning</a> — Vishal Maini &amp; Samer Sabri, 2017</li><li><a href="https://www.youtube.com/watch?v=bxe2T-V8XRs">Neural Networks Demystified</a> — YouTube series</li><li><a href="https://www.youtube.com/watch?v=aircAruvnKk&amp;t=15s">But what *is* a Neural Network? | Deep learning, chapter 1</a> (video) –3Blue1Brown, 2017</li><li>Architectures — <a href="http://colah.github.io/posts/2014-07-Conv-Nets-Modular/">CNN</a>, <a href="http://karpathy.github.io/2015/05/21/rnn-effectiveness/">RNN</a>, <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">LSTM</a>, <a href="https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html">Transformer</a></li></ul><h4><strong>Deep dive</strong></h4><ul><li><a href="http://neuralnetworksanddeeplearning.com/chap1.html">Neural networks and deep learning</a> — Michael Nielsen, 2015</li><li><a href="http://www.deeplearningbook.org/">Deep Learning Book</a> — Ian Goodfellow, Yoshua Bengio, and Aaron Courville, 2016</li><li><a href="http://playground.tensorflow.org/#activation=tanh&amp;batchSize=10&amp;dataset=circle&amp;regDataset=reg-plane&amp;learningRate=0.03&amp;regularizationRate=0&amp;noise=0&amp;networkShape=4,2&amp;seed=0.70277&amp;showTestData=false&amp;discretize=false&amp;percTrainData=50&amp;x=true&amp;y=true&amp;xTimesY=false&amp;xSquared=false&amp;ySquared=false&amp;cosX=false&amp;sinX=false&amp;cosY=false&amp;sinY=false&amp;collectStats=false&amp;problem=classification&amp;initZero=false&amp;hideText=false">TensorFlow Neural Network Playground</a></li><li><a href="https://distill.pub/2018/building-blocks/">The building blocks of interpretability</a> — Chris Olah, 2018</li><li><a href="https://docs.google.com/document/d/1TFkxqU7fPCcM01W7dUv0dT4wOxl84xB1Y_-tcbCgz4c/edit?usp=sharing">How do neural and symbolic technologies mesh?</a> — Eric Drexler, 2018</li><li><a href="https://github.com/floodsung/Deep-Learning-Papers-Reading-Roadmap">Deep Learning Papers Reading Roadmap</a> — Flood Sung, 2018</li></ul><h3>4. Deep Reinforcement Learning</h3><p><em>Artificial agents learning from reward. Value functions. Exploration and exploitation. Reaching superhuman performance in complex games. Key breakthroughs.</em></p><h4><strong>80/20</strong></h4><ul><li><a href="http://incompleteideas.net/book/bookdraft2017nov5.pdf">Reinforcement Learning: An Introduction — Chapter 1</a> — Rich Sutton &amp; Andrew Barto, 2017</li><li><a href="https://medium.com/machine-learning-for-humans/reinforcement-learning-6eacf258b265">Machine Learning for Humans, Part 5: Reinforcement Learning</a> — Vishal Maini &amp; Samer Sabri, 2017</li><li><a href="https://meowni.ca/posts/rl-with-otters/">An intro to Reinforcement Learning (with otters)</a> - Monica Dinculescu, 2018</li><li><a href="https://youtu.be/A3ekFcZ3KNw">NIPS 2017 Keynote: Deep RL Symposium — AlphaZero</a> (video) — Dave Silver, 2017</li><li><a href="http://tim.hibal.org/blog/alpha-zero-how-and-why-it-works/">AlphaGo Zero — How and Why it Works</a> — Tim Wheeler, 2017</li></ul><h4><strong>Deep dive</strong></h4><ul><li><a href="http://incompleteideas.net/book/the-book-2nd.html">Reinforcement Learning: An Introduction</a></li><li><a href="https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf">Deep Q-network (DQN) </a>— Mnih et al., 2015</li><li><a href="http://karpathy.github.io/2016/05/31/rl/">Deep Reinforcement Learning: Pong from Pixels</a> — Andrej Karpathy, 2016</li><li><a href="https://storage.googleapis.com/deepmind-media/alphago/AlphaGoNaturePaper.pdf">AlphaGo</a>,<a href="https://deepmind.com/blog/alphago-zero-learning-scratch/"> AlphaGo Zero</a>,<a href="https://arxiv.org/pdf/1712.01815.pdf"> AlphaZero</a>,<a href="https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/"> AlphaStar</a> — DeepMind</li><li><a href="https://blog.openai.com/spinning-up-in-deep-rl/">Spinning Up in Deep RL</a> — OpenAI, 2018</li><li><a href="https://www.youtube.com/watch?v=EeMCEQa85tw">The Long-Term of AI and Temporal Difference Learning</a> — Richard Sutton, 2017</li></ul><h3>5. Safety &amp; Alignment</h3><p><em>How do we ensure that highly capable and general AI systems reliably understand what we want and help us get it? Identifying and pre-empting potential failure modes. An introduction to specification, robustness, and assurance.</em></p><h4><strong>80/20</strong></h4><ul><li><a href="https://medium.com/@deepmindsafetyresearch/building-safe-artificial-intelligence-52f5f75058f1">Building safe artificial intelligence: specification, robustness, and assurance</a> — DeepMind safety team, 2018</li><li><a href="https://www.amazon.co.uk/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111">Superintelligence: Paths, Dangers, Strategies</a> , Ch. 7–10 &amp; 12–13. — Nick Bostrom, 2014</li><li><a href="https://futureoflife.org/background/benefits-risks-of-artificial-intelligence/">Benefits &amp; Risks of Artificial Intelligence</a> — Future of Life Institute, 2016</li><li><a href="https://www.openphilanthropy.org/blog/potential-risks-advanced-artificial-intelligence-philanthropic-opportunity">Potential Risks from Advanced Artificial Intelligence: The Philanthropic Opportunity</a> — Open Philanthropy, 2015</li><li><a href="https://blog.openai.com/faulty-reward-functions/">Faulty Reward Functions in the Wild</a> — OpenAI, 2016</li></ul><h4><strong>Deep dive</strong></h4><ul><li><a href="https://sideways-view.com/2018/02/24/takeoff-speeds/">Takeoff speeds</a> — Paul Christiano, 2018</li><li><a href="https://arxiv.org/abs/1606.06565">Concrete problems in AI safety</a> — OpenAI, 2016</li><li><a href="https://medium.com/@deepmindsafetyresearch/scalable-agent-alignment-via-reward-modeling-bf4ab06dfd84">Scalable agent alignment via reward modeling</a> — Leike et al., 2018</li><li><a href="https://www.fhi.ox.ac.uk/reframing/">Reframing Superintelligence: Comprehensive AI Services as General Intelligence</a> — Eric Drexler, 2019</li><li><a href="https://twitter.com/iamtrask/status/1101895241857187841">Fairness, privacy, security &amp; verification resources</a> — Andrew Trask, 2019</li><li><a href="https://deepmind.com/blog/robust-and-verified-ai/">Towards Robust and Verified AI</a> — DeepMind, 2019</li><li><a href="https://futureoflife.org/2019/04/11/an-overview-of-technical-ai-alignment-with-rohin-shah-part-1/">An Overview of Technical AI Alignment</a> (podcast) — Lucas Perry &amp; Rohin Shah, 2019</li><li><a href="https://vkrakovna.wordpress.com/ai-safety-resources/">AI safety resources</a> — Victoria Krakovna, 2018</li></ul><h3>6. Strategy &amp; Governance</h3><p><em>Near- and long-term strategic challenges and opportunities presented by AI. An introduction to the AI governance problem: the problem of devising global norms, policies, and institutions to best ensure the beneficial development and use of advanced AI.</em></p><h4><strong>80/20</strong></h4><ul><li><a href="https://www.fhi.ox.ac.uk/wp-content/uploads/GovAIAgenda.pdf">AI governance research agenda</a> — Allan Dafoe, 2018</li><li><a href="https://80000hours.org/articles/ai-policy-guide/">80,000 Hours: Guide to working in AI policy and strategy</a> — Miles Brundage, 2017</li><li><a href="https://maliciousaireport.com/">The Malicious Use of Artificial Intelligence — Forecasting, Prevention, and Mitigation</a></li><li><a href="https://futureoflife.org/ai-principles/">Asilomar AI Principles</a> — Future of Life Institute, 2017</li><li><a href="https://80000hours.org/problem-profiles/positively-shaping-artificial-intelligence/">Positively shaping the development of artificial intelligence</a> — Robert Wiblin, 2017</li><li><a href="https://www.huffpost.com/entry/move-37-or-how-ai-can-change-the-world_b_58399703e4b0a79f7433b675?guccounter=1&amp;guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&amp;guce_referrer_sig=AQAAAJmrkQMVHDxNaID0_1A1qyOxDJhtkJTreNpfakVCi9LSP1TT7j8-vq50BmgUST7QrnKroBGMl64Lfp_lbLIjIgCSZPl8cLmhc1o5-STnulwf-PVnuFLKJRfTJEjN-HvsOH1F5UEpcn3qrcxrPuMBOOQ8A6WHQqfFq7wDnFjrNpWQ">Move 37, or how AI can change the world</a> — George Zarkadakis, 2016</li></ul><h4><strong>Deep dive</strong></h4><ul><li><a href="https://www.nature.com/articles/s42256-018-0003-2?WT.feed_name=subjects_philosophy#Sec3">Bridging near- and long-term concerns about AI</a> - Stephen Cave &amp; Seán S. ÓhÉigeartaigh, 2019</li><li><a href="https://www.fhi.ox.ac.uk/wp-content/uploads/Deciphering_Chinas_AI-Dream.pdf">Deciphering China’s AI Dream</a> — Jeff Ding, 2018</li><li><a href="https://ec.europa.eu/digital-single-market/en/news/communication-artificial-intelligence-europe">Communication on Artificial Intelligence</a> — European Commission 2018</li><li><a href="https://obamawhitehouse.archives.gov/sites/default/files/whitehouse_files/microsites/ostp/NSTC/preparing_for_the_future_of_ai.pdf">Preparing for the Future of Artificial Intelligence</a> — The White House, National Science and Technology Council, Committee on Technology, 2016</li><li><a href="https://www.nber.org/papers/w24689">AI and the Economy</a> — NBER Working Paper, Jason Furman &amp; Robert Seamans, 2018</li><li><a href="https://futureoflife.org/ai-aftermath-scenarios/">AI Aftermath Scenarios</a> — Future of Life Institute</li><li><a href="https://www.amazon.co.uk/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0199678111">Superintelligence: Paths, Dangers, Strategies</a>, Ch. 14–15 — Nick Bostrom, 2014</li><li><a href="https://nickbostrom.com/papers/vulnerable.pdf">The Vulnerable World Hypothesis</a> — Nick Bostrom, 2018</li><li><a href="https://nickbostrom.com/papers/openness.pdf">Strategic Implications of Openness in AI Development</a> — Nick Bostrom, 2017</li><li><a href="https://nickbostrom.com/information-hazards.pdf">Information Hazards: A Typology of Potential Harms from Knowledge</a> — Nick Bostrom, 2011</li><li><a href="https://nickbostrom.com/astronomical/waste.html">Astronomical Waste: The Opportunity Cost of Delayed Technological Development</a> — Nick Bostrom, 2003</li><li><a href="https://www.lawfareblog.com/thinking-about-risks-ai-accidents-misuse-and-structure">Thinking About Risks From AI: Accidents, Misuse and Structure </a>— Zwetsloot, Remco and Dafoe (2019)</li></ul><p>You can also keep up to date with the latest developments in the AI space by signing up for <a href="http://importai.net">Import AI</a> by Jack Clark and the <a href="https://rohinshah.com/alignment-newsletter/">Alignment Newsletter</a> by Rohin Shah, and reading them closely every week.</p><p>If you enjoyed these resources and are interested in working on the challenges and opportunities presented by artificial intelligence research, check out the 80,000 Hours <a href="https://80000hours.org/job-board/ai-strategy-governance/">job board</a> to see who’s hiring. If you have questions or feedback, feel free to <a href="https://twitter.com/v_maini">get in touch</a>.</p><p><em>Thanks to </em><a href="https://www.linkedin.com/in/tantum-collins-7b916b29"><em>Teddy Collins</em></a><em>, </em><a href="https://www.linkedin.com/in/holden-karnofsky-75970b7"><em>Holden Karnofsky</em></a><em>, </em><a href="https://twitter.com/lukeprog?lang=en"><em>Luke Muehlhauser</em></a><em>, </em><a href="https://twitter.com/jackclarksf?lang=en"><em>Jack Clark</em></a><em>, </em><a href="https://twitter.com/Miles_Brundage?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Eauthor"><em>Miles Brundage</em></a><em>, </em><a href="https://twitter.com/rohinmshah?lang=en"><em>Rohin Shah</em></a><em>, </em><a href="https://twitter.com/emilyoehlsen?lang=en"><em>Emily Oehlsen</em></a><em>, </em><a href="https://twitter.com/iamtrask?lang=en"><em>Andrew Trask</em></a><em>, </em><a href="https://twitter.com/janleike?lang=en"><em>Jan Leike</em></a><em>, </em><a href="https://twitter.com/seriousssam"><em>Samer Sabri</em></a><em>, </em><a href="https://twitter.com/AmandaNgo11"><em>Amanda Ngo</em></a><em>, </em><a href="https://www.linkedin.com/in/jade-leung-2620bb9a/"><em>Jade Leung</em></a><em>, and </em><a href="https://www.linkedin.com/in/ales-flidr-96764638/?originalSubdomain=uk"><em>Aleš Flidr</em></a><em> for contributing resources and providing feedback on earlier versions of this list.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=c4753afd97a" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/ai-reading-list-c4753afd97a">AI Reading List</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[A Beginner’s Guide to AI/ML ]]></title>
            <link>https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/6164faf1df12</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Sat, 19 Aug 2017 15:54:00 GMT</pubDate>
            <atom:updated>2018-05-28T18:06:44.597Z</atom:updated>
            <content:encoded><![CDATA[<h3>Machine Learning for Humans🤖👶</h3><h4>Simple, plain-English explanations accompanied by math, code, and real-world examples.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Vkf6A8Mb0wBoL3Fw1u0paA.jpeg" /></figure><pre>This series is available as a full-length e-book! <a href="https://www.dropbox.com/s/e38nil1dnl7481q/machine_learning.pdf?dl=0"><strong>Download here</strong></a>. Free for download, contributions appreciated (<a href="http://paypal.me/ml4h"><strong>paypal.me/ml4h</strong></a><strong>)</strong></pre><h3>Roadmap</h3><p><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12"><strong>Part 1: Why Machine Learning Matters</strong></a><strong>. </strong><em>The big picture of artificial intelligence and machine learning — past, present, and future.</em></p><p><a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><strong>Part 2.1: Supervised Learning</strong></a><strong>. </strong><em>Learning with an answer key. Introducing linear regression, loss functions, overfitting, and gradient descent.</em></p><p><a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d"><strong>Part 2.2: Supervised Learning II</strong></a><strong>. </strong><em>Two methods of classification: logistic regression and SVMs.</em></p><p><a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930"><strong>Part 2.3: Supervised Learning III</strong></a><strong>. </strong><em>Non-parametric learners: k-nearest neighbors, decision trees, random forests. Introducing cross-validation, hyperparameter tuning, and ensemble models.</em></p><p><a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294"><strong>Part 3: Unsupervised Learning</strong></a><strong>. </strong><em>Clustering: k-means, hierarchical. Dimensionality reduction: principal components analysis (PCA), singular value decomposition (SVD).</em></p><p><a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b"><strong>Part 4: Neural Networks &amp; Deep Learning</strong></a><strong>.</strong><em> Why, where, and how deep learning works. Drawing inspiration from the brain. Convolutional neural networks (CNNs), recurrent neural networks (RNNs). Real-world applications.</em></p><p><a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265"><strong>Part 5: Reinforcement Learning</strong></a><strong>. </strong><em>Exploration and exploitation. Markov decision processes. Q-learning, policy learning, and deep reinforcement learning. The value learning problem.</em></p><p><a href="https://medium.com/@v_maini/how-to-learn-machine-learning-24d53bb64aa1"><strong>Appendix: The Best Machine Learning Resources</strong></a><strong>. </strong><em>A curated list of resources for creating your machine learning curriculum.</em></p><h3>Who should read this?</h3><ul><li>Technical people who want to get up to speed on machine learning quickly</li><li>Non-technical people who want a primer on machine learning and are willing to engage with technical concepts</li><li>Anyone who is curious about how machines think</li></ul><p>This guide is intended to be accessible to anyone. Basic concepts in probability, statistics, programming, linear algebra, and calculus will be discussed, but it isn’t necessary to have prior knowledge of them to gain value from this series.</p><pre>This series is a guide for getting up-to-speed on high-level machine learning concepts in ~2-3 hours.</pre><pre>If you&#39;re more interested in figuring out which courses to take, textbooks to read, projects to attempt, etc., take a look at our recommendations in the <a href="https://medium.com/machine-learning-for-humans/how-to-learn-machine-learning-24d53bb64aa1">Appendix: The Best Machine Learning Resources</a>.</pre><h3>Why machine learning matters</h3><p>Artificial intelligence will shape our future more powerfully than any other innovation this century. Anyone who does not understand it will soon find themselves feeling left behind, waking up in a world full of technology that feels more and more like magic.</p><p>The rate of acceleration is already astounding. After a couple of <a href="https://en.wikipedia.org/wiki/History_of_artificial_intelligence#The_first_AI_winter_1974.E2.80.931980">AI winters and periods of false hope</a> over the past four decades, rapid advances in data storage and computer processing power have dramatically changed the game in recent years.</p><p>In 2015, Google trained a conversational agent (AI) that could not only convincingly interact with humans as a tech support helpdesk, but also discuss morality, express opinions, and answer general facts-based questions.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*P1H87bkqILoGBVVT7g0T0A.png" /><figcaption>(<a href="https://arxiv.org/abs/1506.05869">Vinyals &amp; Le, 2017</a>)</figcaption></figure><p>The same year, DeepMind developed an <a href="https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf">agent</a> that surpassed human-level performance at 49 Atari games, receiving only the pixels and game score as inputs. Soon after, in 2016, DeepMind obsoleted their own achievement by releasing a new <a href="https://arxiv.org/pdf/1602.01783.pdf">state-of-the-art gameplay method</a> called A3C.</p><p>Meanwhile, <a href="https://deepmind.com/research/publications/mastering-game-go-deep-neural-networks-tree-search/">AlphaGo</a> defeated one of the best human players at Go — an extraordinary achievement in a game dominated by humans for two decades after machines first conquered chess. Many masters could not fathom how it would be possible for a machine to grasp the full nuance and complexity of this ancient Chinese war strategy game, with its 10¹⁷⁰ possible board positions (there are only <a href="http://www.slate.com/articles/technology/technology/2016/03/google_s_alphago_defeated_go_champion_lee_sedol_ken_jennings_explains_what.html">10⁸⁰atoms in the universe</a>).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*2pYq0Qc3oMDYoVoEA9-6cg.png" /><figcaption>Professional Go player Lee Sedol reviewing his match with AlphaGo after defeat. Photo via <a href="https://www.theatlantic.com/technology/archive/2016/03/the-invisible-opponent/475611/">The Atlantic</a>.</figcaption></figure><p>In March 2017, OpenAI created agents that <a href="https://blog.openai.com/learning-to-communicate/">invented their own language</a> to cooperate and more effectively achieve their goal. Soon after, Facebook reportedly successfully training agents to <a href="https://code.facebook.com/posts/1686672014972296/deal-or-no-deal-training-ai-bots-to-negotiate/">negotiate</a> and even <a href="https://www.theregister.co.uk/2017/06/15/facebook_to_teach_chatbots_negotiation/">lie</a>.</p><p>Just a few days ago (as of this writing), on August 11, 2017, OpenAI reached yet another incredible milestone by defeating the world’s top professionals in 1v1 matches of the online multiplayer game Dota 2.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*eWnzwOQX5QgkQ4FRU_I6sg.png" /><figcaption>See the full match at The International 2017, with Dendi (human) vs. OpenAI (bot), on <a href="https://www.youtube.com/watch?v=wiOopO9jTZw">YouTube</a>.</figcaption></figure><p>Much of our day-to-day technology is powered by artificial intelligence. Point your camera at the menu during your next trip to Taiwan and the restaurant’s selections will magically appear in English via the Google Translate app.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*x8IgnfzPL7iLZHa9Uhy50g.png" /><figcaption><em>Google Translate overlaying English translations on a drink menu in real time using convolutional neural networks.</em></figcaption></figure><p>Today AI is used to design <a href="https://www.ibm.com/watson/health/oncology-and-genomics/oncology/">evidence-based treatment plans</a> for cancer patients, instantly analyze results from medical tests to <a href="https://deepmind.com/applied/deepmind-health/">escalate to the appropriate specialist</a> immediately, and conduct <a href="http://benevolent.ai/">scientific research</a> for drug discovery.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*GEo3QHtN3gcWt0b2k08q7w.png" /><figcaption>A bold proclamation by London-based BenevolentAI (screenshot from <a href="http://benevolent.ai/about-us/">About Us</a> page, August 2017).</figcaption></figure><p>In everyday life, it’s increasingly commonplace to discover machines in roles traditionally occupied by humans. Really, don’t be surprised if a little housekeeping delivery bot shows up instead of a human next time you call the hotel desk to send up some toothpaste.</p><h3>Max Prosperi on Twitter</h3><p>OH MY GOD. I HAVE NO WORDS. MY HOTEL JUST SENT A ROBOT TO MY ROOM TO DELIVER SOME TOOTHPASTE! 😂😂😂 I love @spg hotels. #spglife https://t.co/Bf6OcWtqv8</p><p>In this series, we’ll explore the core machine learning concepts behind these technologies. By the end, you should be able to describe how they work at a conceptual level and be equipped with the tools to start building similar applications yourself.</p><h3>The semantic tree: artificial intelligence and machine learning</h3><blockquote>One bit of advice: it is important to view knowledge as sort of a<strong> semantic tree</strong> — make sure you understand the fundamental principles, ie the trunk and big branches, before you get into the leaves/details or there is nothing for them to hang on to. — Elon Musk, <a href="https://www.reddit.com/r/IAmA/comments/2rgsan/i_am_elon_musk_ceocto_of_a_rocket_company_ama/cnfre0a/">Reddit AMA</a></blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*QJG2nMIqWHmLp2j4c0GVuQ.png" /><figcaption>Machine learning is one of many subfields of artificial intelligence, concerning the ways that computers learn from experience to improve their ability to think, plan, decide, and act.</figcaption></figure><p><strong>Artificial intelligence is the study of agents that perceive the world around them, form plans, and make decisions to achieve their goals.</strong> Its foundations include mathematics, logic, philosophy, probability, linguistics, neuroscience, and decision theory. Many fields fall under the umbrella of AI, such as computer vision, robotics, machine learning, and natural language processing.</p><p><strong>Machine learning is a subfield of artificial intelligence</strong>. Its goal is to enable computers to learn on their own. A machine’s learning algorithm enables it to identify patterns in observed data, build models that explain the world, and predict things without having explicit pre-programmed rules and models.</p><pre><strong>The AI effect: what actually qualifies as “artificial intelligence”?</strong></pre><pre>The exact standard for technology that qualifies as “AI” is a bit fuzzy, and interpretations change over time. The AI label tends to describe machines doing tasks traditionally in the domain of humans. Interestingly, once computers figure out how to do one of these tasks, humans have a tendency to say it wasn’t <em>really</em> intelligence. This is known as the <a href="https://en.wikipedia.org/wiki/AI_effect"><strong>AI effect</strong></a>.</pre><pre>For example, when IBM’s Deep Blue defeated world chess champion <a href="https://medium.com/u/66cdbe693a4f">Garry Kasparov</a> in 1997, people complained that it was using &quot;brute force&quot; methods and it wasn’t “real” intelligence at all. As Pamela McCorduck wrote, <em>“It’s part of the history of the field of artificial intelligence that every time somebody figured out how to make a computer do something — play good checkers, solve simple but relatively informal problems — there was chorus of critics to say, ‘that’s not thinking’”</em>(<a href="http://www.pamelamc.com/html/machines_who_think.html">McCorduck, 2004</a>).</pre><pre>Perhaps there is a certain <em>je ne sais quoi</em> inherent to what people will reliably accept as “artificial intelligence”:</pre><pre><em>&quot;AI is whatever hasn&#39;t been done yet.&quot; - Douglas Hofstadter</em></pre><pre>So does a calculator count as AI? Maybe by some interpretation. What about a self-driving car? Today, yes. In the future, perhaps not. Your cool new chatbot startup that automates a flow chart? Sure… why not.</pre><h3>Strong AI will change our world forever; to understand how, studying machine learning is a good place to start</h3><p>The technologies discussed above are examples of <strong>artificial narrow intelligence (ANI)</strong>, which can effectively perform a narrowly defined task.</p><p>Meanwhile, we’re continuing to make foundational advances towards human-level <strong>artificial general intelligence (AGI), </strong>also known as <a href="https://en.wikipedia.org/wiki/Artificial_general_intelligence"><strong>strong AI</strong></a>. The definition of an AGI is an artificial intelligence that can successfully perform <em>any intellectual task that a human being can</em>, including learning, planning and decision-making under uncertainty, communicating in natural language, making jokes, manipulating people, trading stocks, or… reprogramming itself.</p><p>And this last one is a big deal. Once we create an AI that can improve itself, it will unlock a cycle of recursive self-improvement that could lead to an <strong>intelligence explosion</strong> over some unknown time period, ranging from many decades to a single day.</p><blockquote>Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man however clever. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an ‘intelligence explosion,’ and the intelligence of man would be left far behind. Thus the first ultraintelligent machine is the last invention that man need ever make, provided that the machine is docile enough to tell us how to keep it under control. — I.J. Good, 1965</blockquote><p>You may have heard this point referred to as the <strong>singularity</strong>. The term is borrowed from the gravitational singularity that occurs at the center of a black hole, an infinitely dense one-dimensional point where the laws of physics as we understand them start to break down.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1020/1*rR4Hp7-pfgGBDqyPdcnh8g.png" /><figcaption>We have zero visibility into what happens beyond the event horizon of a black hole because no light can escape. Similarly, <strong>after we unlock AI’s ability to recursively improve itself, it’s impossible to predict what will happen, just as</strong> <strong>mice who intentionally designed a human might have trouble predicting what the human would do to their world.</strong> Would it keep helping them get more cheese, as they originally intended? (Image via <a href="http://www.wired.co.uk/article/what-black-holes-explained">WIRED</a>)</figcaption></figure><p>A recent report by the Future of Humanity Institute surveyed a panel of AI researchers on timelines for<strong> </strong>AGI, and found that <strong>“researchers believe there is a 50% chance of AI outperforming humans in all tasks in 45 years” </strong>(<a href="https://arxiv.org/pdf/1705.08807.pdf">Grace et al, 2017</a>)<strong>.</strong> We’ve personally spoken with a number of sane and reasonable AI practitioners who predict much longer timelines (the upper limit being “never”), and others whose timelines are alarmingly short — as little as a few years.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*2TpuuqUKnhdnr5eK." /><figcaption><em>Image from Kurzweil’s The Singularity Is Near, published in 2005. Now, in 2017, only a couple of these posters could justifiably remain on the wall.</em></figcaption></figure><p>The advent of greater-than-human-level <strong>artificial superintelligence (ASI) </strong>could be one of the best or worst things to happen to our species.<strong> </strong>It carries with it the immense challenge of specifying what AIs will <em>want </em>in a way that is friendly to humans.</p><p>While it’s impossible to say what the future holds, one thing is certain: <strong>2017 is a good time to start understanding how machines think.</strong> To go beyond the abstractions of a philosopher in an armchair and intelligently shape our roadmaps and policies with respect to AI, we must engage with the details of how machines see the world — what they “want”, their potential biases and failure modes, their temperamental quirks — just as we study psychology and neuroscience to understand how humans learn, decide, act, and feel.</p><pre>There are complex, high-stakes questions about AI that will require  our careful attention in the coming years.</pre><pre>How can we combat AI’s propensity to <a href="https://www.google.com/intl/en/about/gender-balance-diversity-important-to-machine-learning/">further entrench systemic biases</a> evident in existing data sets? What should we make of fundamental <a href="http://fortune.com/2017/07/26/mark-zuckerberg-argues-against-elon-musks-view-of-artificial-intelligence-again/">disagreements among the world’s most powerful technologists</a> about the potential risks and benefits of artificial intelligence? What will happen to humans&#39; sense of purpose in a world without work?</pre><p>Machine learning is at the core of our journey towards artificial general intelligence, and in the meantime, it will change every industry and have a massive impact on our day-to-day lives. That’s why we believe it’s worth understanding machine learning, at least at a conceptual level — and we designed this series to be the best place to start.</p><h3>How to read this series</h3><p>You don’t necessarily need to read the series cover-to-cover to get value out of it. Here are three suggestions on how to approach it, depending on your interests and how much time you have:</p><ol><li><strong>T-shaped approach. </strong>Read from beginning to end. Summarize each section in your own words as you go (see: <a href="https://mattyford.com/blog/2014/1/23/the-feynman-technique-model">Feynman technique</a>); this encourages active reading &amp; stronger retention. Go deeper into areas that are most relevant to your interests or work. We’ll include resources for further exploration at the end of each section.</li><li><strong>Focused approach. </strong>Jump straight to the sections you’re most curious about and focus your mental energy there.</li><li><a href="https://www.thebalance.com/pareto-s-principle-the-80-20-rule-2275148"><strong>80/20 approach</strong></a><strong>. </strong>Skim everything in one go, make a few notes on interesting high-level concepts, and call it a night. 😉</li></ol><h3>About the authors</h3><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*UWNsFVQBaDW5dq1HnW9k4w.png" /><figcaption>“Ok, we have to be done with gradient descent by the time we finish this ale.” @ <a href="https://medium.com/u/cfb565c2bdd">The Boozy Cow</a> in Edinburgh</figcaption></figure><p><a href="https://www.linkedin.com/in/vishalmaini/">Vishal</a> most recently led growth at <a href="https://www.upstart.com/about#future-of-credit-2">Upstart</a>, a lending platform that utilizes machine learning to price credit, automate the borrowing process, and acquire users. He spends his time thinking about startups, applied cognitive science, moral philosophy, and the ethics of artificial intelligence.</p><p><a href="https://www.linkedin.com/in/samer-sabri-8995a717/">Samer</a> is a Master’s student in Computer Science and Engineering at UCSD and co-founder of <a href="http://www.conigolabs.com/">Conigo Labs</a>. Prior to grad school, he founded TableScribe, a business intelligence tool for SMBs, and spent two years advising Fortune 100 companies at McKinsey. Samer previously studied Computer Science and Ethics, Politics, and Economics at Yale.</p><p>Most of this series was written during a 10-day trip to the United Kingdom in a frantic blur of trains, planes, cafes, pubs and wherever else we could find a dry place to sit. Our aim was to solidify our own understanding of artificial intelligence, machine learning, and how the methods therein fit together — and hopefully create something worth sharing in the process.</p><p>And now, without further ado, let’s dive into machine learning with <a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><strong>Part 2.1: Supervised Learning</strong></a>!</p><h4>Enter your email below if you’d like to stay up-to-date with future content 💌</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f8c40%3Fas_embed%3Dtrue&amp;url=https%3A%2F%2Fupscri.be%2F8f8c40%2F&amp;image=https%3A%2F%2Fupscri.be%2Fmedia%2Fform.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href">https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href</a></iframe><h4>On Twitter? So are we. Feel free to keep in touch — <a href="https://twitter.com/v_maini">Vishal</a> and <a href="https://twitter.com/seriousssam">Samer</a> 🙌🏽.</h4><p><em>More from</em><strong><em> Machine Learning for Humans </em></strong>🤖👶</p><ul><li><strong><em>Part 1: Why Machine Learning Matters </em>✅</strong></li><li><a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><em>Part 2.1: Supervised Learning</em></a></li><li><a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d"><em>Part 2.2: Supervised Learning II</em></a></li><li><a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930"><em>Part 2.3: Supervised Learning III</em></a></li><li><a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294"><em>Part 3: Unsupervised Learning</em></a></li><li><a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b"><em>Part 4: Neural Networks &amp; Deep Learning</em></a></li><li><a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265"><em>Part 5: Reinforcement Learning</em></a></li><li><a href="https://medium.com/@v_maini/how-to-learn-machine-learning-24d53bb64aa1"><em>Appendix: The Best Machine Learning Resources</em></a></li></ul><h4>Contact: <a href="mailto:ml4humans@gmail.com">ml4humans@gmail.com</a></h4><p><em>A special thanks to </em><a href="https://www.linkedin.com/in/jonathaneng1/"><em>Jonathan Eng</em></a><em>, </em><a href="https://www.linkedin.com/in/edoardoconti/"><em>Edoardo Conti</em></a><em>, </em><a href="https://www.linkedin.com/in/grantwschneider/"><em>Grant Schneider</em></a><em>, </em><a href="https://www.linkedin.com/in/sunnykumar1/"><em>Sunny Kumar</em></a><em>, </em><a href="https://www.linkedin.com/in/stephanieyhe/"><em>Stephanie He</em></a><em>, </em><a href="https://www.linkedin.com/in/tarunw/"><em>Tarun Wadhwa</em></a><em>, and </em><a href="https://www.linkedin.com/in/sachinmaini/"><em>Sachin Maini</em></a><em> (series editor) for their significant contributions and feedback.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6164faf1df12" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12">A Beginner’s Guide to AI/ML 🤖👶</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The Best Machine Learning Resources]]></title>
            <link>https://medium.com/machine-learning-for-humans/how-to-learn-machine-learning-24d53bb64aa1?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/24d53bb64aa1</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Sat, 19 Aug 2017 15:25:02 GMT</pubDate>
            <atom:updated>2019-05-27T17:39:16.604Z</atom:updated>
            <content:encoded><![CDATA[<h4>A compendium of resources for crafting a curriculum on artificial intelligence, machine learning, and deep learning.</h4><pre>This article is an addendum to the series <a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12">Machine Learning for Humans 🤖👶</a>, a guide for getting up-to-speed on machine learning concepts in 2-3 hours.</pre><h3>General advice on crafting a curriculum</h3><p>Going to school for a formal degree program for isn’t always possible or desirable. For those considering an autodidactic alternative, this is for you.</p><p><strong>1. Build foundations, and then specialize in areas of interest.</strong></p><p>You can’t go deeply into every machine learning topic. There’s too much to learn, and the field is advancing rapidly. Master foundational concepts and then focus on projects in a specific domain of interest — whether it’s natural language understanding, computer vision, deep reinforcement learning, robotics, or whatever else.</p><p><strong>2. Design your curriculum around topics that personally excite you.</strong></p><p>Motivation is far more important than micro-optimizing a learning strategy for some long-term academic or career goal. If you’re having fun, you’ll make fast progress. If you’re trying to force yourself forward, you’ll slow down.</p><pre>We’ve included resources that we explored personally or came highly recommended. This list is not meant to be exhaustive. There are endless options, and too much choice is counterproductive. But if we’re missing a great resource that belongs here, please <a href="mailto:ml4humans@gmail.com">reach out</a>!</pre><h3>Foundations</h3><h4>Programming</h4><pre>Syntax and basic concepts: <a href="https://developers.google.com/edu/python/">Google’s Python Class</a>, <a href="https://learnpythonthehardway.org/book/ex0.html">Learn Python the Hard Way</a>.</pre><pre>Practice: <a href="https://coderbyte.com/">Coderbyte</a>, <a href="https://www.codewars.com/">Codewars</a>, <a href="https://www.hackerrank.com/">HackerRank</a>.</pre><h4>Linear algebra</h4><pre><a href="http://www.deeplearningbook.org/contents/linear_algebra.html">Deep Learning Book, Chapter 2: Linear Algebra</a>. A quick review of the linear algebra concepts relevant to machine learning.</pre><pre><a href="https://www.amazon.com/First-Course-Linear-Model-Theory/dp/1584882476">A First Course in Linear Model Theory</a> by Nalini Ravishanker and Dipak Dey. Textbook introducing linear algebra in a statistical context. </pre><h4>Probability &amp; statistics</h4><pre>MIT 18.05, <a href="https://ocw.mit.edu/courses/mathematics/18-05-introduction-to-probability-and-statistics-spring-2014/index.htm">Introduction to Probability and Statistics</a>, taught by Jeremy Orloff and Jonathan Bloom. Provides intuition for probabilistic reasoning &amp; statistical inference, which is invaluable for understanding how machines think, plan, and make decisions.</pre><pre><a href="http://www.ic.unicamp.br/~wainer/cursos/1s2013/ml/livro.pdf">All of Statistics: A Concise Course in Statistical Inference</a>, by Larry Wasserman. Introductory text on statistics.</pre><h4>Calculus</h4><pre>Khan Academy: <a href="https://www.khanacademy.org/math/calculus-home/differential-calculus">Differential Calculus</a>. Or, any introductory calculus course or textbook.</pre><pre>Stanford CS231n: <a href="http://cs231n.stanford.edu/handouts/derivatives.pdf">Derivatives, Backpropagation, and Vectorization</a>, prepared by Justin Johnson.</pre><h3>Machine learning</h3><pre><strong>Courses</strong></pre><pre>Andrew Ng’s <a href="https://www.coursera.org/learn/machine-learning">Machine Learning</a> course on Coursera (or, for more rigor, <a href="http://cs229.stanford.edu/materials.html">Stanford CS229</a> - see <a href="https://github.com/afshinea/stanford-cs-229-machine-learning">cheat sheet</a>).</pre><pre>Data science bootcamps: <a href="https://www.galvanize.com/san-francisco/data-science">Galvanize</a> (full-time, 3 months, $$$$), <a href="https://www.thinkful.com/bootcamp/data-science/flexible/">Thinkful</a> (flexible schedule, 6 months, $$).</pre><pre><strong>Textbook</strong></pre><pre><a href="http://www-bcf.usc.edu/~gareth/ISL/">An Introduction to Statistical Learning</a> by Gareth James et al. Excellent reference for essential machine learning concepts, available free online.</pre><h3>Deep learning</h3><pre><strong>Courses</strong></pre><pre><a href="http://deeplearning.ai">Deeplearning.ai</a>, Andrew Ng’s introductory deep learning course.</pre><pre><a href="http://cs231n.stanford.edu/syllabus.html">CS231n: Convolutional Neural Networks for Visual Recognition</a>, Stanford’s deep learning course. Helpful for building foundations, with engaging lectures and illustrative problem sets.</pre><pre><strong>Projects</strong></pre><pre><a href="http://fast.ai">Fast.ai</a>, a fun and hands-on project-based course. Projects include classifying images of dogs vs. cats and generating Nietzschean writing.</pre><pre><a href="https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/#0">MNIST handwritten digit classification with TensorFlow</a>. Classify handwritten digits with &gt;99% accuracy in 3 hours with this tutorial by Google.</pre><pre>Try your hand at <a href="https://www.kaggle.com/">a Kaggle competition</a>. Implement a deep learning paper that you found interesting, using other versions on GitHub as reference material.</pre><pre><strong>Reading</strong></pre><pre><a href="http://www.deeplearningbook.org/">Deep Learning Book</a>, a.k.a. the Bible of Deep Learning, authored by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.</pre><pre><a href="http://neuralnetworksanddeeplearning.com/chap1.html">Neural Networks and Deep Learning</a>, a clear and accessible online deep learning text by Michael Nielsen. Ends with commentary on reaching human-level intelligence.</pre><pre><a href="https://github.com/songrotek/Deep-Learning-Papers-Reading-Roadmap">Deep Learning Papers Reading Roadmap</a>, a compilation of key papers organized by chronology and research area. </pre><h3>Reinforcement learning</h3><pre><strong>Courses</strong></pre><pre>John Schulman’s <a href="http://rll.berkeley.edu/deeprlcourse/">CS 294: Deep Reinforcement Learning</a> at Berkeley.</pre><pre>David Silver’s <a href="http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html">Reinforcement Learning</a> course at University College London.</pre><pre><a href="https://www.deepbootcamp.io/">Deep RL Bootcamp</a>, organized by OpenAI and UC Berkeley. Applications are currently closed, but it’s worth keeping an eye out for future sessions.</pre><pre><strong>Projects</strong></pre><pre>Andrej Karpathy’s <a href="http://karpathy.github.io/2016/05/31/rl/">Pong from Pixels</a>. Implement a Pong-playing agent from scratch in 130 lines of code. </pre><pre>Arthur Juliani’s <a href="https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0">Simple Reinforcement Learning with Tensorflow</a> series. Implement Q-learning, policy-learning, actor-critic methods, and strategies for exploration using TensorFlow. </pre><pre>See OpenAI’s <a href="https://openai.com/requests-for-research/">requests for research</a> for more project ideas.</pre><pre><strong>Reading</strong></pre><pre>Richard Sutton’s book, <a href="http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.pdf">Reinforcement Learning: An Introduction</a>.</pre><h3>Artificial intelligence</h3><pre><a href="http://aima.cs.berkeley.edu/">Artificial Intelligence: A Modern Approach</a> by Stuart Russell and Peter Norvig.  </pre><pre>Sebastian Thrun’s Udacity course, <a href="https://www.udacity.com/course/intro-to-artificial-intelligence--cs271">Intro to Artificial Intelligence</a>.</pre><pre>Fellowships: <a href="http://insightdata.ai/">Insight AI Fellows Program</a>, <a href="https://research.google.com/teams/brain/residency/">Google Brain Residency Program</a></pre><h3>Artificial intelligence safety</h3><pre>For the short version, read: (1) Johannes Heidecke’s <a href="https://thinkingwires.com/posts/2017-07-05-risks.html">Risks of Artificial Intelligence</a>, (2) OpenAI and Google Brain’s collaboration on <a href="https://blog.openai.com/concrete-ai-safety-problems/">Concrete Problems in AI Safety</a>, and (3) Wait But Why’s article on the <a href="https://waitbutwhy.com/2015/01/artificial-intelligence-revolution-1.html">AI Revolution</a>.</pre><pre>For the longer version, see Nick Bostrom’s <a href="https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834">Superintelligence</a>.</pre><pre>Check out the research published by the <a href="https://intelligence.org/research/">Machine Intelligence Research Institute</a> (MIRI) and <a href="https://www.fhi.ox.ac.uk/research/research-areas/">Future of Humanity Institute</a> (FHI) on AI safety.</pre><pre>Keep up-to-date with <a href="https://www.reddit.com/r/ControlProblem/">/r/ControlProblem</a> on Reddit.</pre><h3>Newsletters</h3><pre><a href="https://jack-clark.net/import-ai/">Import AI</a>, weekly AI newsletter covering the latest developments in the industry. Prepared by Jack Clark of OpenAI.</pre><pre><a href="https://machinelearnings.co/">Machine Learnings</a>, prepared by Sam DeBrule. Frequent guest appearances from experts in the field.</pre><pre><a href="http://nathan.ai">Nathan.ai</a>, covering recent news and commenting on AI/ML from a venture capital perspective.</pre><pre><a href="https://www.getrevue.co/profile/wildml">The Wild Week in AI</a> by Denny Britz. The title says it all.</pre><h3>Advice from others</h3><pre><a href="https://www.forbes.com/sites/quora/2017/03/22/what-is-the-best-way-to-learn-machine-learning-without-taking-any-online-courses/#30fc6e5d5d87">“What is the best way to learn machine learning without taking any online courses?</a> — answered by Eric Jang, Google Brain</pre><pre><a href="https://www.quora.com/What-are-the-best-ways-to-pick-up-Deep-Learning-skills-as-an-engineer">What are the best ways to pick up deep learning skills as an engineer?&quot;</a> - answered by Greg Brockman, CTO of OpenAI</pre><pre>A16z&#39;s <a href="http://aiplaybook.a16z.com/">AI Playbook</a>, a more code-based introduction to AI</pre><pre><a href="https://80000hours.org/ai-safety-syllabus/">AI safety syllabus</a>, designed by 80,000 Hours</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*dmj5qeZdqnVLZTqj25SCpw.png" /></figure><blockquote>“You take the blue pill, the story ends. You wake up in your bed and believe whatever you want to believe. You take the red pill, you stay in Wonderland, and I show you how deep the rabbit hole goes.” — Morpheus</blockquote><p>Good luck!</p><pre>If you&#39;re interested in sponsoring future work, we appreciate any amount you are able to contribute: <a href="http://paypal.me/ml4h"><strong>paypal.me/ml4h</strong></a></pre><p><em>More from</em><strong><em> Machine Learning for Humans </em></strong>🤖👶</p><ul><li><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12"><em>Part 1: Why Machine Learning Matters</em></a><strong><em> </em>✅</strong></li><li><a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><em>Part 2.1: Supervised Learning</em></a><em> </em><strong>✅</strong></li><li><a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d"><em>Part 2.2: Supervised Learning II</em></a><strong><em> </em>✅</strong></li><li><a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930"><em>Part 2.3: Supervised Learning III</em></a><em> </em><strong>✅</strong></li><li><a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294"><em>Part 3: Unsupervised Learning</em></a><em> </em><strong>✅</strong></li><li><a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b"><em>Part 4: Neural Networks &amp; Deep Learning</em></a><em> </em><strong>✅</strong></li><li><a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265"><em>Part 5: Reinforcement Learning</em></a><em> </em><strong>✅</strong></li><li><strong><em>Appendix: The Best Machine Learning Resources </em>✅</strong></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=24d53bb64aa1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/how-to-learn-machine-learning-24d53bb64aa1">The Best Machine Learning Resources</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning for Humans, Part 5: Reinforcement Learning]]></title>
            <link>https://medium.com/machine-learning-for-humans/reinforcement-learning-6eacf258b265?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/6eacf258b265</guid>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tech]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Sat, 19 Aug 2017 15:24:42 GMT</pubDate>
            <atom:updated>2019-05-29T20:51:05.431Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>Exploration and exploitation. Markov decision processes. Q-learning, policy learning, and deep reinforcement learning.</em></h4><pre>This series is available as a full-length e-book! <a href="https://www.dropbox.com/s/e38nil1dnl7481q/machine_learning.pdf?dl=0"><strong>Download here</strong></a>. Free for download, contributions appreciated (<a href="http://paypal.me/ml4h"><strong>paypal.me/ml4h</strong></a><strong>)</strong></pre><h4><em>“I just ate some chocolate for finishing the last section.”</em></h4><p>In supervised learning, training data comes with an answer key from some godlike “supervisor”. If only life worked that way!</p><p>In <strong>reinforcement learning (RL) </strong>there’s no answer key, but your reinforcement learning <strong>agent</strong> still has to decide how to act to perform its task. In the absence of existing training data, the agent learns from experience. It collects the training examples (“this action was good, that action was bad”) through <strong>trial-and-error</strong> as it attempts its task, with the goal of maximizing long-term <strong>reward</strong>.</p><p>In this final section of Machine Learning for Humans, we will explore:</p><ul><li>The <strong>exploration/exploitation</strong> tradeoff</li><li><strong>Markov Decision Processes (MDPs)</strong>, the classic setting for RL tasks</li><li><strong>Q-learning, policy learning, </strong>and <strong>deep reinforcement learning</strong></li><li>and lastly, the <strong>value learning problem</strong></li></ul><p>At the end, as always, we’ve compiled some favorite resources for further exploration.</p><h3>Let’s put a robot mouse in a maze</h3><p>The easiest context in which to think about reinforcement learning is in games with a clear objective and a point system.</p><p>Say we’re playing a game where our mouse 🐭 is seeking the ultimate reward of cheese at the end of the maze (🧀 +1000 points), or the lesser reward of water along the way (💧+10 points). Meanwhile, robo-mouse wants to avoid locations that deliver an electric shock (⚡ -100 points).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*jne9wcY21o_e_ztLyQSPSw.png" /><figcaption>The reward is cheese.</figcaption></figure><p>After a bit of <strong>exploration, </strong>the mouse might find the mini-paradise of three water sources clustered near the entrance and spend all its time <strong>exploiting </strong>that discovery by continually racking up the small rewards of these water sources and never going further into the maze to pursue the larger prize.</p><p>But as you can see, the mouse would then miss out on an even better oasis further in the maze, or the ultimate reward of cheese at the end!</p><p>This brings up the <strong>exploration/exploitation</strong> tradeoff. One simple strategy for exploration would be for the mouse to take the best known action most of the time (say, 80% of the time), but occasionally explore a new, randomly selected direction even though it might be walking away from known reward.</p><p>This strategy is called the <strong>epsilon-greedy </strong>strategy, where <strong>epsilon</strong> is the percent of the time that the agent takes a randomly selected action rather than taking the action that is most likely to maximize reward given what it knows so far (in this case, 20%). We usually start with a lot of exploration (i.e. a higher value for epsilon). Over time, as the mouse learns more about the maze and which actions yield the most long-term reward, it would make sense to steadily reduce epsilon to 10% or even lower as it settles into exploiting what it knows.</p><p>It’s important to keep in mind that the reward is not always immediate: in the robot-mouse example, there might be a long stretch of the maze you have to walk through and several decision points before you reach the cheese.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/478/1*HvoLc50Dpq1ESKuejhICHg.png" /><figcaption>The agent <strong>observes</strong> the environment, takes an <strong>action</strong> to interact with the environment, and receives positive or negative <strong>reward. </strong>Diagram from Berkeley’s <a href="http://rll.berkeley.edu/deeprlcourse-fa15/">CS 294: Deep Reinforcement Learning</a> by John Schulman &amp; Pieter Abbeel</figcaption></figure><h3>Markov Decision Processes (MDPs)</h3><p>The mouse’s wandering through the maze can be formalized as a <strong>Markov Decision Process</strong>, which is a process that has specified transition probabilities from state to state. We will explain it by referring to our robot-mouse example. MDPs include:</p><ol><li><strong>A finite set of states.</strong> These are the possible positions of our mouse within the maze.</li><li><strong>A set of actions available in each state.</strong> This is {forward, back} in a corridor and {forward, back, left, right} at a crossroads.</li><li><strong>Transitions between states.</strong> For example, if you go left at a crossroads you end up in a new position. These can be a set of probabilities that link to more than one possible state (e.g. when you use an attack in a game of Pokémon you can either miss, inflict some damage, or inflict enough damage to knock out your opponent).</li><li><strong>Rewards associated with each transition.</strong> In the robot-mouse example, most of the rewards are 0, but they’re positive if you reach a point that has water or cheese and negative if you reach a point that has an electric shock.</li><li><strong>A discount factor γ between 0 and 1. </strong>This quantifies the difference in importance between immediate rewards and future rewards. For example, if γ is .9, and there’s a reward of 5 after 3 steps, the present value of that reward is .9³*5.</li><li><strong>Memorylessness. </strong>Once the current state is known, the history of the mouse’s travels through the maze can be erased because the current Markov state contains all useful information from the history. In other words, <strong>“</strong>the future is independent of the past given the present”.</li></ol><p>Now that we know what an MDP is, we can formalize the mouse’s objective. We’re trying to maximize the sum of rewards in the long term:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/204/0*qHo_9UFMQd7kJlr2." /></figure><p>Let’s look at this sum term by term. First of all, we’re summing across all time steps t. Let’s set γ at 1 for now and forget about it. r(x,a) is a reward function. For state <em>x</em> and action <em>a</em> (i.e., go left at a crossroads) it gives you the reward associated with taking that action <em>a</em> at state <em>x</em>. Going back to our equation, we’re trying to maximize the sum of future rewards by taking the best action in each state.</p><p>Now that we’ve set up our reinforcement learning problem and formalized the goal, let’s explore some possible solutions.</p><h3>Q-learning: learning the action-value function</h3><p>Q-learning is a technique that evaluates which action to take based on an <strong>action-value function</strong> that determines the value of being in a certain state and taking a certain action at that state.</p><p>We have a function Q that takes as an input one state and one action and returns the <strong>expected reward</strong> of that action (and all subsequent actions) at that state. Before we explore the environment, Q gives the same (arbitrary) fixed value. But then, as we explore the environment more, Q gives us a better and better approximation of the value of an action <em>a</em> at a state <em>s</em>. We update our function Q as we go.</p><p>This equation from the <a href="https://en.wikipedia.org/wiki/Q-learning">Wikipedia page on Q-learning</a> explains it all very nicely. It shows how we update the value of Q based on the reward we get from our environment:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*q8Dnp4guvDD230if." /></figure><p>Let’s ignore the discount factor γ by setting it to 1 again. First, keep in mind that Q is supposed to show you the full sum of rewards from choosing action Q and all the optimal actions afterward.</p><p>Now let’s go through the equation from left to right. When we take action at in state st, we update our value of Q(st,at) by adding a term to it. This term contains:</p><ul><li>Learning rate <strong>alpha</strong>: this is how aggressive we want to be when updating our value. When alpha is close to 0, we’re not updating very aggressively. When alpha is close to 1, we’re simply replacing the old value with the updated value.</li><li>The <strong>reward</strong> is the reward we got by taking action at<em> </em>at state st. So we’re adding this reward to our old estimate.</li><li>We’re also adding the <strong>estimated future reward</strong>, which is the maximum achievable reward Q for all available actions at xt+1.</li><li>Finally, we subtract the old value of Q to make sure that we’re only incrementing or decrementing by the difference in the estimate (multiplied by alpha of course).</li></ul><p>Now that we have a value estimate for each state-action pair, we can select which action to take according to our <strong>action-selection strategy</strong> (we don’t necessarily just choose the action that leads to the most expected reward every time, e.g. with an epsilon-greedy exploration strategy we’d take a random action some percentage of the time).</p><p>In the robot mouse example, we can use Q-learning to figure out the value of each position in the maze and the value of the actions {forward, backward, left, right} at each position. Then we can use our action-selection strategy to choose what the mouse actually does at each time step.</p><h3>Policy learning: a map from state to action</h3><p>In the Q-learning approach, we learned a <strong>value function</strong> that estimated the value of each state-action pair.</p><p><strong>Policy learning </strong>is a more straightforward alternative in which we learn a <strong>policy function, π</strong>,<strong> </strong>which is a direct map from each state to the best corresponding action at that state. Think of it as a behavioral policy: “when I observe state <em>s</em>, the best thing to do is take action <em>a</em>”. For example, an autonomous vehicle’s policy might effectively include something like: “if I see a yellow light and I am more than 100 feet from the intersection, I should brake. Otherwise, keep moving forward.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/109/1*k-34jR4lNcf2UddHBwBNFQ.png" /><figcaption>A <strong>policy</strong> is a map from <strong>state</strong> to <strong>action.</strong></figcaption></figure><p>So we’re learning a function that will maximize expected reward. What do we know that’s really good at learning complex functions? Deep neural networks!</p><p>Andrej Karpathy’s <a href="http://karpathy.github.io/2016/05/31/rl/">Pong from Pixels</a> provides an excellent walkthrough on using <strong>deep reinforcement learning</strong> to learn a policy for the Atari game Pong that takes raw pixels from the game as the input (state) and outputs a probability of moving the paddle up or down (action).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/483/1*NR9OWGVDOwRKqmHW0GUpfw.png" /><figcaption>In a policy gradient network, the agent learns the optimal policy by adjusting its weights through gradient descent based on reward signals from the environment. Image via <a href="http://karpathy.github.io/2016/05/31/rl/">http://karpathy.github.io/2016/05/31/rl/</a></figcaption></figure><p>If you want to get your hands dirty with deep RL, work through Andrej’s post. You will implement a 2-layer policy network in 130 lines of code, and will also learn how to plug into OpenAI’s <a href="https://gym.openai.com/">Gym</a>, which allows you to quickly get up and running with your first reinforcement learning algorithm, test it on a variety of games, and see how its performance compares to other submissions.</p><h3>DQNs, A3C, and advancements in deep RL</h3><p>In 2015, DeepMind used a method called <strong>deep Q-networks (DQN)</strong>, an approach that approximates Q-functions using deep neural networks, to beat human benchmarks across many Atari games:</p><p><em>We demonstrate that the deep Q-network agent, receiving only the pixels and the game score as inputs, was able to surpass the performance of all previous algorithms and achieve a level comparable to that of a professional human games tester across a set of 49 games, using the same algorithm, network architecture and hyperparameters. This work bridges the divide between high-dimensional sensory inputs and actions, resulting in the first artificial agent that is capable of learning to excel at a diverse array of challenging tasks. (</em><a href="https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf"><em>Mnih et al., 2015</em></a><em>)</em></p><p>Here is a snapshot of where DQN agents stand relative to linear learners and humans in various domains:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/738/1*EbPjTL7wylg1kHId55TODw.png" /><figcaption>These are normalized with respect to professional human games testers: 0% = random play, 100% = human performance. Source: DeepMind’s DQN paper, <a href="https://storage.googleapis.com/deepmind-media/dqn/DQNNaturePaper.pdf">Human-level control through deep reinforcement learning</a></figcaption></figure><p>To help you build some intuition for how advancements are made in RL research, here are some examples of improvements on attempts at non-linear Q-function approximators that can improve performance and stability:</p><ul><li><strong>Experience replay</strong>, which learns by randomizing over a longer sequence of previous observations and corresponding reward to avoid overfitting to recent experiences. This idea is <a href="https://deepmind.com/blog/ai-and-neuroscience-virtuous-circle/">inspired by biological brains</a>: rats traversing mazes, for example, “replay” patterns of neural activity during sleep in order to optimize future behavior in the maze.</li><li><strong>Recurrent neural networks (RNNs) </strong>augmenting DQNs. When an agent can only see its immediate surroundings (e.g. robot-mouse only seeing a certain segment of the maze vs. a birds-eye view of the whole maze), the agent needs to remember the bigger picture so it remembers where things are. This is similar to how humans babies develop <a href="https://en.wikipedia.org/wiki/Object_permanence">object permanence</a> to know things exist even if they leave the baby’s visual field. RNNs are “recurrent”, i.e. <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">they allow information to persist</a> on a longer-term basis. Here’s an impressive video of a <strong>deep recurrent Q-network (DQRN)</strong> playing Doom.</li></ul><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fvideoseries%3Flist%3DPLduGZax9wmiHg-XPFSgqGg8PEAV51q1FT&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Doo0TraGu6QY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Foo0TraGu6QY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2286543cfd01ba0ac858ada4857dc635/href">https://medium.com/media/2286543cfd01ba0ac858ada4857dc635/href</a></iframe><p>In 2016, just one year after the DQN paper, DeepMind revealed another algorithm called <strong>Asynchronous Advantage Actor-Critic (A3C)</strong> that surpassed state-of-the-art performance on Atari games after training for half as long (<a href="https://arxiv.org/pdf/1602.01783.pdf">Mnih et al., 2016</a>). A3C is an <strong>actor-critic</strong> algorithm that combines the best of both approaches we explored earlier: it uses an <strong>actor</strong> (a policy network that decides how to act) AND a <strong>critic </strong>(a Q-network that decides how valuable things are). Arthur Juliani has a nice <a href="https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-8-asynchronous-actor-critic-agents-a3c-c88f72a5e9f2">writeup</a> on how A3C works specifically. A3C is now OpenAI’s <a href="https://github.com/openai/universe-starter-agent">Universe Starter Agent</a>.</p><p>Since then, there have been countless fascinating breakthroughs — from AIs <a href="https://blog.openai.com/learning-to-cooperate-compete-and-communicate/">inventing their own language</a> to <a href="https://deepmind.com/blog/producing-flexible-behaviours-simulated-environments/">teaching themselves to walk in a variety of terrains</a>. This series only scratches the surface on the cutting edge of RL, but hopefully it will serve as a starting point for further exploration!</p><p>As a parting note, we’d like to share this incredible video of DeepMind’s agents that learned to walk… with added sound. Grab some popcorn, turn up the volume, and witness the full glory of artificial intelligence.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FitACOKJHYmw%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DitACOKJHYmw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FitACOKJHYmw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e7187ecd760a815468c4e79c622dc625/href">https://medium.com/media/e7187ecd760a815468c4e79c622dc625/href</a></iframe><h3>Practice materials &amp; further reading</h3><h4>Code</h4><ul><li><em>Andrej Karpathy’s </em><a href="http://karpathy.github.io/2016/05/31/rl/"><em>Pong from Pixels</em></a><em> will get you up-and-running quickly with your first reinforcement learning agent. As the article describes, “we’ll learn to play an ATARI game (Pong!) with PG, from scratch, from pixels, with a deep neural network, and the whole thing is 130 lines of Python only using numpy as a dependency (</em><a href="https://gist.github.com/karpathy/a4166c7fe253700972fcbc77e4ea32c5"><em>Gist link</em></a><em>).”</em></li><li><em>Next, we’d highly recommend Arthur Juliani’s </em><a href="https://medium.com/emergent-future/simple-reinforcement-learning-with-tensorflow-part-0-q-learning-with-tables-and-neural-networks-d195264329d0"><em>Simple Reinforcement Learning with Tensorflow</em></a><em> tutorial. It walks through DQNs, policy learning, actor-critic methods, and strategies for exploration with implementations using TensorFlow. Try understanding and then re-implementing the methods covered.</em></li></ul><h4><em>Reading + lectures</em></h4><ul><li><em>Richard Sutton’s book, </em><a href="http://people.inf.elte.hu/lorincz/Files/RL_2006/SuttonBook.pdf"><em>Reinforcement Learning: An Introduction</em></a><em> — a fantastic book, very readable</em></li><li><em>John Schulman’s </em><a href="http://rll.berkeley.edu/deeprlcourse/"><em>CS 294: Deep Reinforcement Learning</em></a><em> (Berkeley)</em></li><li><em>David Silver’s </em><a href="http://www0.cs.ucl.ac.uk/staff/d.silver/web/Teaching.html"><em>Reinforcement Learning</em></a><em> course (UCL)</em></li></ul><h3>YOU DID IT!</h3><blockquote><em>If you’ve made it this far, that is all the reward we could hope for.</em></blockquote><blockquote><em>We hope you enjoyed the series as an introduction to machine learning. We’ve compiled some of our favorite ML resources in the </em><a href="https://medium.com/@v_maini/machine-learning-for-humans-part-5-the-best-machine-learning-24d53bb64aa1"><em>Appendix</em></a><em> if you’re ready to see how deep this rabbit hole goes.</em></blockquote><blockquote><em>Please don’t hesitate to </em><a href="mailto:ml4humans@gmail.com"><em>reach out</em></a><em> with thoughts, questions, feedback, or your favorite GIFs!</em></blockquote><blockquote><em>Until next time,</em></blockquote><blockquote><em>Vishal and Samer</em></blockquote><pre><strong>Closing thoughts</strong></pre><pre>There is a fundamental question that inspired this series, and we&#39;d like to pose it to you as well. </pre><pre><em>What is our objective function, as humans? </em>How do we define the reward<em> </em>that we maximize in our real lives? Beyond base pleasure and pain, our definition of reward also tends to include messy things like right and wrong, fulfillment, love, spirituality, and purpose.</pre><pre>There has been an intellectual field dedicated to the question of what our objective functions are or should be since ancient times, and it’s called <strong>moral philosophy</strong>. The central question of moral philosophy is: what ought we do? How should we live? Which actions are right or wrong? The answer is, quite clearly: it depends on your values.</pre><pre>As we create more and more advanced AI, it will start to depart from the realm of toy problems like Atari games, where “reward” is cleanly defined by how many points are won in the game,  and exist more and more in the real world. Autonomous vehicles, for example, have to make decisions with a somewhat more complex definition of reward. At first, reward might be tied to something like “getting safely to the destination&quot;. But if forced to choose between staying the course and hitting five pedestrians or swerving and hitting one, should the vehicle swerve? What if the one pedestrian is a child, or a gunman on the loose, or the next Einstein? How does that change the decision, and why? What if swerving also destroys a piece of valuable art? Suddenly we have a much more complex problem when we try to define the objective function, and the answers are not as simple.</pre><pre>In this series, we explored why it’s difficult to specify explicitly to a computer what a cat looks like - if asked how we know ourselves, the answer is, most simply, “intuition” - but we’ve explored <strong>machine vision</strong> approaches to teaching the machine to learn this intuition by itself. Similarly, in the domain of <strong>machine morality</strong>, it might be difficult to specify exactly how to evaluate the rightness or wrongness of one action vs. another, but perhaps it is possible for a machine to learn these values in some way. This is called the <strong>values learning problem</strong>, and it may be one of the most important technical problems humans will ever have to solve.</pre><pre>For more on this topic, see this synoptic post on the <a href="https://thinkingwires.com/posts/2017-07-05-risks.html">Risks of Artificial Intelligence</a>. And as you go forth into the world of making machine smarter and smarter, we&#39;d encourage you to keep in mind that AI progress is a double-edged sword, of particular keenness on both sides.</pre><h4>Enter your email below if you’d like to stay up-to-date with future content 💌</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f8c40%3Fas_embed%3Dtrue&amp;url=https%3A%2F%2Fupscri.be%2F8f8c40%2F&amp;image=https%3A%2F%2Fupscri.be%2Fmedia%2Fform.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href">https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href</a></iframe><h4>On Twitter? So are we. Feel free to keep in touch — <a href="https://twitter.com/v_maini">Vishal</a> and <a href="https://twitter.com/seriousssam">Samer</a> 🙌🏽</h4><ul><li><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12"><em>Part 1: Why Machine Learning Matters</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><em>Part 2.1: Supervised Learning</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d"><em>Part 2.2: Supervised Learning II</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930"><em>Part 2.3: Supervised Learning III</em></a><em> ✅</em></li><li><a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294"><em>Part 3: Unsupervised Learning</em></a> ✅</li><li><a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b"><em>Part 4: Neural Networks &amp; Deep Learning</em></a><em> </em>✅</li><li><strong><em>Part 5: Reinforcement Learning</em> ✅</strong></li><li><a href="https://medium.com/@v_maini/how-to-learn-machine-learning-24d53bb64aa1"><em>Appendix: The Best Machine Learning Resources</em></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6eacf258b265" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/reinforcement-learning-6eacf258b265">Machine Learning for Humans, Part 5: Reinforcement Learning</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning for Humans, Part 4: Neural Networks & Deep Learning]]></title>
            <link>https://medium.com/machine-learning-for-humans/neural-networks-deep-learning-cdad8aeae49b?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/cdad8aeae49b</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[neural-networks]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Sat, 19 Aug 2017 15:23:58 GMT</pubDate>
            <atom:updated>2018-05-28T18:08:11.802Z</atom:updated>
            <content:encoded><![CDATA[<h4>Where, why, and how deep neural networks work. Drawing inspiration from the brain. Convolutional neural networks (CNNs) and recurrent neural networks (RNNs). Real-world applications.</h4><pre>This series is available as a full-length e-book! <a href="https://www.dropbox.com/s/e38nil1dnl7481q/machine_learning.pdf?dl=0"><strong>Download here</strong></a>. Free for download, contributions appreciated (<a href="http://paypal.me/ml4h"><strong>paypal.me/ml4h</strong></a><strong>)</strong></pre><p>With deep learning, we’re still learning a function <em>f</em> to map input X to output Y with minimal loss on the test data, just as we’ve been doing all along. Recall our initial “problem statement” from <a href="https://medium.com/@v_maini/supervised-learning-740383a2feab">Part 2.1</a> on supervised learning:</p><pre><strong>Y = <em>f</em>(X) + ϵ</strong></pre><pre><strong>Training</strong>: machine learns <em>f </em>from labeled training data</pre><pre><strong>Testing</strong>: machine predicts Y from unlabeled testing data</pre><p>The real world is messy, so sometimes <em>f</em> is complicated. In natural language problems large vocabulary sizes mean lots of features. Vision problems involve lots of visual information about pixels. Playing games requires making a decision based on complex scenarios with many possible futures. The learning techniques we’ve covered so far do well when the data we’re working with is not insanely complex, but it’s not clear how they’d generalize to scenarios like these.</p><p>Deep learning is really good at learning <em>f</em>, particularly in situations where the data is complex. In fact, artificial neural networks are known as <strong>universal function approximators </strong>because they’re able to <a href="http://neuralnetworksanddeeplearning.com/chap4.html">learn any function</a>, no matter how wiggly, with just a single <strong>hidden layer.</strong></p><p>Let’s look at the problem of image classification. We take an image as an input, and output a class (e.g., dog, cat, car).</p><p>Graphically, a deep neural network solving image classification looks something like this:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/618/1*v-cD3i3sa1uztkQliinImQ.png" /><figcaption>Image from Jeff Clune’s 1-hour <a href="https://www.youtube.com/watch?v=3lp9eN5JE2A&amp;t=1631s">Deep Learning Overview</a> on YouTube</figcaption></figure><p>But really, this is a giant mathematical equation with millions of terms and lots of parameters. The input X is, say, a greyscale image represented by a <em>w</em>-by-<em>h </em>matrix of pixel brightnesses. The output Y is a vector of class probabilities. This means we have as an output the probability of each class being the correct label. If this neural net is working well, the highest probability should be for the correct class. And the layers in the middle are just doing a bunch of matrix multiplication by summing <strong>activations</strong> x <strong>weights</strong> with non-linear transformations (<strong>activation functions</strong>) after every hidden layer to enable the network to learn a <strong>non-linear function.</strong></p><p>Incredibly, you can use <strong>gradient descent </strong>in the <em>exact same way</em> that we did with linear regression in <a href="https://medium.com/@v_maini/supervised-learning-740383a2feab">Part 2.1</a> to train these parameters in a way that minimizes loss. So with a lot of examples and a lot of gradient descent, the model can learn how to classify images of animals correctly. And that, in a nutshell’s nutshell, is “deep learning”.</p><h3>Where deep learning does well, and some history</h3><p>Artificial neural networks have actually been around for a long time. Their application has been historically referred to as <strong>cybernetics </strong>(1940s-1960s), <strong>connectionism</strong> (1980s-1990s), and then came into vogue as <strong>deep learning </strong>circa 2006 when neural networks started getting, well, “deeper” (<a href="http://www.deeplearningbook.org/contents/intro.html">Goodfellow et al., 2016</a>). But only recently have we really started to scratch the surface of their full potential.</p><p>As described by Andrej Karpathy (Director of AI at Tesla, whom we tend to think of as the Shaman of Deep Learning), there are generally <em>“four separate factors that hold back AI:</em></p><ol><li><em>Compute (the obvious one: Moore’s Law, GPUs, ASICs),</em></li><li><em>Data (in a nice form, not just out there somewhere on the internet — e.g. ImageNet),</em></li><li><em>Algorithms (research and ideas, e.g. backprop, CNN, LSTM), and</em></li><li><em>Infrastructure (software under you — Linux, TCP/IP, Git, ROS, PR2, AWS, AMT, TensorFlow, etc.)” </em>(<a href="http://karpathy.github.io/2016/05/31/rl/">Karpathy, 2016</a>).</li></ol><p>In the past decade or so, the full potential of deep learning is finally being unlocked by advances in (1) and (2), which in turn has led to further breakthroughs in (3) and (4) — and so the cycle continues, with exponentially more humans rallying to the frontlines of deep learning research along the way (just think about what you’re doing <em>right now!</em>)</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Jf1NcBvaDH7fP0-QZp4XiA.png" /><figcaption>Illustration by <a href="https://blogs.nvidia.com/blog/2016/07/29/whats-difference-artificial-intelligence-machine-learning-deep-learning-ai/">NVIDIA</a>, a leading maker of graphics processing units (GPUs) which were originally built for for gaming but turned out to be well-suited to the type of <a href="http://www.nvidia.com/object/what-is-gpu-computing.html">parallel computing</a> required by deep neural networks</figcaption></figure><p>In the rest of this section, we’ll provide some background from biology and statistics to explain what happens inside neural nets, and then talk through some amazing applications of deep learning. Finally, we’ll link to a few resources so you can apply deep learning yourself, even sitting on the couch in your pajamas with a laptop, to quickly achieve greater-than-human-level performance on certain types of problems.</p><h3>Drawing inspiration from the brain (or is it just statistics?) — what happens inside neural nets</h3><h4>Neurons, feature learning, and layers of abstraction</h4><p>As you read these words you aren’t examining every letter of every word, or every pixel making up each letter, to derive the meaning of the words. You’re abstracting away from the details and grouping things into higher-level concepts: words, phrases, sentences, paragraphs.</p><pre>Yuor abiilty to exaimne hgiher-lveel fteaures is waht aollws yuo to unedrtsand waht is hpapening in tihs snetecne wthiout too mcuh troulbe (or myabe yuo sned too mnay dnruk txets).</pre><p>The same thing happens in vision, not just in humans but in animals’ visual systems generally.</p><p>Brains are made up of neurons which “fire” by emitting electrical signals to other neurons after being sufficiently “activated”. These neurons are malleable in terms of how much a signal from other neurons will add to the activation level of the neuron (vaguely speaking, the <em>weights</em> connecting neurons to each other end up being <em>trained </em>to make the neural connections more useful, just like the parameters in a linear regression can be trained to improve the mapping from input to output).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0UxcK0BoePVAeIdzeqEhzQ.png" /><figcaption>Side-by-side illustrations of biological and artificial neurons, via <a href="http://cs231n.stanford.edu/slides/2017/cs231n_2017_lecture4.pdf">Stanford’s CS231n</a>. This analogy can’t be taken too literally — biological neurons can do things that artificial neurons can’t, and vice versa — but it’s useful to understand the biological inspiration. See Wikipedia’s description of <a href="https://en.wikipedia.org/wiki/Artificial_neuron#Biological_models">biological vs. artificial neurons</a> for more detail.</figcaption></figure><p>Our biological networks are arranged in a hierarchical manner, so that certain neurons end up detecting not extremely <em>specific</em> features of the world around us, but rather more <em>abstract</em> features, i.e. patterns or groupings of more low-level features. For example, the <a href="https://en.wikipedia.org/wiki/Fusiform_face_area">fusiform face area</a> in the human visual system is specialized for facial recognition.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/942/1*ThI4Ew1o74EIaN8fX98ozw.png" /></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/611/1*0a1b64C8S7fI7XmM4tCGfw.png" /><figcaption>Top: Illustration of learning increasingly abstract features, via <a href="https://www.slideshare.net/NVIDIA/gpuaccelerated-deep-learning-for-cudnn-v2">NVIDIA</a>. Bottom: diagram of how an artificial neural network takes raw pixel inputs, develops intermediate “neurons” to detect higher-level features (e.g. presence of a nose), and combines the outputs of these to create a final output. Illustration from Neural Networks and Deep Learning (<a href="http://neuralnetworksanddeeplearning.com/chap1.html">Nielsen, 2017</a>).</figcaption></figure><p>This hierarchical structure exhibited by biological neural networks was discovered in the 1950s when researchers David Hubel and Torsten Wiesel were studying neurons in the visual cortex of cats. They were unable to observe neural activation after exposing the cat to a variety of stimuli: dark spots, light spots, hand-waving, and even pictures of women in magazines. But in their frustration, as they removed a slide from the projector at a diagonal angle, they noticed some neural activity! It turned out that diagonal edges at a very particular angle were causing certain neurons to be activated.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FIOHayh06LJ4%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DIOHayh06LJ4&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FIOHayh06LJ4%2Fhqdefault.jpg&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/a9eddb961e4bc9ca2d118b64525c2659/href">https://medium.com/media/a9eddb961e4bc9ca2d118b64525c2659/href</a></iframe><p>This makes sense evolutionarily since natural environments are generally noisy and random (imagine a grassy plain or a rocky terrain). So when a feline in the wild perceives an “edge”, i.e. a line that contrasts from its background, this might indicate that an object or creature is in the visual field. When a certain combination of edge neurons are activated, those activations will combine to yield a yet more abstract activation, and so on, until the final abstraction is a useful concept, like “bird” or “wolf”.</p><p>The idea behind a deep neural network is to mimic a similar structure with layers of artificial neurons.</p><h4>Why linear models don’t work</h4><p>To draw from Stanford’s excellent deep learning course, <a href="http://cs231n.stanford.edu/syllabus.html">CS231n: Convolutional Neural Networks and Visual Recognition</a>, imagine that we want to train a neural network to classify images with the correct one of the following labels: [&quot;plane&quot;, &quot;car&quot;, &quot;bird&quot;, &quot;cat&quot;, &quot;deer&quot;, &quot;dog&quot;, &quot;frog&quot;, &quot;horse&quot;, &quot;ship&quot;, &quot;truck&quot;].</p><p>One approach could be to construct a “template”, or average image, of each class of image using the training examples, and then use a nearest-neighbors algorithm during testing to measure the distance of each unclassified image’s pixel values, in aggregate, to each template. This approach involves no layers of abstraction. It’s a linear model that combines all the different orientations of each type of image into one averaged blur.</p><p>For instance, it would take all the cars — regardless of whether they’re facing left, right, center, and regardless of their color — and average them. The template then ends up looking rather vague and blurry.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/917/1*6o4FaHE9ehj_eN5HykKjSA.png" /><figcaption>Example drawn from Stanford’s <a href="http://cs231n.stanford.edu/syllabus.html">CS231n: Convolutional Neural Networks and Visual Recognition</a>, Lecture 2.</figcaption></figure><p>Notice that the horse template above appears to have two heads. This doesn’t really help us: we want to be able to detect right-facing horse <em>or</em> a left-facing horse separately, and then if <em>either one</em> of those features is detected, <em>then</em> we want to say we’re looking at a horse. This flexibility is provided by deep neural nets, as we will see in the next section.</p><p><strong>Deep neural networks approach the image classification problem using layers of abstraction</strong></p><p>To repeat what we explained earlier in this section: the input layer will take raw pixel brightnesses of an image. The final layer will be an output vector of <strong>class probabilities</strong> (i.e. the probability of the image being a “cat”, “car”, “horse”, etc.)</p><p>But instead of learning a simple linear model<strong> </strong>relating input to output, we’ll instead construct intermediate <strong>hidden layers</strong> of the network will learn increasingly abstract features, which enables us to not lose all the nuance in the complex data.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/960/1*Jyjubgdk41aFI41dT4FY0Q.png" /><figcaption>Source: <a href="https://www.analyticsvidhya.com/blog/2017/04/comparison-between-deep-learning-machine-learning/">Analytics Vidhya</a></figcaption></figure><p>Just as we described animal brains detecting abstract features, the artificial neurons in the hidden layers will learn to detect abstract concepts — whichever concepts are ultimately most useful for capturing the most information and minimizing loss in the accuracy of the network’s output (this is an instance of unsupervised learning happening within the network).</p><p>This comes at the cost of model interpretability, since as you add in more hidden layers the neurons start representing more and more abstract and ultimately unintelligible features — to the point that you may hear deep learning referred to as “black box optimization”, where you basically are just trying stuff somewhat at random and seeing what comes out, without <em>really</em> understanding what’s happening inside.</p><p>Linear regression is interpretable because you <em>decided </em>which features to include in the model. Deep neural networks are harder to interpret because the features are learned and aren’t explained anywhere in English. It’s all in the machine’s imagination.</p><h4>Some extensions and further concepts worth noting</h4><ul><li><strong>Deep learning software packages. </strong>You’ll rarely need to implement all the parts of neural networks from scratch because of existing libraries and tools that make deep learning implementations easier. There are many of these: TensorFlow, Caffe, Torch, Theano, and more.</li><li><strong>Convolutional neural networks (CNNs).</strong> CNNs are designed specifically for taking images as input, and are effective for computer vision tasks. They are also instrumental in deep reinforcement learning. CNNs are specifically inspired by the way animal visual cortices work, and they’re the focus of the deep learning course we’ve been referencing throughout this article, Stanford’s CS231n.</li><li><strong>Recurrent neural networks (RNNs). </strong>RNNs have a sense of built-in memory and are well-suited for language problems. They’re also important in reinforcement learning since they enable the agent to keep track of where things are and what happened historically even when those elements aren’t all visible at once. Christopher Olah wrote an <a href="http://colah.github.io/posts/2015-08-Understanding-LSTMs/">excellent walkthrough</a> of RNNs and LSTMs in the context of language problems.</li><li><strong>Deep reinforcement learning.</strong> This is one of the most exciting areas of deep learning research, at the heart of recent achievements like OpenAI defeating professional Dota 2 players and DeepMind’s AlphaGo surpassing humans in the game of Go. We’ll dive deeper in <a href="https://medium.com/@v_maini/machine-learning-for-humans-part-5-reinforcement-learning-6eacf258b265">Part 5</a>, but essentially the goal is to apply all of the techniques in this post to the problem of teaching an agent to maximize reward. This can be applied in any context that can be gamified — from actual games like Counter Strike or Pacman, to self-driving cars, to trading stocks, to (ultimately) real life and the real world.</li></ul><h4>Deep learning applications</h4><p>Deep learning is reshaping the world in virtually every domain. Here are a few examples of the incredible things that deep learning can do…</p><ul><li>Facebook trained a neural network augmented by short-term memory to intelligently answer questions about the plot of Lord of the Rings.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Qrt9WiiK5JMx7juBqEs7RA.png" /><figcaption>Research from <a href="https://www.facebook.com/FBAIResearch/posts/362517620591864">FAIR</a> (Facebook AI Research) applying deep neural networks augmented by separate short-term memory to intelligently answer questions about the LOTR storyline. This is the definition of epic.</figcaption></figure><ul><li>Self-driving cars rely on deep learning for visual tasks like understanding road signs, detecting lanes, and recognizing obstacles.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*tJylgT3R40F3qQORIBQgnw.jpeg" /><figcaption>Source: <a href="http://www.businessinsider.com/uber-driverless-car-in-pittsburgh-review-photos-2016-9/#to-try-the-cars-we-lined-up-at-ubers-advanced-technologies-center-in-the-strip-district-of-pittsburgh-a-small-neighborhood-on-the-allegheny-river-with-nearby-warehouses-the-atc-is-tucked-under-an-overpass-for-a-freight-train-keeping-it-secluded-1">Business Insider</a></figcaption></figure><ul><li>Deep learning can be used for fun stuff like art generation. A tool called <a href="https://github.com/jcjohnson/neural-style">neural style</a> can impressively mimic an artist’s style and use it to remix another image.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*wYmEJYUiFz0ugsGcmXn0QQ.png" /><figcaption>The style of Van Gogh’s <a href="https://en.wikipedia.org/wiki/The_Starry_Night">Starry Night</a> applied to a picture of Stanford’s campus, via Justin Johnson’s neural style implementation: <a href="https://github.com/jcjohnson/neural-style">https://github.com/jcjohnson/neural-style</a></figcaption></figure><p>Other noteworthy examples include:</p><ul><li>Predicting molecule bioactivity for <a href="https://arxiv.org/abs/1510.02855">drug discovery</a></li><li>Face and object recognition for photo and video tagging</li><li>Powering Google search results</li><li>Natural language understanding and generation, e.g. <a href="https://translate.google.com/">Google Translate</a></li><li>The Mars explorer robot Curiosity is <a href="https://www.theatlantic.com/technology/archive/2017/06/mars-curiosity-rover/531339/">autonomously selecting inspection-worthy soil targets</a> based on visual examination</li></ul><p>…and many, many, more.</p><h3>Now go do it!</h3><p>We haven’t gone into as much detail here on how neural networks are set up in practice because it’s much easier to understand the details by implementing them yourself. Here are some amazing hands-on resources for getting started.</p><ul><li>Play around with the architecture of neural networks to see how different configurations affect network performance with the Google’s <a href="http://playground.tensorflow.org/#activation=tanh&amp;batchSize=10&amp;dataset=circle&amp;regDataset=reg-plane&amp;learningRate=0.03&amp;regularizationRate=0&amp;noise=0&amp;networkShape=4,2&amp;seed=0.70277&amp;showTestData=false&amp;discretize=false&amp;percTrainData=50&amp;x=true&amp;y=true&amp;xTimesY=false&amp;xSquared=false&amp;ySquared=false&amp;cosX=false&amp;sinX=false&amp;cosY=false&amp;sinY=false&amp;collectStats=false&amp;problem=classification&amp;initZero=false&amp;hideText=false">Neural Network Playground</a>.</li><li>Get up-and-running quickly with this tutorial by Google: <a href="https://codelabs.developers.google.com/codelabs/cloud-tensorflow-mnist/">TensorFlow and deep learning, without a PhD</a>. Classify handwritten digits at &gt;99% accuracy, get familiar with TensorFlow, and learn deep learning concepts within 3 hours.</li><li>Then, work through at least the first few lectures of <a href="http://cs231n.stanford.edu/syllabus.html">Stanford’s CS231n</a> and the first assignment of building a two-layer neural network from scratch to really solidify the concepts covered in this article.</li></ul><h3>Further resources</h3><p><em>Deep learning is an expansive subject area. Accordingly, we’ve also compiled some of the best resources we’ve encountered on the topic, in case you’d like to go… deeper.</em></p><ul><li><a href="http://deeplearning.ai"><em>Deeplearning.ai</em></a><em>, Andrew Ng’s new deep learning course with a comprehensive syllabus on the subject</em></li><li><a href="http://cs231n.stanford.edu/syllabus.html"><em>CS231n: Convolutional Neural Networks for Visual Recognition</em></a><em>, Stanford’s deep learning course. One of the best treatments we’ve seen, with excellent lectures and illustrative problem sets</em></li><li><a href="http://neuralnetworksanddeeplearning.com/chap1.html"><em>Deep Learning &amp; Neural Networks</em></a><em> — accessible but rigorous</em></li><li><a href="http://www.deeplearningbook.org/"><em>Deep Learning Book</em></a><em> — foundational, more mathematical</em></li><li><a href="http://fast.ai"><em>Fast.ai</em></a><em> — less theoretical, much more applied and black-boxy</em></li><li><em>See Greg Brockman (CTO of OpenAI)’s answer to the question “What are the best ways to pick up Deep Learning skills as an engineer?” </em><a href="https://www.quora.com/What-are-the-best-ways-to-pick-up-Deep-Learning-skills-as-an-engineer/answer/Greg-Brockman?srid=2sq8"><em>on Quora</em></a></li></ul><h3>Next up: time to play some games!</h3><p>Last, but most certainly not least, is <a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265">Part 5: Reinforcement Learning</a>.</p><h4>Enter your email below if you’d like to stay up-to-date with future content 💌 💌</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f8c40%3Fas_embed%3Dtrue&amp;url=https%3A%2F%2Fupscri.be%2F8f8c40%2F&amp;image=https%3A%2F%2Fupscri.be%2Fmedia%2Fform.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href">https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href</a></iframe><h4>On Twitter? So are we. Feel free to keep in touch — <a href="https://twitter.com/v_maini">Vishal</a> and <a href="https://twitter.com/seriousssam">Samer</a> 🙌🏽</h4><p><em>More from</em><strong><em> Machine Learning for Humans </em></strong>🤖👶</p><ul><li><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12"><em>Part 1: Why Machine Learning Matters</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><em>Part 2.1: Supervised Learning</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d"><em>Part 2.2: Supervised Learning II</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930"><em>Part 2.3: Supervised Learning III</em></a> ✅</li><li><a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294"><em>Part 3: Unsupervised Learning</em></a> ✅</li><li><strong><em>Part 4: Neural Networks &amp; Deep Learning</em> </strong>✅</li><li><a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265"><em>Part 5: Reinforcement Learning</em></a></li><li><a href="https://medium.com/@v_maini/how-to-learn-machine-learning-24d53bb64aa1"><em>Appendix: The Best Machine Learning Resources</em></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=cdad8aeae49b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/neural-networks-deep-learning-cdad8aeae49b">Machine Learning for Humans, Part 4: Neural Networks &amp; Deep Learning</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning for Humans, Part 3: Unsupervised Learning]]></title>
            <link>https://medium.com/machine-learning-for-humans/unsupervised-learning-f45587588294?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/f45587588294</guid>
            <category><![CDATA[unsupervised-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Sat, 19 Aug 2017 15:23:15 GMT</pubDate>
            <atom:updated>2018-05-28T18:08:01.484Z</atom:updated>
            <content:encoded><![CDATA[<h4>Clustering and dimensionality reduction: k-means clustering, hierarchical clustering, principal component analysis (PCA), singular value decomposition (SVD)</h4><pre>This series is available as a full-length e-book! <a href="https://www.dropbox.com/s/e38nil1dnl7481q/machine_learning.pdf?dl=0"><strong>Download here</strong></a>. Free for download, contributions appreciated (<a href="http://paypal.me/ml4h"><strong>paypal.me/ml4h</strong></a><strong>)</strong></pre><p>How do you find the underlying structure of a dataset? How do you summarize it and group it most usefully? How do you effectively represent data in a compressed format? These are the goals of unsupervised learning, which is called “unsupervised” because you start with <strong>unlabeled data </strong>(there’s no Y).</p><p>The two unsupervised learning tasks we will explore are <strong>clustering</strong> the data into groups by similarity and <strong>reducing dimensionality</strong> to compress the data while maintaining its structure and usefulness.</p><pre>Examples of where unsupervised learning methods might be useful:</pre><pre>- An advertising platform segments the U.S. population into smaller groups with similar demographics and purchasing habits so that advertisers can reach their target market with relevant ads.</pre><pre>- Airbnb groups its housing listings into neighborhoods so that users can navigate listings more easily. </pre><pre>- A data science team reduces the number of dimensions in a large data set to simplify modeling and reduce file size.</pre><p>In contrast to supervised learning, it’s not always easy to come up with metrics for how well an unsupervised learning algorithm is doing. “Performance” is often subjective and domain-specific.</p><h3>Clustering</h3><p>An interesting example of clustering in the real world is marketing data provider Acxiom’s life stage clustering system, Personicx. This service segments U.S. households into 70 distinct clusters within 21 life stage groups that are used by advertisers when targeting Facebook ads, display ads, direct mail campaigns, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ldqddWP4gYgVhucG4Q883Q.png" /><figcaption>A selection of Personicx demographic clusters</figcaption></figure><p>Their <a href="http://c.ymcdn.com/sites/dema.site-ym.com/resource/resmgr/Member_Resources/Lifestage_Clustering.pdf">white paper</a> reveals that they used <strong>centroid clustering </strong>and <strong>principal component analysis</strong>, both of which are techniques covered in this section.</p><p>You can imagine how having access to these clusters is extremely useful for advertisers who want to (1) understand their existing customer base and (2) use their ad spend effectively by targeting potential new customers with relevant demographics, interests, and lifestyles.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*6J6ppCRAj8UaWhPLg_4B9A.png" /><figcaption>You can actually find out which cluster you personally would belong to by answering a few simple questions in Acxiom’s <a href="https://isapps.acxiom.com/personicx/personicx.aspx">“What’s My Cluster?” tool</a>.</figcaption></figure><p>Let’s walk through a couple of clustering methods to develop intuition for how this task can be performed.</p><h3>k-means clustering</h3><p><em>“And </em>k<em> rings were given to the race of Centroids, who above all else, desire power.”</em></p><p>The goal of clustering is to create groups of data points such that points in different clusters are dissimilar while points within a cluster are similar.</p><p>With <strong>k-means clustering</strong>, we want to cluster our data points into <em>k</em> groups. A larger <em>k </em>creates smaller groups with more granularity, a lower <em>k</em> means larger groups and less granularity.</p><p>The output of the algorithm would be a set of “labels” assigning each data point to one of the <em>k</em> groups. In k-means clustering, the way these groups are defined is by creating a <strong>centroid</strong> for each group. The centroids are like the heart of the cluster, they “capture” the points closest to them and add them to the cluster.</p><p>Think of these as the people who show up at a party and soon become the centers of attention because they’re so magnetic. If there’s just one of them, everyone will gather around; if there are lots, many smaller centers of activity will form.</p><pre>Here are the steps to k-means clustering:</pre><pre><strong>1. Define the <em>k</em> centroids</strong>. Initialize these at random (there are also fancier algorithms for initializing the centroids that end up converging more effectively).</pre><pre><strong>2. Find the closest centroid &amp; update cluster assignments. </strong>Assign each data point to one of the <em>k </em>clusters. Each data point is assigned to the nearest centroid’s cluster. Here, the measure of “nearness” is a hyperparameter — often Euclidean distance.</pre><pre><strong>3. Move the centroids to the center of their clusters. </strong>The new position of each centroid is calculated as the average position of all the points in its cluster.</pre><pre>Keep repeating steps 2 and 3 until the centroid stop moving a lot at each iteration (i.e., until the algorithm converges).</pre><p>That, in short, is how k-means clustering works! Check out this <a href="https://www.naftaliharris.com/blog/visualizing-k-means-clustering/">visualization</a> of the algorithm — read it like a comic book. Each point in the plane is colored according the centroid that it is closest to at each moment. You’ll notice that the centroids (the larger blue, red, and green circles) start randomly and then quickly adjust to capture their respective clusters.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*i8ErSXipZpjNWjk4." /></figure><p>Another real-life application of k-means clustering is classifying handwritten digits. Suppose we have images of the digits as a long vector of pixel brightnesses. Let’s say the images are black and white and are 64x64 pixels. Each pixel represents a dimension. So the world these images live in has 64x64=4,096 dimensions. In this 4,096-dimensional world, k-means clustering allows us to group the images that are close together and assume they represent the same digit, which can achieve <a href="http://ieeexplore.ieee.org/document/6755106/?reload=true">pretty good results</a> for digit recognition.</p><h3>Hierarchical clustering</h3><p><em>“Let’s make a million options become seven options. Or five. Or twenty? Meh, we can decide later.”</em></p><p>Hierarchical clustering is similar to regular clustering, except that you’re aiming to build a hierarchy of clusters. This can be useful when you want flexibility in how <em>many</em> clusters you ultimately want. For example, imagine grouping items on an online marketplace like Etsy or Amazon. On the homepage you’d want a few broad categories of items for simple navigation, but as you go into more specific shopping categories you’d want increasing levels of granularity, i.e. more distinct clusters of items.</p><p>In terms of outputs from the algorithm, in addition to cluster assignments you also build a nice tree that tells you about the hierarchies between the clusters. You can then pick the number of clusters you want from this tree.</p><pre>Here are the steps for hierarchical clustering:</pre><pre><strong>1. Start with N clusters, </strong>one for each data point.</pre><pre><strong>2. Merge the two clusters that are closest to each other</strong>.<strong> </strong>Now you have N-1 clusters.</pre><pre><strong>3. Recompute the distances between the clusters. </strong>There are several ways to do this (see this <a href="https://home.deib.polimi.it/matteucc/Clustering/tutorial_html/hierarchical.html">tutorial</a> for more details). One of them (called average-linkage clustering) is to consider the distance between two clusters to be the average distance between all their respective members.</pre><pre><strong>4. Repeat steps 2 and 3 until you get one cluster of N data points. </strong>You get a tree (also known as a <strong>dendrogram</strong>) like the one below.</pre><pre><strong>5. Pick a number of clusters and draw a horizontal line in the dendrogram.</strong> For example, if you want k=2 clusters, you should draw a horizontal line around “distance=20000.” You’ll get one cluster with data points 8, 9, 11, 16 and one cluster with the rest of the data points. In general, the number of clusters you get is the number of intersection points of your horizontal line with the vertical lines in the dendrogram.</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/914/1*DSrma_nUW4w9NLSccMAeYQ.png" /><figcaption>Source: <a href="https://www.solver.com/hierarchical-clustering-example">Solver.com</a>. For more detail on hierarchical clustering, you can check <a href="https://www.youtube.com/watch?v=OcoE7JlbXvY">this video</a> out.</figcaption></figure><h3>Dimensionality reduction</h3><p><em>“It is not the daily increase, but the daily decrease. Hack away at the unessential.” — Bruce Lee</em></p><p>Dimensionality reduction looks a lot like compression. This is about trying to reduce the complexity of the data while keeping as much of the relevant structure as possible. If you take a simple 128 x 128 x 3 pixels image (length x width x RGB value), that’s 49,152 dimensions of data. If you’re able to reduce the dimensionality of the space in which these images live without destroying too much of the meaningful content in the images, then you’ve done a good job at dimensionality reduction.</p><p>We’ll take a look at two common techniques in practice: <strong>principal component analysis </strong>and <strong>singular value decomposition</strong>.</p><h3>Principal component analysis (PCA)</h3><p>First, a little linear algebra refresher — let’s talk about <strong>spaces</strong> and <strong>bases</strong>.</p><p>You’re familiar with the coordinate plane with origin O(0,0) and <strong>basis vectors</strong> i(1,0) and j(0,1). It turns out you can choose a completely different basis and still have all the math work out. For example, you can keep O as the origin and choose the basis to vectors i’=(2,1) and j’=(1,2). If you have the patience for it, you’ll convince yourself that the point labeled (2,2) in the i’, j’ coordinate system is labeled (6, 6) in the i, j system.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/518/0*YtPe9Mj2x1oTlZyQ." /><figcaption>Plotted using Mathisfun’s “<a href="https://www.mathsisfun.com/data/cartesian-coordinates-interactive.html">Interactive Cartesian Coordinates</a>”</figcaption></figure><p>This means we can change the basis of a space. Now imagine much higher-dimensional space. Like, 50K dimensions. You can select a basis for that space, and then select only the 200 most significant vectors of that basis. These basis vectors are called <strong>principal components</strong>, and the subset you select constitute a new space that is smaller in dimensionality than the original space but maintains as much of the complexity of the data as possible.</p><p>To select the most significant principal components, we look at how much of the data’s variance they capture and order them by that metric.</p><p>Another way of thinking about this is that PCA remaps the space in which our data exists to make it more compressible. The transformed dimension is smaller than the original dimension.</p><p>By making use of the first several dimensions of the remapped space only, we can start gaining an understanding of the dataset’s organization. This is the promise of dimensionality reduction: reduce complexity (dimensionality in this case) while maintaining structure (variance). Here’s a <a href="http://bit.ly/2v0nxrA">fun paper</a> Samer wrote on using PCA (and diffusion mapping, another technique) to try to make sense of the Wikileaks cable release.</p><h3>Singular value decomposition (SVD)</h3><p>Let’s represent our data like a big A = <em>m</em> x <em>n</em> matrix. SVD is a computation that allows us to decompose that big matrix into a product of 3 smaller matrices (U=<em>m </em>x <em>r</em>, diagonal matrix Σ=<em>r </em>x <em>r</em>, and V=<em>r </em>x <em>n</em> where r is a small number).</p><p>Here’s a more visual illustration of that product to start with:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/675/1*uqm44L4wsZyg0ndgtrShdQ.png" /></figure><p>The values in the r*r diagonal matrix Σ are called singular values. What’s cool about them is that these singular values can be used to compress the original matrix. If you drop the smallest 20% of singular values and the associated columns in matrices U and V, you save quite a bit of space and still get a decent representation of the underlying matrix.</p><p>To examine what that means more precisely, let’s work with this image of a dog:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/550/0*q6djHRS0H8t1beye." /></figure><p>We’ll use the code written in Andrew Gibiansky’s <a href="http://andrew.gibiansky.com/blog/mathematics/cool-linear-algebra-singular-value-decomposition/">post</a> on SVD. First, we show that if we rank the singular values (the values of the matrix Σ) by magnitude, the first 50 singular values contain 85% of the magnitude of the whole matrix Σ.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*TOPSIsdW8kCzUTnl." /></figure><p>We can use this fact to discard the next 250 values of sigma (i.e., set them to 0) and just keep a “rank 50” version of the image of the dog. Here, we create a rank 200, 100, 50, 30, 20, 10, and 3 dog. Obviously, the picture is smaller, but let’s agree that the rank 30 dog is still good. Now let’s see how much compression we achieve with this dog. The original image matrix is 305*275 = 83,875 values. The rank 30 dog is 305*30+30+30*275=17,430 — almost 5 times fewer values with very little loss in image quality. The reason for the calculation above is that we also discard the parts of the matrix U and V that get multiplied by zeros when the operation UΣ’V is carried out (where Σ’ is the modified version of Σ that only has the first 30 values in it).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/267/0*rsIGV0O1n-shAoYm." /></figure><p>Unsupervised learning is often used to preprocess the data. Usually, that means compressing it in some meaning-preserving way like with PCA or SVD before feeding it to a deep neural net or another supervised learning algorithm.</p><h3>Onwards!</h3><p>Now that you’ve finished this section, you’ve earned an awful, horrible, never-to-be-mentioned-again joke about unsupervised learning. Here goes…</p><p><em>Person-in-joke-#1: Y would u ever need to use unsupervised tho?</em></p><p><em>Person-in-joke-#2: Y? there’s no Y.</em></p><p>Next up… <a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b">Part 4: Neural Networks &amp; Deep Learning</a>!</p><h3>Practice materials &amp; further reading</h3><h4>3a — k-means clustering</h4><p><em>Play around with this clustering </em><a href="https://www.naftaliharris.com/blog/visualizing-k-means-clustering/"><em>visualization</em></a><em> to build intuition for how the algorithm works. Then, take a look at this implementation of </em><a href="https://github.com/Datamine/MNIST-K-Means-Clustering"><em>k-means clustering for handwritten digits</em></a><em> and the associated tutorial.</em></p><h4>3b — SVD</h4><p><em>For a good reference on SVD, go no further than Andrew Gibiansky’s </em><a href="http://andrew.gibiansky.com/blog/mathematics/cool-linear-algebra-singular-value-decomposition/"><em>post</em></a><em>.</em></p><h4>Enter your email below if you’d like to stay up-to-date with future content 💌</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f8c40%3Fas_embed%3Dtrue&amp;url=https%3A%2F%2Fupscri.be%2F8f8c40%2F&amp;image=https%3A%2F%2Fupscri.be%2Fmedia%2Fform.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href">https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href</a></iframe><h4>On Twitter? So are we. Feel free to keep in touch — <a href="https://twitter.com/v_maini">Vishal</a> and <a href="https://twitter.com/seriousssam">Samer</a> 🙌🏽</h4><p><em>More from</em><strong><em> Machine Learning for Humans </em></strong>🤖👶</p><ul><li><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12"><em>Part 1: Why Machine Learning Matters</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><em>Part 2.1: Supervised Learning</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d"><em>Part 2.2: Supervised Learning II</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930"><em>Part 2.3: Supervised Learning III</em></a> ✅</li><li><strong><em>Part 3: Unsupervised Learning</em></strong> ✅</li><li><a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b"><em>Part 4: Neural Networks &amp; Deep Learning</em></a></li><li><a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265"><em>Part 5: Reinforcement Learning</em></a></li><li><a href="https://medium.com/@v_maini/how-to-learn-machine-learning-24d53bb64aa1"><em>Appendix: The Best Machine Learning Resources</em></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f45587588294" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/unsupervised-learning-f45587588294">Machine Learning for Humans, Part 3: Unsupervised Learning</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning for Humans, Part 2.3: Supervised Learning III]]></title>
            <link>https://medium.com/machine-learning-for-humans/supervised-learning-3-b1551b9c4930?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/b1551b9c4930</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Sat, 19 Aug 2017 15:22:15 GMT</pubDate>
            <atom:updated>2018-05-28T18:07:48.836Z</atom:updated>
            <content:encoded><![CDATA[<h4>Non-parametric models: k-nearest neighbors, decision trees, and random forests. Introducing cross-validation, hyperparameter tuning, and ensemble models.</h4><pre>This series is available as a full-length e-book! <a href="https://www.dropbox.com/s/e38nil1dnl7481q/machine_learning.pdf?dl=0"><strong>Download here</strong></a>. Free for download, contributions appreciated (<a href="http://paypal.me/ml4h"><strong>paypal.me/ml4h</strong></a><strong>)</strong></pre><h4>Non-parametric learners.</h4><p><em>Things are about to get a little… wiggly.</em></p><p>In contrast to the methods we’ve covered so far — linear regression, logistic regression, and SVMs where the form of the model was pre-defined — <strong>non-parametric learners </strong>do not have a model structure specified <em>a priori. </em>We don’t speculate about the form of the function <em>f </em>that we’re trying to learn <em>before</em> training the model, as we did previously with linear regression. Instead, the model structure is <em>purely determined from the data</em>.</p><p>These models are more flexible to the shape of the training data, but this sometimes comes at the cost of interpretability. This will make more sense soon. Let’s jump in.</p><h3>k-nearest neighbors (k-NN)</h3><p><em>“You are the average of your k closest friends.”</em></p><p>k-NN seems almost too simple to be a machine learning algorithm. The idea is to label a test data point <em>x</em> by finding the <strong>mean</strong> (or <strong>mode</strong>) of the <em>k</em> closest data points’ labels.</p><p>Take a look at the image below. Let’s say you want to figure out whether Mysterious Green Circle is a Red Triangle or a Blue Square. What do you do?</p><p>You could try to come up with a fancy equation that looks at where Green Circle lies on the coordinate plane below and makes a prediction accordingly. Or, you could just look its three <strong>nearest neighbors</strong>, and guess that Green Circle is probably a Red Triangle. You could also expand the circle further and look at the five nearest neighbors, and make a prediction that way (3/5 of its five nearest neighbors are Blue Squares, so we’d guess that Mysterious Green Circle is a Blue Square when k=5).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/440/0*chgrJFN_Y5S0QksO." /><figcaption><strong><em>k-NN illustration with k=1, 3, and 5.</em></strong><em> To classify </em>the <em>Mysterious Green Circle (x) above, look at its single nearest neighbor, a “Red Triangle”. So, we’d guess that </em>ŷ<em> = “Red Triangle”. With k=3, look at the 3 nearest neighbors: the mode of these is again “Red Triangle” so </em>ŷ<em>= “Red Triangle”. With k=5, we take the mode of the 5 nearest neighbors instead. Now, notice that </em>ŷ<em> becomes “Blue Square”. </em>Image from <a href="https://en.wikipedia.org/wiki/K-nearest_neighbors_algorithm">Wikipedia</a>.</figcaption></figure><p>That’s it. That’s <strong>k-nearest neighbors</strong>. You look at the <em>k</em> closest data points and take the average of their values if variables are continuous (like housing prices), or the mode if they’re categorical (like cat vs. dog).</p><p>If you wanted to guess unknown house prices, you could just take the average of some number of geographically nearby houses, and you’d end up with some pretty nice guesses. These might even outperform a parametric regression model built by some economist that estimates model coefficients for # of beds/baths, nearby schools, distance to public transport, etc.</p><pre>How to use k-NN to predict housing prices:</pre><pre><strong>1) Store the training data, </strong>a matrix X of features like zip code, neighborhood, # of bedrooms, square feet, distance from public transport, etc., and a matrix Y of corresponding sale prices.</pre><pre><strong>2) Sort the houses in your training data set by</strong> <strong>similarity</strong> to the house in question, based on the features in X. We’ll define “similarity” below.</pre><pre><strong>3) Take the mean of the <em>k</em> closest houses. </strong>That is your guess at the sale price (i.e. ŷ)</pre><p>The fact that k-NN doesn’t require a pre-defined parametric function f(X) relating Y to X makes it well-suited for situations where the relationship is too complex to be expressed with a simple linear model.</p><h4><strong>Distance metrics: defining and calculating “nearness”</strong></h4><p>How do you calculate distance from the data point in question when finding the “nearest neighbors”? How do you mathematically determine which of the Blue Squares and Red Triangles in the example above are closest to Green Circle, especially if you can’t just draw a nice 2D graph and eyeball it?</p><p>The most straightforward measure is <strong>Euclidean distance</strong> (a straight line, “as the crow flies”). Another is Manhattan distance, like walking city blocks. You could imagine that Manhattan distance is more useful in a model involving fare calculation for Uber drivers, for example.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/400/0*M00nbLfTUSsmSoEM." /><figcaption>Green line = Euclidean distance. Blue line = Manhattan distance. Source: <a href="https://en.wikipedia.org/wiki/Metric_(mathematics)">Wikipedia</a></figcaption></figure><p>Remember the Pythagorean theorem for finding the length of the hypotenuse of a right triangle?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/105/1*MY-JAQc6a1II6ekwA7lklg.png" /><figcaption>c = length of hypotenuse (green line above). a and b = length of the other sides, at a right angle (red lines above).</figcaption></figure><p>Solving in terms of <em>c</em>, we find the length of the hypotenuse by taking the square root of the sum of squared lengths of <em>a</em> and <em>b</em>, where <em>a </em>and <em>b</em> are <strong>orthogonal</strong> sides of the triangle (i.e. they are at a 90-degree angle from one another, going in perpendicular directions in space).</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/115/1*hYUDQR79ZFdkE-YbH8vneg.png" /></figure><p>This idea of finding the length of the hypotenuse given vectors in two orthogonal directions generalizes to many dimensions, and this is how we derive the formula for Euclidean distance <strong>d(p,q) </strong>between points <em>p</em> and <em>q</em> in <em>n</em>-dimensional space:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/523/1*tiEVrfP_qr7-5Tsih7L0AA.png" /><figcaption>Formula for Euclidean distance, derived from the Pythagorean theorem.</figcaption></figure><p>With this formula, you can calculate the nearness of all the training data points to the data point you’re trying to label, and take the mean/mode of the <em>k</em> nearest neighbors to make your prediction.</p><p>Typically you won’t need to calculate any distance metrics by hand — a quick Google search reveals pre-built functions in NumPy or SciPy that will do this for you, e.g.euclidean_dist = numpy.linalg.norm(p-q)— but it’s fun to see how geometry concepts from eighth grade end up being helpful for building ML models today!</p><h4><strong>Choosing k: tuning hyperparameters with cross-validation</strong></h4><p>To decide which value of <em>k </em>to use, you can test different k-NN models using different values of <em>k</em> with <strong>cross-validation:</strong></p><ol><li>Split your training data into segments, and train your model on all but one of the segments; use the held-out segment as the “test” data.</li><li>See how your model performs by comparing your model’s predictions (ŷ) to the actual values of the test data (y).</li><li>Pick whichever yields the lowest error, on average, across all iterations.</li></ol><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*djxKabpzFuYctka8." /><figcaption>Cross-validation illustrated. The number of splits and iterations can be varied.</figcaption></figure><p><strong>Higher <em>k </em>prevents overfitting</strong></p><p>Higher values of <em>k</em> help address overfitting, but if the value of <em>k</em> is too high your model will be very <strong>biased</strong> and <strong>inflexible</strong>. To take an extreme example: if <em>k</em> = N (the total number of data points), the model would just dumbly blanket-classify all the test data as the mean or mode of the training data.</p><p>If the <em>single most common</em> <em>animal</em> in a data set of animals is a Scottish Fold kitten, k-NN with <em>k</em> set to N (the # of training observations) would then predict that <em>every other animal in the world is also a Scottish Fold kitten. </em>Which, in Vishal’s opinion, would be awesome. Samer disagrees.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/294/1*XcDqlYz7CE4f8hHUyR798Q.gif" /><figcaption>Completely gratuitous Scottish Fold .gif. We’ll call it a study break. 😊</figcaption></figure><h4>Where to use k-NN in the real world</h4><p>Some examples of where you can use k-NN:</p><ul><li><strong>Classification: fraud detection.</strong> The model can update virtually instantly with new training examples since you’re just storing more data points, which allows quick adaptation to new methods of fraud.</li><li><strong>Regression: predicting housing prices</strong>. In housing price prediction, literally being a “near neighbor” is actually a good indicator of being similar in price. k-NN is useful in domains where physical proximity matters.</li><li><strong>Imputing missing training data. </strong>If one of the columns in your .csv has lots of missing values, you can <strong>impute</strong> the data by taking the mean or mode. k-NN could give you a somewhat more accurate guess at each missing value.</li></ul><h3>Decision trees, random forests</h3><p>Making a good decision tree is like playing a game of “20 questions”.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/965/0*pZ0pgQT3i-CFPyaf." /><figcaption>The decision tree on the right describes survival patterns on the Titanic.</figcaption></figure><p>The first <strong>split</strong> at the <strong>root</strong> of a decision tree should be like the first question you should ask in 20 questions: you want to separate the data as cleanly as possible, thereby maximizing <strong>information gain</strong> from that split.</p><p>If your friend says <em>“I’m thinking of a noun, ask me up to 20 yes/no questions to guess what it is”</em> and your first question is <em>“is it a potato?”</em>, then you’re a dumbass, because they’re going to say no and you gained almost no information. Unless you happen to know your friend thinks about potatoes all the time, or is thinking about one right now. Then you did a great job.</p><p>Instead, a question like <em>“is it an object?”</em> might make more sense.</p><p>This is kind of like how hospitals triage patients or approach differential diagnoses. They ask a few questions up front and check some basic vitals to determine if you’re going to die imminently or something. They don’t start by doing a biopsy to check if you have pancreatic cancer as soon as you walk in the door.</p><p>There are ways to quantify information gain so that you can essentially evaluate every possible split of the training data and <em>maximize information gain for every split. </em>This way you can predict every label or value as efficiently as possible.</p><p>Now, let’s look at a particular data set and talk about how we choose splits.</p><h4><strong>The Titanic dataset</strong></h4><p>Kaggle has a Titanic <a href="https://www.kaggle.com/c/titanic">dataset </a>that is used for a lot of machine learning intros. When the titanic sunk, 1,502 out of 2,224 passengers and crew were killed. Even though there was some luck involved, women, children, and the upper-class were more likely to survive. If you look back at the decision tree above, you’ll see that it somewhat reflects this variability across gender, age, and class.</p><h4><strong>Choosing splits in a decision tree</strong></h4><p>Entropy is the amount of disorder in a set (measured by <a href="https://en.wikipedia.org/wiki/Gini_coefficient"><strong>Gini index</strong></a> or <a href="https://en.wikipedia.org/wiki/Cross_entropy"><strong>cross-entropy</strong></a>). If the values are really mixed, there’s lots of entropy; if you can cleanly split values, there’s no entropy. For every split at a parent node, you want the child nodes to be as pure as possible — minimize entropy. For example, in the Titanic, gender is a big determinant of survival, so it makes sense for this feature to be used in the first split as it’s the one that leads to the most information gain.</p><p>Let’s take a look at our Titanic variables:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*s-rhRHAV4h3eHzHG44DbFQ.png" /><figcaption>Source: <a href="https://www.kaggle.com/c/titanic/data">Kaggle</a></figcaption></figure><p>We build a tree by picking one of these variables and splitting the dataset according to it.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/592/0*f-3Buln8oa_xroHP." /></figure><p>The first split separates our dataset into men and women. Then, the women branch gets split again in age (the split that minimizes entropy). Similarly, the men branch gets split by class. By following the tree for a new passenger, you can use the tree to make a guess at whether they died.</p><p>The Titanic example is solving a classification problem (“survive” or “die”). If we were using decision trees for regression — say, to predict housing prices — we would create splits on the most important features that determine housing prices. How many square feet: more than or less than ___? How many bedrooms &amp; bathrooms: more than or less than ___?</p><p>Then, during testing, you would run a specific house through all the splits and take the average of all the housing prices in the final <strong>leaf node</strong> (bottom-most node) where the house ends up as your prediction for the sale price.</p><p>There are a few hyperparameters you can tune with decision trees models, including max_depth and max_leaf_nodes. See the <a href="http://scikit-learn.org/stable/modules/tree.html">scikit-learn module</a> on decision trees for advice on defining these parameters.</p><p>Decision trees are effective because they are easy to read, powerful even with messy data, and computationally cheap to deploy once after training. Decision trees are also good for handling mixed data (numerical or categorical).</p><p>That said, decision trees are computationally expensive to train, carry a big risk of overfitting, and tend to find local optima because they can’t go back after they have made a split. To address these weaknesses, we turn to a method that illustrates the power of combining many decision trees into one model.</p><h3>Random forest: an ensemble of decision trees</h3><p>A model comprised of many models is called an <strong>ensemble model</strong>, and this is usually a winning strategy.</p><p>A single decision tree can make a lot of wrong calls because it has very black-and-white judgments. A <strong>random forest</strong> is a meta-estimator that aggregates many decision trees, with some helpful modifications:</p><ol><li>The number of features that can be split on at each node is limited to some percentage of the total (this is a hyperparameter you can choose — see <a href="http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html">scikit-learn documentation</a> for details). This ensures that the ensemble model does not rely too heavily on any individual feature, and makes fair use of all potentially predictive features.</li><li>Each tree draws a random sample from the original data set when generating its splits, adding a further element of randomness that prevents overfitting.</li></ol><p>These modifications also prevent the trees from being too highly correlated. Without #1 and #2 above, every tree would be identical, since recursive binary splitting is deterministic.</p><p>To illustrate, see these nine decision tree classifiers below.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*Rin05YSBfRZBlCxI." /><figcaption>Source: <a href="http://xenon.stanford.edu/~jianzh/ml/">http://xenon.stanford.edu/~jianzh/ml/</a></figcaption></figure><p>These decision tree classifiers can be aggregated into a random forest ensemble which combines their input. Think of the horizontal and vertical axes of each decision tree output as features x1 and x2. At certain values of each feature, the decision tree outputs a classification of “blue”, “green”, “red”, etc.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/800/0*bZ2NR-JYpcYhb3WJ." /><figcaption>Source: <a href="http://xenon.stanford.edu/~jianzh/ml/">http://xenon.stanford.edu/~jianzh/ml/</a></figcaption></figure><p>These results are aggregated, through modal votes or averaging, into a single ensemble model that ends up outperforming any individual decision tree’s output.</p><p>Random forests are an excellent starting point for the modeling process, since they tend to have strong performance with a high tolerance for less-cleaned data and can be useful for figuring out which features actually matter among many features.</p><p>There are many other clever ensemble models that combine decision trees and yield excellent performance — check out <a href="http://xgboost.readthedocs.io/en/latest/model.html">XGBoost</a> (Extreme Gradient Boosting) as an example.</p><h3>And with that, we conclude our study of supervised learning!</h3><p>Nice work. In this section we’ve covered:</p><ul><li>Two <strong>non-parametric</strong> supervised learning algorithms: <strong>k-NN</strong> and <strong>decision trees</strong></li><li>Measures of <strong>distance</strong> and <strong>information gain</strong></li><li><strong>Random forests, </strong>which are an example of an <strong>ensemble model</strong></li><li><strong>Cross-validation</strong> and <strong>hyperparameter tuning</strong></li></ul><p>Hopefully, you now have some solid intuitions for how we learn <em>f</em> given a training data set and use this to make predictions with the test data.</p><p>Next, we’ll talk about how to approach problems where we don’t have any labeled training data to work with, in <a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294">Part 3: Unsupervised Learning</a>.</p><h3>Practice materials &amp; further reading</h3><h4>2.3a — Implementing k-NN</h4><p><em>Try this </em><a href="http://machinelearningmastery.com/tutorial-to-implement-k-nearest-neighbors-in-python-from-scratch/"><em>walkthrough</em></a><em> for implementing k-NN from scratch in Python. You may also want to take a look at the </em><a href="http://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html"><em>scikit-learn</em></a><em> documentation to get a sense of how pre-built implementations work.</em></p><h4>2.3b — Decision trees</h4><p><em>Try the decision trees lab in Chapter 8 of </em><a href="http://www-bcf.usc.edu/~gareth/ISL/"><em>An Introduction to Statistical Learning</em></a><em>. You can also play with the </em><a href="https://www.kaggle.com/c/titanic/data"><em>Titanic</em></a><em> dataset, and check out this </em><a href="https://www.analyticsvidhya.com/blog/2016/04/complete-tutorial-tree-based-modeling-scratch-in-python/"><em>tutorial</em></a><em> which covers the same concepts as above with accompanying code. Here is the </em><a href="http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html"><em>scikit-learn implementation</em></a><em> of random forest for out-of-the-box use on data sets.</em></p><h4>Enter your email below if you’d like to stay up-to-date with future content 💌</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f8c40%3Fas_embed%3Dtrue&amp;url=https%3A%2F%2Fupscri.be%2F8f8c40%2F&amp;image=https%3A%2F%2Fupscri.be%2Fmedia%2Fform.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href">https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href</a></iframe><h4>On Twitter? So are we. Feel free to keep in touch — <a href="https://twitter.com/v_maini">Vishal</a> and <a href="https://twitter.com/seriousssam">Samer</a> 🙌🏽</h4><p><em>More from</em><strong><em> Machine Learning for Humans </em></strong>🤖👶</p><ul><li><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12"><em>Part 1: Why Machine Learning Matters</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><em>Part 2.1: Supervised Learning</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d"><em>Part 2.2: Supervised Learning II</em></a> ✅</li><li><strong><em>Part 2.3: Supervised Learning III</em> ✅</strong></li><li><a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294"><em>Part 3: Unsupervised Learning</em></a></li><li><a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b"><em>Part 4: Neural Networks &amp; Deep Learning</em></a></li><li><a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265"><em>Part 5: Reinforcement Learning</em></a></li><li><a href="https://medium.com/@v_maini/how-to-learn-machine-learning-24d53bb64aa1"><em>Appendix: The Best Machine Learning Resources</em></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=b1551b9c4930" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/supervised-learning-3-b1551b9c4930">Machine Learning for Humans, Part 2.3: Supervised Learning III</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning for Humans, Part 2.2: Supervised Learning II]]></title>
            <link>https://medium.com/machine-learning-for-humans/supervised-learning-2-5c1c23f3560d?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/5c1c23f3560d</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[statistics]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Sat, 19 Aug 2017 15:20:17 GMT</pubDate>
            <atom:updated>2018-05-28T18:07:35.238Z</atom:updated>
            <content:encoded><![CDATA[<h4>Classification with logistic regression and support vector machines (SVMs).</h4><pre>This series is available as a full-length e-book! <a href="https://www.dropbox.com/s/e38nil1dnl7481q/machine_learning.pdf?dl=0"><strong>Download here</strong></a>. Free for download, contributions appreciated (<a href="http://paypal.me/ml4h"><strong>paypal.me/ml4h</strong></a><strong>)</strong></pre><h3>Classification: predicting a label</h3><p><em>Is this email spam or not? Is that borrower going to repay their loan? Will those users click on the ad or not? Who is that person in your Facebook picture?</em></p><p>Classification<strong> </strong>predicts a<strong> discrete target label Y. </strong>Classification is the problem of assigning new observations to the <strong>class</strong> to which they most likely belong, based on a classification model built from labeled training data.</p><p>The accuracy of your classifications will depend on the effectiveness of the algorithm you choose, how you apply it, and how much useful training data you have.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*m-5HaOh9R0O775rw6JWuQw.png" /></figure><h3>Logistic regression: 0 or 1?</h3><p>Logistic regression is a method of <strong>classification</strong>: the model outputs the probability of a <strong>categorical</strong> target variable Y belonging to a certain class.</p><pre>A good example of classification is determining whether a loan application is fraudulent.</pre><pre>Ultimately, the lender wants to know whether they should give the borrower a loan or not, and they have some tolerance for risk that the application is in fact fraudulent. In this case, the goal of logistic regression is to calculate the probability (between 0% and 100%) that the application is fraud. With these probabilities, we can set some threshold above which we’re willing to lend to the borrower, and below which we deny their loan application or flag the application for further review.</pre><p>Though logistic regression is often used for <strong>binary classification </strong>where there are two classes, keep in mind that classification can performed with any number of categories (e.g. when assigning handwritten digits a label between 0 and 9, or using facial recognition to detect which friends are in a Facebook picture).</p><p><strong>Can I just use ordinary least squares?</strong></p><p>Nope. If you trained a linear regression model on a bunch of examples where Y = 0 or 1, you might end up predicting some probabilities that are less than 0 or greater than 1, which doesn’t make sense. Instead, we’ll use a logistic regression model (or <strong>logit model</strong>) which was designed for assigning a probability between 0% and 100% that Y belongs to a certain class.</p><p><strong>How does the math work?</strong></p><p><em>Note: the math in this section is interesting but might be on the more technical side. Feel free to skim through it if you’re more interested in the high-level concepts.</em></p><p>The logit model is a modification of linear regression that makes sure to output a probability between 0 and 1 by applying the <strong>sigmoid function,</strong> which, when graphed, looks like the characteristic S-shaped curve that you’ll see a bit later.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/135/1*AkcGCgBljPjRMaKFB4eftA.png" /><figcaption>Sigmoid function, which squashes values between 0 and 1.</figcaption></figure><p>Recall the original form of our simple linear regression model, which we’ll now call g(x) since we’re going to use it within a compound function:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/160/1*0zBUxDoyVcorgyE3VLTG5w.png" /></figure><p>Now, to solve this issue of getting model outputs less than 0 or greater than 1, we’re going to define a <em>new</em> function F(g(x)) that transforms g(x) by squashing the output of linear regression to a value in the [0,1] range. Can you think of a function that does this?</p><p>Are you thinking of the sigmoid function? Bam! Presto! You’re correct.</p><p>So we plug g(x) into the sigmoid function above, resulting in a function of our original function (yes, things are getting meta) that outputs a probability between 0 and 1:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/548/1*moFHMRLr7Ozvxmw7TrgQcg.png" /><figcaption>In other words, we’re calculating the probability that the training example belongs to a certain class: P(Y=1).</figcaption></figure><p>Here we’ve isolated <em>p, </em>the probability that Y=1, on the left side of the equation. If we want to solve for a nice clean β0 + β1x + ϵ on the right side so we can straightforwardly interpret the beta coefficients we’re going to learn, we’d instead end up with the <strong>log-odds ratio</strong>, or <strong>logit</strong>, on the left side — hence the name “logit model”:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/851/1*aetAOm7gZEhnAfJzI4gl9A.png" /></figure><p>The log-odds ratio is simply the natural log of the <strong>odds ratio, p/(1-p), </strong>which crops up in everyday conversations:</p><blockquote>“Yo, what do you think are the <strong>odds</strong> that Tyrion Lannister dies in this season of Game of Thrones?”</blockquote><blockquote>“Hmm. It’s definitely 2x more likely to happen than not. <strong>2-to-1 odds. </strong>Sure, he might seem too important to be killed, but we all saw what they did to Ned Stark…”</blockquote><figure><img alt="" src="https://cdn-images-1.medium.com/max/853/1*4lHjWIpVYbRagEI95GX7qg.png" /><figcaption>Note that in the logit model, <em>β1</em> now represents the <strong>rate of change in the log-odds ratio </strong>as X changes. In other words, it’s the “slope of log-odds”, not the “slope of the probability”.</figcaption></figure><p>Log-odds might be slightly unintuitive but it’s worth understanding since it will come up again when you’re interpreting the output of neural networks performing classification tasks.</p><p><strong>Using the output of a logistic regression model to make decisions</strong></p><p>The output of the logistic regression model from above looks like an S-curve showing P(Y=1) based on the value of X:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/408/0*xBLfyr2KkNXajfQp." /><figcaption>Source: <a href="https://en.wikipedia.org/wiki/Logistic_regression">Wikipedia</a></figcaption></figure><p>To predict the Y label — spam/not spam, cancer/not cancer, fraud/not fraud, etc. — you have to set a probability cutoff, or <strong>threshold, </strong>for a positive result. For example: <em>“If our model thinks the probability of this email being spam is higher than 70%, label it spam. Otherwise, don’t.”</em></p><p>The threshold depends on your tolerance for <strong>false positives</strong> vs. <strong>false negatives</strong>. If you’re diagnosing cancer, you’d have a very low tolerance for false negatives, because even if there’s a very small chance the patient has cancer, you’d want to run further tests to make sure. So you’d set a very low threshold for a positive result.</p><p>In the case of fraudulent loan applications, on the other hand, the tolerance for false positives might be higher, particularly for smaller loans, since further vetting is costly and a small loan may not be worth the additional operational costs and friction for non-fraudulent applicants who are flagged for further processing.</p><h4>Minimizing loss with logistic regression</h4><p>As in the case of linear regression, we use gradient descent to learn the beta parameters that minimize loss.</p><p>In logistic regression, the cost function is basically a measure of how often you predicted 1 when the true answer was 0, or vice versa. Below is a <strong>regularized cost function</strong> just like the one we went over for linear regression.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/938/1*Tn4B-RYmcb45YsEiZM5Ktg.png" /></figure><p>Don’t panic when you see a long equation like this! Break it into chunks and think about what’s going on in each part conceptually. Then the specifics will start to make sense.</p><p>The first chunk is the <strong>data loss</strong>, i.e. how much discrepancy there is between the model’s predictions and reality. The second chunk is the <strong>regularization loss</strong>, i.e. how much we penalize the model for having large parameters that heavily weight certain features (remember, this prevents overfitting).</p><p>We’ll minimize this cost function with gradient descent, as above, and <em>voilà</em>! we’ve built a logistic regression model to make class predictions as accurately as possible.</p><h3>Support vector machines (SVMs)</h3><p><em>“We’re in a room full of marbles again. Why are we always in a room full of marbles? I could’ve sworn we already lost them.”</em></p><p>SVM is the last parametric model we’ll cover. It typically solves the same problem as logistic regression — classification with two classes — and yields similar performance. It’s worth understanding because the algorithm is geometrically motivated in nature, rather than being driven by probabilistic thinking.</p><p>A few examples of the problems SVMs can solve:</p><ul><li>Is this an image of a cat or a dog?</li><li>Is this review positive or negative?</li><li>Are the dots in the 2D plane red or blue?</li></ul><p>We’ll use the third example to illustrate how SVMs work. Problems like these are called <strong>toy problems </strong>because they’re not real — but nothing is real, so it’s fine.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/0*8iRUPIB6XjkOSihP." /></figure><p>In this example, we have points in a 2D space that are either red or blue, and we’d like to cleanly separate the two.</p><p>The training set is plotted the graph above. We would like to classify new, unclassified points in this plane. To do this, SVMs use a separating line (or, in more than two dimensions, a multi-dimensional <strong>hyperplane</strong>) to split the space into a red zone and a blue zone. You can already imagine how a separating line might look in the graph above.</p><p>How, specifically, do we choose where to draw the line?</p><p>Below are two examples of such a line:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/0*_J_T8BQq8g46GJ6n." /><figcaption>These charts were made with Microsoft Paint, which was <a href="https://www.theguardian.com/technology/2017/jul/24/microsoft-paint-kill-off-after-32-years-graphics-editing-program">deprecated a few weeks ago</a> after 32 wonderful years. RIP Paint :(</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/989/0*uqvkhDvtLXy9lZG6." /></figure><p>Hopefully, you share the intuition that the first line is superior. The distance to the nearest point on either side of the line is called the <strong>margin</strong>, and SVM tries to <strong>maximize the margin</strong>. You can think about it like a safety space: the bigger that space, the less likely that noisy points get misclassified.</p><p>Based on this short explanation, a few big questions come up.</p><p><strong>1. How does the math behind this work?</strong></p><p>We want to find the optimal hyperplane (a line, in our 2D example). This hyperplane needs to (1) separate the data cleanly, with blue points on one side of the line and red points on the other side, and (2) maximize the margin. This is an <strong>optimization</strong> problem. The solution has to respect constraint (1) while maximizing the margin as is required in (2).</p><p>The human version of solving this problem would be to take a ruler and keep trying different lines separating all the points until you get the one that maximizes the margin.</p><p>It turns out there’s a clean mathematical way to do this maximization, but the specifics are beyond our scope. To explore it further, here’s a <a href="https://www.youtube.com/watch?v=_PwhiWxHK8o">video </a>lecture that shows how it works using <a href="https://en.wikipedia.org/wiki/Lagrange_multiplier">Lagrangian Optimization</a>.</p><p>The solution hyperplane you end up with is defined in relation to its position with respect to certain x_i’s, which are called the <strong>support vectors</strong>, and they’re usually the ones closest to the hyperplane.</p><p><strong>2. What happens if you can’t separate the data cleanly?</strong></p><p>There are two methods for dealing with this problem.</p><p><em>2.1. Soften the definition of “separate”.</em></p><p>We allow a few mistakes, meaning we allow some blue points in the red zone or some red points in the blue zone. We do that by adding a cost C for misclassified examples in our loss function. Basically, we say it’s acceptable but costly to misclassify a point.</p><p><em>2.2. Throw the data into higher dimensions.</em></p><p>We can create nonlinear classifiers by increasing the number of dimensions, i.e. include x², x³, even cos(x), etc. Suddenly, you have boundaries that can look more squiggly when we bring them back to the lower dimensional representation.</p><p>Intuitively, this is like having red and blue marbles lying on the ground such that they can’t be cleanly separated by a line — but if you could make all the red marbles levitate off the ground in just the right way, you could draw a plane separating them. Then you let them fall back to the ground knowing where the blues stop and reds begin.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ln9m5l2YUAe57R_T.png" /><figcaption>A nonseparable dataset in a two-dimensional space R², and the same dataset mapped onto threedimensions with the third dimension being x²+y² (source: <a href="http://www.eric-kim.net/eric-kim-net/posts/1/kernel_trick.html">http://www.eric-kim.net/eric-kim-net/posts/1/kernel_trick.html</a>)</figcaption></figure><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SYE5OjcSFe1MWdQC.png" /><figcaption>The decision boundary is shown in green, first in the three-dimensional space (left), then back in the two-dimensional space (right). Same source as previous image.</figcaption></figure><p>In summary, SVMs are used for classification with two classes. They attempt to find a plane that separates the two classes cleanly. When this isn’t possible, we either soften the definition of “separate,” or we throw the data into higher dimensions so that we <em>can</em> cleanly separate the data.</p><h3>Success!</h3><p>In this section we covered:</p><ul><li>The <strong>classification </strong>task of supervised learning</li><li>Two foundational classification methods: <strong>logistic regression</strong> and <strong>support vector machines (SVMs)</strong></li><li>Recurring concepts: the <strong>sigmoid</strong> function, <strong>log-odds (“logit”)</strong>, and <strong>false positives </strong>vs. <strong>false negatives</strong>,</li></ul><p>In <a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930">Part 2.3: Supervised Learning III</a>, we’ll go into <strong>non-parametric </strong>supervised learning, where the ideas behind the algorithms are very intuitive and performance is excellent for certain kinds of problems, but the models can be harder to interpret.</p><h3>Practice materials &amp; further reading</h3><h4>2.2a — Logistic regression</h4><p><em>Data School has an excellent </em><a href="http://www.dataschool.io/guide-to-logistic-regression/"><em>in-depth guide to logistic regression</em></a><em>. We’ll also continue to refer you to </em><a href="http://www-bcf.usc.edu/~gareth/ISL/"><em>An Introduction to Statistical Learning</em></a><em>. See Chapter 4 on logistic regression, and Chapter 9 on support vector machines.</em></p><p><em>To implement logistic regression, we recommend working on </em><a href="https://datahack.analyticsvidhya.com/contest/practice-problem-1/"><em>this problem set</em></a><em>. You have to register on the site to work through it, unfortunately. C’est la vie.</em></p><h4>2.2b—Down the SVM rabbit hole</h4><p><em>To dig into the math behind SVMs, watch Prof. Patrick Winston’s </em><a href="https://www.youtube.com/watch?v=_PwhiWxHK8o"><em>lecture</em></a><em> from MIT 6.034: Artificial Intelligence. And check out </em><a href="https://pythonprogramming.net/svm-in-python-machine-learning-tutorial/"><em>this tutorial</em></a><em> to work through a Python implementation.</em></p><h4>Enter your email below if you’d like to stay up-to-date with future content 💌</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f8c40%3Fas_embed%3Dtrue&amp;url=https%3A%2F%2Fupscri.be%2F8f8c40%2F&amp;image=https%3A%2F%2Fupscri.be%2Fmedia%2Fform.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href">https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href</a></iframe><h4>On Twitter? So are we. Feel free to keep in touch — <a href="https://twitter.com/v_maini">Vishal</a> and <a href="https://twitter.com/seriousssam">Samer</a> 🙌🏽</h4><p><em>More from</em><strong><em> Machine Learning for Humans </em></strong>🤖👶</p><ul><li><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12"><em>Part 1: Why Machine Learning Matters</em></a> ✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-740383a2feab"><em>Part 2.1: Supervised Learning</em></a><em> </em>✅</li><li><strong><em>Part 2.2: Supervised Learning II</em></strong><em> </em>✅</li><li><a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930"><em>Part 2.3: Supervised Learning III</em></a></li><li><a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294"><em>Part 3: Unsupervised Learning</em></a></li><li><a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b"><em>Part 4: Neural Networks &amp; Deep Learning</em></a></li><li><a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265"><em>Part 5: Reinforcement Learning</em></a></li><li><a href="https://medium.com/@v_maini/how-to-learn-machine-learning-24d53bb64aa1"><em>Appendix: The Best Machine Learning Resources</em></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=5c1c23f3560d" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/supervised-learning-2-5c1c23f3560d">Machine Learning for Humans, Part 2.2: Supervised Learning II</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Machine Learning for Humans, Part 2.1: Supervised Learning]]></title>
            <link>https://medium.com/machine-learning-for-humans/supervised-learning-740383a2feab?source=rss-19a4c39d50a8------2</link>
            <guid isPermaLink="false">https://medium.com/p/740383a2feab</guid>
            <category><![CDATA[tech]]></category>
            <category><![CDATA[deep-learning]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[supervised-learning]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[Vishal Maini]]></dc:creator>
            <pubDate>Sat, 19 Aug 2017 15:19:54 GMT</pubDate>
            <atom:updated>2018-05-28T18:06:59.094Z</atom:updated>
            <content:encoded><![CDATA[<h4>The two tasks of supervised learning: regression and classification. Linear regression, loss functions, and gradient descent.</h4><pre>This series is available as a full-length e-book! <a href="https://www.dropbox.com/s/e38nil1dnl7481q/machine_learning.pdf?dl=0"><strong>Download here</strong></a>. Free for download, contributions appreciated (<a href="http://paypal.me/ml4h"><strong>paypal.me/ml4h</strong></a><strong>)</strong></pre><p><em>How much money will we make by spending more dollars on digital advertising? Will this loan applicant pay back the loan or not? What’s going to happen to the stock market tomorrow?</em></p><p>In supervised learning problems, we start with a data set containing <strong>training examples</strong> with associated correct <strong>labels</strong>. For example, when learning to classify handwritten digits, a supervised learning algorithm takes thousands of pictures of handwritten digits along with labels containing the correct number each image represents. The algorithm will then learn the relationship between the images and their associated numbers, and apply that learned relationship to classify completely new images (without labels) that the machine hasn’t seen before. This is how you’re able to deposit a check by taking a picture with your phone!</p><p>To illustrate how supervised learning works, let’s examine the problem of <strong>predicting annual income</strong> based on the number of years of higher education someone has completed. Expressed more formally, we’d like to build a model that approximates the relationship <strong><em>f</em></strong> between the number of years of higher education <strong>X</strong> and corresponding annual income <strong>Y</strong>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/264/1*dBkmjNMkgxFV-OQJWjtfDw.png" /></figure><pre><strong>X (input)</strong> = years of higher education<br><strong>Y (output)</strong> = annual income<br><strong><em>f</em></strong> = function describing the relationship between X and Y<br><strong>ϵ (epsilon)</strong> = random error term (positive or negative) with mean zero</pre><pre><em>Regarding epsilon: </em></pre><pre>(1) ϵ<em> represents </em><strong><em>irreducible error </em></strong><em>in the model, which is a theoretical limit around the performance of your algorithm due to inherent noise in the phenomena you are trying to explain. For example, imagine building a model to predict the outcome of a coin flip.</em></pre><pre><em>(2) Incidentally, mathematician </em><a href="https://en.wikipedia.org/wiki/Paul_Erd%C5%91s"><em>Paul Erdős</em></a><em> referred to children as “epsilons” because in calculus (but not in stats!) </em>ϵ<em> denotes an arbitrarily small positive quantity. Fitting, no?</em></pre><p>One method for predicting income would be to create a rigid rules-based model for how income and education are related. For example:<em> “I’d estimate that for every additional year of higher education, annual income increases by $5,000.”</em></p><pre>income = ($5,000 * years_of_education) + baseline_income</pre><pre><em>This approach is an example of </em><strong><em>engineering</em></strong><em> a solution (vs. </em><strong><em>learning </em></strong><em>a solution, as with the linear regression method described below).</em></pre><p>You could come up with a more complex model by including some rules about degree type, years of work experience, school tiers, etc. For example: <em>“If they completed a Bachelor’s degree or higher, give the income estimate a 1.5x multiplier.”</em></p><p>But this kind of explicit rules-based programming doesn’t work well with complex data. Imagine trying to design an image classification algorithm made of if-then statements describing the combinations of pixel brightnesses that should be labeled “cat” or “not cat”.</p><p>Supervised machine learning solves this problem by getting the computer to <em>do the work for you. </em>By identifying patterns in the data, the machine is able to form heuristics. The primary difference between this and human learning is that machine learning runs on computer hardware and is best understood through the lens of computer science and statistics, whereas human pattern-matching happens in a biological brain (while accomplishing the same goals).</p><p>In supervised learning, the machine attempts to learn the relationship between income and education <em>from scratch</em>, by running <strong>labeled training data</strong> through a <strong>learning algorithm.</strong> This learned function can be used to estimate the income of people whose income Y is unknown, as long as we have years of education X as inputs. In other words, we can apply our model to the <strong>unlabeled test data </strong>to estimate Y.</p><p>The goal of supervised learning is to <strong>predict Y as accurately as possible</strong> when given new examples where X is known and Y is unknown. In what follows we’ll explore several of the most common approaches to doing so.</p><h3>The two tasks of supervised learning: regression and classification</h3><pre><strong>Regression:</strong> predict a continuous numerical value. <em>How much will that house sell for?</em></pre><pre><strong>Classification</strong>: assign a label.<em> Is this a picture of a cat or a dog?</em></pre><p>The rest of this section will focus on regression. In <a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d">Part 2.2</a> we’ll dive deeper into classification methods.</p><h3>Regression: predicting a continuous value</h3><p>Regression predicts a <strong>continuous</strong> <strong>target variable Y</strong>. It allows you to estimate a value, such as housing prices or human lifespan, based on input data X.</p><p>Here, <strong>target variable</strong> means the unknown variable we care about predicting, and <strong>continuous</strong> means there aren’t gaps (discontinuities) in the value that Y can take on. A person’s weight and height are continuous values. <strong>Discrete</strong> variables, on the other hand, can only take on a finite number of values — for example, the number of kids somebody has is a discrete variable.</p><p>Predicting income is a classic regression problem. Your <strong>input data X </strong>includes all relevant information about individuals in the data set that can be used to predict income, such as years of education, years of work experience, job title, or zip code. These attributes are called <strong>features</strong>, which can be <strong>numerical</strong> (e.g. years of work experience) or <strong>categorical</strong> (e.g. job title or field of study).</p><p>You’ll want as many training observations as possible relating these features to the target output Y, so that your model can learn the relationship <em>f</em> between X and Y.</p><p>The data is split into a <strong>training data set</strong> and a <strong>test data set</strong>. The training set has labels, so your model can learn from these labeled examples. The test set does <em>not</em> have labels, i.e. you don’t yet know the value you’re trying to predict. It’s important that your model can generalize to situations it hasn’t encountered before so that it can perform well on the test data.</p><pre><strong>Regression</strong></pre><pre><strong>Y = f(X) + ϵ</strong>, where X = (x1, x2…xn)</pre><pre><strong>Training</strong>: machine learns <em>f </em>from labeled training data</pre><pre><strong>Test</strong>: machine predicts Y from unlabeled testing data</pre><pre><em>Note that X can be a </em><strong><em>tensor</em></strong><em> with an any number of dimensions. </em>A 1D tensor is a vector (1 row, many columns), a 2D tensor is a matrix (many rows, many columns), and then you can have tensors with 3, 4, 5 or more dimensions (e.g. a 3D tensor with rows, columns, and depth). For a review of these terms, see the first few pages of this <a href="http://www.deeplearningbook.org/contents/linear_algebra.html">linear algebra review</a>.</pre><p>In our trivially simple 2D example, this could take the form of a .csv file where each row contains a person’s education level and income. Add more columns with more features and you’ll have a more complex, but possibly more accurate, model.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*caLjbWlrKbLLqwFlGLbiuQ.png" /></figure><h3>So how do we solve these problems?</h3><p>How do we build models that make accurate, useful predictions in the real world? We do so by using <strong>supervised learning algorithms.</strong></p><p><strong>Now let’s get to the fun part: getting to know the algorithms.</strong> We’ll explore some of the ways to approach regression and classification and illustrate key machine learning concepts throughout.</p><h3>Linear regression (ordinary least squares)</h3><p><em>“Draw the line. Yes, this counts as machine learning.”</em></p><p>First, we’ll focus on solving the income prediction problem with linear regression, since linear models don’t work well with image recognition tasks (this is the domain of deep learning, which we’ll explore later).</p><p>We have our data set X<em>,</em> and corresponding target values Y. The goal of <strong>ordinary least squares (OLS)</strong> regression is to learn a linear model that we can use to predict a new <em>y</em> given a previously unseen <em>x</em> with as little error as possible. We want to guess how much income someone earns based on how many years of education they received.</p><pre>X_train = [4, 5, 0, 2, …, 6] # years of post-secondary education</pre><pre>Y_train = [80, 91.5, 42, 55, …, 100] # corresponding annual incomes, in thousands of dollars</pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/718/0*2J891Tbkau5ELp_5." /></figure><p>Linear regression is a <strong>parametric method</strong>, which means it makes an assumption about the form of the function relating X and Y (we’ll cover examples of non-parametric methods later). Our model will be a function that predicts <em>ŷ</em> given a specific <em>x</em>:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/203/0*tA7lgj0WSW64d9_k." /><figcaption>In this case, we make the explicit assumption that there is a <strong>linear relationship </strong>between X and Y — that is, for each one-unit increase in <em>X</em>, we see a constant increase (or decrease) in Y.</figcaption></figure><p><em>β0</em> is the y-intercept and <em>β1</em> is the slope of our line, i.e. how much income increases (or decreases) with one additional year of education.</p><p>Our goal is to learn the <strong>model parameters </strong>(in this case, <em>β0</em> and <em>β1</em>) that minimize error in the model’s predictions.</p><p>To find the best parameters:</p><blockquote>1. Define a <strong>cost function</strong>, or <strong>loss function</strong>, that measures how inaccurate our model’s predictions are.</blockquote><blockquote>2. Find the parameters that <strong>minimize loss</strong>, i.e. make our model as accurate as possible.</blockquote><p>Graphically, in two dimensions, this results in a line of best fit. In three dimensions, we would draw a plane, and so on with higher-dimensional hyperplanes.</p><pre><em>A note on dimensionality: our example is two-dimensional for simplicity, but you’ll typically have more features (x’s) and coefficients (betas) in your model, e.g. when adding more relevant variables to improve the accuracy of your model predictions. The same principles generalize to higher dimensions, though things get much harder to visualize beyond three dimensions.</em></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/720/0*fg6I9JUUT5X2VP3R." /></figure><p>Mathematically, we look at the difference between each real data point (<em>y</em>) and our model’s prediction (ŷ). Square these differences to avoid negative numbers and penalize larger differences, and then add them up and take the average. This is a measure of how well our data fits the line.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/341/0*4YosVQ8oGBg6ZAWv." /><figcaption><em>n = # of observations. Using 2*n instead of n makes the math work out more cleanly when taking the derivative to minimize loss, though some stats people say this is blasphemy. When you start having opinions on this kind of stuff, you’ll know you are all the way in the rabbit hole.</em></figcaption></figure><p>For a simple problem like this, we can compute a closed form solution using calculus to find the optimal beta parameters that minimize our loss function. But as a cost function grows in complexity, finding a closed form solution with calculus is no longer feasible. This is the motivation for an iterative approach called <strong>gradient descent</strong>, which allows us to minimize a complex loss function.</p><h3>Gradient descent: learn the parameters</h3><p><em>“Put on a blindfold, take a step downhill. You’ve found the bottom when you have nowhere to go but up.”</em></p><p>Gradient descent will come up over and over again, especially in neural networks. Machine learning libraries like <a href="http://scikit-learn.org/stable/">scikit-learn</a> and <a href="https://www.tensorflow.org/">TensorFlow</a> use it in the background everywhere, so it’s worth understanding the details.</p><p>The goal of gradient descent is to find the minimum of our model’s loss function by iteratively getting a better and better approximation of it.</p><p>Imagine yourself walking through a valley with a blindfold on. Your goal is to find the bottom of the valley. How would you do it?</p><p>A reasonable approach would be to touch the ground around you and move in whichever direction the ground is sloping down most steeply. Take a step and repeat the same process continually until the ground is flat. Then you know you’ve reached the bottom of a valley; if you move in any direction from where you are, you’ll end up at the same elevation or further uphill.</p><p>Going back to mathematics, the ground becomes our loss function, and the elevation at the bottom of the valley is the minimum of that function.</p><p>Let’s take a look at the loss function we saw in regression:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/341/0*XNjJ4YFPEF08f-1L." /></figure><p>We see that this is really a function of two variables: <em>β0</em> and <em>β1</em>. All the rest of the variables are determined, since X, Y, and <em>n </em>are given during training. We want to try to minimize this function.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ZaEKARNxNgB7-H3F." /></figure><p>The function is f(β0,β1)=z. To begin gradient descent, you make some guess of the parameters β0 and β1 that minimize the function.</p><p>Next, you find the <a href="https://en.wikipedia.org/wiki/Partial_derivative">partial derivatives</a><strong> </strong>of the loss function with respect to each beta parameter: [<em>dz/dβ0, dz/dβ1</em>]. A <strong>partial derivative</strong> indicates how much total loss is increased or decreased if you increase<em> β0</em> or <em>β1 </em>by a very small amount.</p><p>Put another way, how much would increasing your estimate of annual income assuming zero higher education (<em>β0</em>) increase the loss (i.e. inaccuracy) of your model? You want to go in the <em>opposite</em> direction so that you end up walking <em>downhill</em> and minimizing loss.</p><p>Similarly, if you increase your estimate of how much each incremental year of education affects income (<em>β</em>1), how much does this increase loss (<em>z</em>)? If the partial derivative <em>dz/β1 </em>is a <em>negative</em> number, then <em>increasing</em> <em>β1 </em>is good because it will reduce total loss. If it’s a <em>positive</em> number, you want to <em>decrease</em> <em>β1</em>. If it’s zero, don’t change <em>β1 because it means you’ve reached an optimum</em>.</p><p>Keep doing that until you reach the bottom, i.e. the algorithm <strong>converged</strong> and loss has been minimized. There are lots of tricks and exceptional cases beyond the scope of this series, but generally, this is how you find the optimal <strong>parameters </strong>for your <strong>parametric</strong> model.</p><h3>Overfitting</h3><p><strong><em>Overfitting:</em></strong><em> “Sherlock, your explanation of what just happened is too specific to the situation.” </em><strong><em>Regularization:</em></strong><em> “Don’t overcomplicate things, Sherlock. I’ll punch you for every extra word.” </em><strong><em>Hyperparameter (λ):</em></strong><em> “Here’s the strength with which I will punch you for every extra word.”</em></p><p>A common problem in machine learning is <strong>overfitting</strong>: learning a function that perfectly explains the training data that the model learned from, but doesn’t generalize well to unseen test data. Overfitting happens when a model <em>overlearns</em> from the training data to the point that it starts picking up idiosyncrasies that aren’t representative of patterns in the real world. This becomes especially problematic as you make your model increasingly complex. <em>Underfitting</em> is a related issue where your model is not complex enough to capture the underlying trend in the data.</p><pre><strong>Bias-Variance Tradeoff</strong></pre><pre><strong>Bias</strong> is the amount of error introduced by approximating real-world phenomena with a simplified model.</pre><pre><strong>Variance</strong> is how much your model&#39;s test error changes based on variation in the training data. It reflects the model&#39;s sensitivity to the idiosyncrasies of the data set it was trained on.</pre><pre>As a model increases in complexity and it becomes more wiggly (<strong>flexible)</strong>, its bias decreases (it does a good job of explaining the training data), but variance increases (it doesn&#39;t generalize as well). <strong>Ultimately, in order to have a good model, you need one with low bias <em>and</em> low variance.</strong></pre><figure><img alt="" src="https://cdn-images-1.medium.com/max/781/1*lb7lEh2Ob5PAJLtnAyGSBA.png" /><figcaption>Source: Coursera’s <a href="https://medium.com/@v_maini/machine-learning-for-humans-part-1-why-machine-learning-matters-965dfadf19d4">ML course</a>, taught by Andrew Ng</figcaption></figure><p>Remember that <em>the only thing we care about is how the model performs on test data</em>. You want to predict<em> </em>which emails will be marked as spam before they’re marked, not just build a model that is 100% accurate at reclassifying the emails it used to build itself in the first place. Hindsight is 20/20 — the real question is whether the lessons learned will help in the future.</p><p>The model on the right has zero loss for the <em>training data</em> because it perfectly fits every data point. But the lesson doesn’t generalize. It would do a horrible job at explaining a new data point that isn’t yet on the line.</p><p>Two ways to combat overfitting:</p><p><strong>1. Use more training data</strong>. The more you have, the harder it is to overfit the data by learning too much from any single training example.</p><p><strong>2. Use regularization</strong>. Add in a penalty in the loss function for building a model that assigns too much explanatory power to any one feature or allows too many features to be taken into account.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*rFT6mtU45diT0OJhlgDcBg.png" /></figure><p>The first piece of the sum above is our normal cost function. The second piece is a <strong>regularization term </strong>that adds a penalty for large beta coefficients that give too much explanatory power to any specific feature. With these two elements in place, the cost function now balances between two priorities: explaining the training data and preventing that explanation from becoming overly specific.</p><p>The <strong>lambda</strong> coefficient of the regularization term in the cost function is a <strong>hyperparameter: </strong>a general setting of your model that can be increased or decreased (i.e. <strong>tuned</strong>)<strong> </strong>in order to improve performance. A higher lambda value will more harshly penalize large beta coefficients that could lead to potential overfitting. To decide the best value of lambda, you’d use a method called <strong>cross-validation</strong> which involves holding out a portion of the training data during training, and then seeing how well your model explains the held-out portion. We’ll go over this in more depth</p><h3>Woo! We made it.</h3><p>Here’s what we covered in this section:</p><ul><li>How <strong>supervised machine learning</strong> enables computers to learn from labeled training data without being explicitly programmed</li><li>The tasks of supervised learning: <strong>regression</strong> and <strong>classification</strong></li><li><strong>Linear regression, </strong>a bread-and-butter <strong>parametric </strong>algorithm</li><li>Learning <strong>parameters</strong> with <strong>gradient descent</strong></li><li><strong>Overfitting </strong>and <strong>regularization</strong></li></ul><p>In the next section — <a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d">Part 2.2: Supervised Learning II</a> — we’ll talk about two foundational methods of classification: <strong>logistic regression</strong> and <strong>support vector machines</strong>.</p><h3>Practice materials &amp; further reading</h3><h4><em>2.1a — Linear regression</em></h4><p><em>For a more thorough treatment of linear regression, read chapters 1–3 of </em><a href="http://www-bcf.usc.edu/~gareth/ISL/index.html"><em>An Introduction to Statistical Learning</em></a><em>. The book is available for free online and is an excellent resource for understanding machine learning concepts with accompanying exercises.</em></p><p><em>For more practice:</em></p><ul><li><em>Play with the </em><a href="http://www.cs.toronto.edu/~delve/data/boston/bostonDetail.html"><em>Boston Housing dataset</em></a><em>. You can either use software with nice GUIs like Minitab and Excel or do it the hard (but more rewarding) way with </em><a href="https://medium.com/@haydar_ai/learning-data-science-day-9-linear-regression-on-boston-housing-dataset-cd62a80775ef"><em>Python</em></a><em> or </em><a href="https://datascienceplus.com/linear-regression-from-scratch-in-r/"><em>R</em></a><em>.</em></li><li><em>Try your hand at a </em><a href="http://www.kaggle.com"><em>Kaggle</em></a><em> challenge, e.g. </em><a href="https://www.kaggle.com/c/house-prices-advanced-regression-techniques"><em>housing price prediction</em></a><em>, and see how others approached the problem after attempting it yourself.</em></li></ul><h4><em>2.1b — Implementing gradient descent</em></h4><p><em>To actually implement gradient descent in Python, check out </em><a href="https://spin.atomicobject.com/2014/06/24/gradient-descent-linear-regression/"><em>this tutorial</em></a><em>. And </em><a href="http://eli.thegreenplace.net/2016/understanding-gradient-descent/"><em>here</em></a><em> is a more mathematically rigorous description of the same concepts.</em></p><p><em>In practice, you’ll rarely need to implement gradient descent from scratch, but understanding how it works behind the scenes will allow you to use it more effectively and understand why things break when they do.</em></p><h4>Enter your email below if you’d like to stay up-to-date with future content 💌</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fupscri.be%2F8f8c40%3Fas_embed%3Dtrue&amp;url=https%3A%2F%2Fupscri.be%2F8f8c40%2F&amp;image=https%3A%2F%2Fupscri.be%2Fmedia%2Fform.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=upscri" width="800" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href">https://medium.com/media/ba3502c3c2cc3159fdd3bda87c87164c/href</a></iframe><h4>On Twitter? So are we. Feel free to keep in touch — <a href="https://twitter.com/v_maini">Vishal</a> and <a href="https://twitter.com/seriousssam">Samer</a> 🙌🏽</h4><p><em>More from</em><strong><em> Machine Learning for Humans </em></strong>🤖👶</p><ul><li><a href="https://medium.com/machine-learning-for-humans/why-machine-learning-matters-6164faf1df12"><em>Part 1: Why Machine Learning Matters</em></a><em> </em>✅</li><li><strong><em>Part 2.1: Supervised Learning</em> ✅</strong></li><li><a href="https://medium.com/@v_maini/supervised-learning-2-5c1c23f3560d"><em>Part 2.2: Supervised Learning II</em></a></li><li><a href="https://medium.com/@v_maini/supervised-learning-3-b1551b9c4930"><em>Part 2.3: Supervised Learning III</em></a></li><li><a href="https://medium.com/@v_maini/unsupervised-learning-f45587588294"><em>Part 3: Unsupervised Learning</em></a></li><li><a href="https://medium.com/@v_maini/neural-networks-deep-learning-cdad8aeae49b"><em>Part 4: Neural Networks &amp; Deep Learning</em></a></li><li><a href="https://medium.com/@v_maini/reinforcement-learning-6eacf258b265"><em>Part 5: Reinforcement Learning</em></a></li><li><a href="https://medium.com/@v_maini/how-to-learn-machine-learning-24d53bb64aa1"><em>Appendix: The Best Machine Learning Resources</em></a></li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=740383a2feab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/machine-learning-for-humans/supervised-learning-740383a2feab">Machine Learning for Humans, Part 2.1: Supervised Learning</a> was originally published in <a href="https://medium.com/machine-learning-for-humans">Machine Learning for Humans</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>