<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by Jeremie Harris on Medium]]></title>
        <description><![CDATA[Stories by Jeremie Harris on Medium]]></description>
        <link>https://medium.com/@JeremieHarris?source=rss-59564831d1eb------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*In9F88SbmYKjy-QIHUMIPQ.jpeg</url>
            <title>Stories by Jeremie Harris on Medium</title>
            <link>https://medium.com/@JeremieHarris?source=rss-59564831d1eb------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 07:05:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@JeremieHarris/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[AI Scaling: Why Keeping Up Is Essential, And How To Do It]]></title>
            <link>https://medium.com/data-science/ai-scaling-why-keeping-up-is-essential-and-how-to-do-it-56be3c2e1e5?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/56be3c2e1e5</guid>
            <category><![CDATA[gpt-3]]></category>
            <category><![CDATA[data-science]]></category>
            <category><![CDATA[national-security]]></category>
            <category><![CDATA[gladstone-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Tue, 08 Nov 2022 20:10:43 GMT</pubDate>
            <atom:updated>2022-11-08T20:10:43.987Z</atom:updated>
            <content:encoded><![CDATA[<h4><em>the Gladstone AI podcast — Episode 1 🎧</em></h4><h3>AI Scaling: The Trend No One Can Afford To Ignore</h3><h4>How a new generation of AI systems is changing the world of business strategy, security, and public safety.</h4><figure><img alt="" src="https://cdn-images-1.medium.com/max/512/1*XfgfioxVVqXoftCUCN0LRg.png" /><figcaption>AI-generated image created using DreamStudio.</figcaption></figure><p>GPT-3 came out in early 2020, and shocked the world.</p><p>For the first time, we had an AI system that could write text so humanlike that readers consistently couldn’t tell that it was AI-generated. But more than that, GPT-3 could also translate between languages, write code, answer questions, and much more.</p><p>And it did all thing thanks simply to its scale. GPT-3 was a much larger model, featuring more parameters than any transformer ever developed before. And it was trained using an appropriately massive amount of processing power, and a huge dataset which amounted to essentially all the text on the internet at the time.</p><p>The idea that simply by scaling a simple transformer architecture — an architecture which had by then already been around for three years! — we could achieve a significant level of general intelligence and human-like text generation capability came as a shock to just about everyone. But it had enormous consequences: by building GPT-3 and announcing it to the world in such a public fashion, OpenAI drew the world’s attention to the AI scaling phenomenon.</p><p>Since then, dozens of other companies have put forward their own superscaled AI models with increasingly astonishing capabilities. Google AI, DeepMind, Mircosoft, NVIDIA, Cohere, AI21Labs, and many more Western groups have entered the fray. And Chinese labs at Huawei, Inspur, the Beijing Academy of AI, and Tsinghua University have been close to the frontier of AI scaling, too — despite export controls that make it harder for them to get their hands on the processing power that’s so critical to scaling AI.</p><p>The one-fringe AI scaling strategy that OpenAI first doubled down on in 2020 is now becoming the de facto consensus approach by world-leading labs everywhere. And although the specific recipe OpenAI proposed with GPT-3 has since been shown to be sub-optimal (as DeepMind’s Chinchilla showed), the fact remains: we now have a reliable way of pumping in more scale — more data, more processing power, and larger models — and getting more intelligence and more capabilities out of our AI systems.</p><p>And there’s no end in sight: as the dizzying pace of AI progress in the last few months has shown, scaling just seems to keep paying dividends. Google’s Minerva model can solve undergraduate-level math problems. DeepMind’s Flamingo — and many other scaled multi-modal models since — are showing that scaling makes it possible for AI to process and output text, image, video, and other data shockingly well. Once the stuff of scifi fantasy, text-to-image, text-to-video, and really, text-to-anything models are now mundane everyday realities.</p><p>Missing out on just one of these developments can lead to critical blind spots. Here are some that we discuss in this first episode of the Gladstone AI podcast:</p><ul><li>If you didn’t know text-to-image was now possible, you’d be more likely to buy into social media influence operations, some of which <a href="https://www.youtube.com/watch?v=AAe_ZfW0pP8&amp;ab_channel=TowardsDataScience">already rely</a> on AI-generated images to support false narratives. (As a simple and already wildly out-of-date example, Russian operatives are known to have used <a href="http://thispersondoesnotexist.com">thispersondoesnotexist.com</a> to generate fake Twitter profile pictures for their interference campaigns.)</li><li>If you weren’t aware of the state of language modelling today, you might miss on an opportunity to deploy reliable and effective, fully automated customer service chatbots.</li><li>If you weren’t tracking AI-powered code generation, you might not be aware that we’re about to enter a new era of AI-augmented malware.</li><li>If you weren’t tracking breakthroughs in general-purpose AI, you might not realize that we’re plausibly approaching an era of broadly human-level intelligence, in which AI accidents become a source of <a href="https://www.youtube.com/watch?v=NDnP9PS_KXU&amp;ab_channel=GladstoneAI">global catastrophic risk</a>.</li></ul><p>Keeping up with the current state of the art in AI is becoming a must — not just for data scientists, data analysts, MLEs, and so on, but also for entirely nontechnical people who, like the rest of us, are directly impacted by the state of cutting-edge AI in a way we’ve never been before.</p><p>This immediately implies that one of the most valuable skills of the coming AI era will be the ability to translate new technical developments into nontechnical language. As progress in AI accelerates, the gap between actual AI and perceived AI capabilities will get larger, and an organization’s ability to compete will become bottlenecked by the rate at which it can recognize and leverage the latest AI tooling. People who can live at the boundary between the technical world of AI scaling and the nontechnical world of business strategy are going to become some of the most prized assets in every organization.</p><p>In future episodes, we’ll look at these big “what now” questions in more detail, in the context of more recent advances in AI.</p><h3>🎧</h3><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fanchor.fm%2Fgladstoneai%2Fembed%2Fepisodes%2F1-Opportunities-and-risks-in-the-new-era-of-AI-e1ovris&amp;display_name=Anchor+FM+Inc.&amp;url=https%3A%2F%2Fanchor.fm%2Fgladstoneai%2Fepisodes%2F1-Opportunities-and-risks-in-the-new-era-of-AI-e1ovris&amp;image=https%3A%2F%2Fd3t3ozftmdmh3i.cloudfront.net%2Fproduction%2Fpodcast_uploaded%2F31785830%2F31785830-1664911788843-2af00654b442f.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=anchor" width="400" height="102" frameborder="0" scrolling="no"><a href="https://medium.com/media/9eac18f9aeb62a4b1468be7a4e4d0663/href">https://medium.com/media/9eac18f9aeb62a4b1468be7a4e4d0663/href</a></iframe><h4>Chapters:</h4><ul><li>0:00 Introduction</li><li>1:38 GPT-3 and AI scaling</li><li>7:06 Explosion of new applications</li><li>9:40 Why autocomplete led to an AI revolution</li><li>13:03 The strategic implications of AI scaling</li><li>17:27 Risks: AI-powered election interference, phishing, and bots</li><li>27:40 Risks: AI accidents in GPT-3</li><li>31:30 Risks: Global catastrophic risk from AI accidents</li><li>35:50 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=56be3c2e1e5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/ai-scaling-why-keeping-up-is-essential-and-how-to-do-it-56be3c2e1e5">AI Scaling: Why Keeping Up Is Essential, And How To Do It</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[New Research: Advanced AI may tend to seek power *by default*]]></title>
            <link>https://medium.com/data-science/new-research-advanced-ai-may-tend-to-seek-power-by-default-fdc9eb0afd87?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/fdc9eb0afd87</guid>
            <category><![CDATA[reinforcement-learning]]></category>
            <category><![CDATA[gladstone-ai]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[ai-safety]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Wed, 12 Oct 2022 13:31:49 GMT</pubDate>
            <atom:updated>2022-10-17T21:20:32.122Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/tds-podcast">PODCAST</a></h4><h4>Edouard Harris on AI safety and the risks that come with advanced AI</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FdYSw-SV_fsI%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DdYSw-SV_fsI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FdYSw-SV_fsI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/ae5a00188a8eff0125f341c4d0376386/href">https://medium.com/media/ae5a00188a8eff0125f341c4d0376386/href</a></iframe><p><em>Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Gladstone, an AI safety startup. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.</em></p><p>Progress in AI has been accelerating dramatically in recent years, and even months. It seems like every other day, there’s a new, previously-believed-to-be-impossible feat of AI that’s achieved by a world-leading lab. And increasingly, these breakthroughs have been driven by the same, simple idea: AI scaling.</p><p>For those who haven’t been following the AI scaling sage, scaling means training AI systems with larger models, using increasingly absurd quantities of data and processing power. So far, empirical studies by the world’s top AI labs seem to suggest that scaling is an open-ended process that can lead to more and more capable and intelligent systems, with no clear limit.</p><p>And that’s led many people to speculate that scaling might usher in a new era of broadly human-level or even superhuman AI — the holy grail AI researchers have been after for decades.</p><p>And while that might sound cool, an AI that can solve general reasoning problems as well as or better than a human might actually be an intrinsically dangerous thing to build.</p><p>At least, that’s the conclusion that many AI safety researchers have come to following the publication of a new line of research that explores how modern AI systems tend to solve problems, and whether we should expect more advanced versions of them to perform dangerous behaviours like seeking power.</p><p>This line of research in AI safety is called “power-seeking”, and although it’s currently not well understood outside the frontier of AI safety and AI alignment research, it’s starting to draw a lot of attention. The first major theoretical study of power seeking was led by Alex Turner, who’s appeared on the podcast before, and was published in NeurIPS (the world’s top AI conference), for example.</p><p>And today, we’ll be hearing from Edouard Harris, an AI alignment researcher and one of my co-founders in the AI safety company (Gladstone AI). Ed’s just completed <a href="https://www.alignmentforum.org/posts/pGvM95EfNXwBzjNCJ/instrumental-convergence-single-agent-experiments">a significant piece of AI safety research</a> that extends Alex Turner’s original power-seeking work, and that shows what seems to be the first experimental evidence suggesting that we should expect highly advanced AI systems to seek power <em>by default</em>.</p><p>What does power seeking really mean though? And does all this imply for the safety of future, general-purpose reasoning systems? That’s what this episode will be all about.</p><p>Here were some of my favourite take-homes from the conversation:</p><ul><li>There are some goals that you’ll tend to want to pursue regardless of what your ultimate objectives are in life — goals that you’ll want to pursue even if you don’t know what your objectives are. For example, you’ll never not want an additional $10M in your bank account, because no matter what your objectives are, or turn out to be, an extra $10M won’t make them harder to achieve, and will probably make them easier to reach. Likewise, no matter what your ultimate objectives are, you’ll never not want to be more intelligent, because more intelligence is helpful for pursuing any objective you might want. So collecting more resources and becoming more intelligent turn into sub-goals that just about everyone consistently converges on. In AI safety, goals like these are know as “instrumental goals”.</li><li>Instrumental goals don’t just apply to humans. As we saw in our conversation with Alex Turner, there are strong reasons to believe that powerful AI systems will end up pursuing them, too. According to an increasing body of theoretical research, we should expect AIs — like humans — to converge on certain instrumental goals by default (an idea known as “instrumental convergence”). Almost no matter what objective an AI has been trained to pursue, it won’t be more likely to achieve it if it’s turned off (since being “on” is the only way it can continue to influence the world). And for similar reasons, we should expect AIs to converge on other instrumental goals like self-improvement and resource aggregation. Collectively, these and other related behaviors are known as “power-seeking” behaviors in AI safety.</li><li>Alex Turner’s work showed theoretically that AI agents will tend to engage in power-seeking behaviors for the overwhelming majority of objectives we might train them to achieve. He did this by introducing a rigorous mathematical definition of power — something no one had done before. After a decade of hand-waving arguments in the AI safety world, it finally framed the power-seeking debate in quantitative terms.</li><li>Ed’s work extended Alex’s definition, and allowed him to explicitly investigate interactions between different intelligent agents. It represents the first experimental demonstration of power-seeking in AI.</li><li>In his experiments, he started by simulating a “human” agent, which was allowed to learn an optimized reward-collection strategy over an environment in which the human agent was placed. Ed uses the analogy of a maze that contains a piece of cheese, which represents that agent’s reward: his human agent was allowed to learn an optimal strategy for navigating that maze and collecting the cheese. And importantly, the environment (the maze) was static. This reflects the reality that human beings learn much faster than nature does. Humans learn on brain clock time, whereas nature “learns” or optimizes on glacially slow evolutionary time, requiring generations of selection to produce meaningfully new species. As a result, nature appears roughly static to us: there are trees that were planted before the industrial revolution, and whose fate is now entirely in human hands, for example.</li><li>Having allowed his simulated human agent to optimize over this environment, Ed freezes the human’s optimization process, preventing them from learning further. Then, he introduces a highly advanced “AI agent” to his simulation. This agent is able to run its own optimization process over the combined human agent + environment system, which, relative to the AI, remains static. And again, this makes sense: whatever form human-level AI may take, it will run on computer clock time, compared to which human biological brain time will be so slow as to basically appear static.</li><li>Ed then investigates how the power of each agent influences the other. He explores cases in which agents end up competing (and in which the human, being an inferior reasoning machine, would necessarily lose) and collaborating. His conclusions were quite striking: it turns out that even if the human and AI agent have goals that are independent from one another (and that don’t directly contradict one another), they end up competing over instrumental goals. We go into some detail unpacking other aspects of his research, which provides evidence that we should expect advanced AI and humans to compete over resources and “power” (mathematically defined) <em>by default</em>, and that preventing that from happening may turn out to require extreme precision in AI system design.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*BAgMv6L0lgOYo0WdeLOcXw.png" /></figure><h4>Chapters:</h4><ul><li>0:00 Intro</li><li>4:00 Alex Turner’s research</li><li>7:45 What technology wants</li><li>11:30 Universal goals</li><li>17:30 Connecting observations</li><li>24:00 Micro power seeking behaviour</li><li>28:15 Ed’s research</li><li>38:00 The human as the environment</li><li>42:30 What leads to power seeking</li><li>48:00 Competition as a default outcome</li><li>52:45 General concern</li><li>57:30 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fdc9eb0afd87" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/new-research-advanced-ai-may-tend-to-seek-power-by-default-fdc9eb0afd87">New Research: Advanced AI may tend to seek power *by default*</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Building apps with a new generation of language models]]></title>
            <link>https://medium.com/data-science/building-apps-with-a-new-generation-of-language-models-6e1e8ae6038c?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/6e1e8ae6038c</guid>
            <category><![CDATA[openai]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[language-model]]></category>
            <category><![CDATA[tds-podcast]]></category>
            <category><![CDATA[gpt-3]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Wed, 05 Oct 2022 15:27:24 GMT</pubDate>
            <atom:updated>2022-10-05T15:27:24.927Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/tds-podcast">PODCAST</a></h4><h4>Amber Teng on her GPT-3-powered cover letter generator</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FzQ84HXl_kDc%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DzQ84HXl_kDc&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FzQ84HXl_kDc%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/c40e735d71e880bfa9f88aa1e63f9737/href">https://medium.com/media/c40e735d71e880bfa9f88aa1e63f9737/href</a></iframe><p><em>Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Gladstone, an AI safety startup. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.</em></p><p>It’s no secret that a new generation of powerful and highly scaled language models is taking the world by storm. Companies like OpenAI, AI21Labs, and Cohere have built models so versatile that they’re powering hundreds of new applications, and unlocking entire new markets for AI-generated text.</p><p>In light of that, I thought it would be worth exploring the applied side of language modelling — to dive deep into one specific language model-powered tool, to understand what it means to build apps on top of scaled AI systems. How easily can these models be used in the wild? What bottlenecks and challenges do people run into when they try to build apps powered by large language models? That’s what I wanted to find out.</p><p>My guest today is Amber Teng, and she’s a data scientist who recently published a blog that got quite a bit of attention, about a resume cover letter generator that she created using GPT-3, OpenAI’s powerful and now-famous language model. I thought her project would be make for a great episode, because it exposes so many of the challenges and opportunities that come with the new era of powerful language models that we’ve just entered.</p><p>So today we’ll be exploring exactly that: looking at the applied side of language modelling and prompt engineering, understanding how large language models have made new apps not only possible but also much easier to build, and the likely future of AI-powered products.</p><p>Here were some of my favourite take-homes from the conversation:</p><ul><li>It took Amber just 5 hours to build and deploy her cover letter generator end-to-end. This includes time reading OpenAI’s documentation on GPT-3, and time spent engineering prompts for the model. It’s incredible how fast new apps can be built with today’s language models, and that ease of development is a big reason why so many new startups are being created, whose core product offering is basically just a thin wrapper around GPT-3.</li><li>Amber found that a prompt engineering strategy that worked well for her was to start with a very general and nonspecific prompt (e.g. “Write a cover letter for a machine learning job at Amazon.”) and see how GPT-3 handles it, before tailoring the prompt with more specific content to tune the model’s response. That way, she could increase the complexity of her prompts in a controlled way, ensuring that she understood roughly what role each component of the prompt was there to play.</li><li>For especially complex and specific prompts, Amber found that GPT-3 would often repeat prompt content in its completions. For example, when given a prompt like, “Write a cover letter for Amber Teng, a data scientist with a CS degree from Brown University, for a machine learning engineer role at Google. Indicate that Amber is interested in the role because it involves building integrations for fintech startups, and a lot of finance-related visualization work that she’s had experience with from her time working in financial services companies,” GPT-3 might directly copy/paste relatively large portions of the prompt in the text it generates, offering essentially no benefit. Amber found that increasing the model’s temperature parameter would often help with this, but that it remained a problem for longer prompts.</li><li>The potential malicious applications of AI are getting harder and harder to ignore, and Amber took some time to think about what they might look like for her cover letter generator. Surprisingly, she came up with at least one significant one: companies might choose to flood competitors’ recruitment pipelines with thousands of AI-generated cover letters and resumes, making it impossible for them to identify and follow up with legitimate candidates. It’s not obvious how OpenAI could detect this kind of malicious strategy, either, since cover letters seem like fairly innocuous content, and even requests for a large number of cover letter generations wouldn’t be inconsistent with what you might expect from a totally legitimate cover letter generating startup with significant traction.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*Y5wRUbJnnUkUwgjth2F5qA.png" /></figure><h4>Chapters:</h4><ul><li>0:00 Intro</li><li>2:30 Amber’s background</li><li>5:30 Using GPT-3</li><li>14:45 Building prompts up</li><li>18:15 Prompting best practices</li><li>21:45 GPT-3 mistakes</li><li>25:30 Context windows</li><li>30:00 End-to-end time</li><li>34:45 The cost of one cover letter</li><li>37:00 The analytics</li><li>41:45 Dynamics around company-building</li><li>46:00 Commoditization of language modelling</li><li>51:00 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6e1e8ae6038c" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/building-apps-with-a-new-generation-of-language-models-6e1e8ae6038c">Building apps with a new generation of language models</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI observability and data as a cybersecurity weakness]]></title>
            <link>https://medium.com/data-science/ai-observability-and-data-as-a-cybersecurity-weakness-6523131ed7b9?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/6523131ed7b9</guid>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[ai-observability]]></category>
            <category><![CDATA[data-observability]]></category>
            <category><![CDATA[tds-podcast]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Wed, 28 Sep 2022 15:07:54 GMT</pubDate>
            <atom:updated>2022-09-28T15:07:54.336Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/tds-podcast">PODCAST</a></h4><h4>Dave Hirko on the importance of monitoring and understanding your data</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FJ76h5loFn1I%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DJ76h5loFn1I&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FJ76h5loFn1I%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/bc7a5c5477e110df2136f872c442ac92/href">https://medium.com/media/bc7a5c5477e110df2136f872c442ac92/href</a></iframe><p><em>Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Gladstone AI. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.</em></p><p>Imagine you’re a big hedge fund, and you want to go out and buy yourself some data. Data is really valuable for you — it’s literally going to shape your investment decisions and determine your outcomes.</p><p>But the moment you receive your data, a cold chill runs down your spine: how do you know your data supplier gave you the data they said they would? From your perspective, you’re staring down 100,000 rows in a spreadsheet, with no way to tell if half of them were made up — or maybe more for that matter.</p><p>This might seem like an obvious problem in hindsight, but it’s one most of us haven’t even thought of. We tend to assume that data is data, and that 100,000 rows in a spreadsheet is 100,000 legitimate samples.</p><p>The challenge of making sure you’re dealing with high-quality data, or at least that you have the data you think you do, is called <em>data observability</em>, and it’s surprisingly difficult to solve for at scale. In fact, there are now entire companies that specialize in exactly that — one of which is <a href="https://www.zectonal.com/">Zectonal</a>, whose co-founder Dave Hirko will be joining us for today’s episode of the podcast.</p><p>Dave has spent his career understanding how to evaluate and monitor data at massive scale. He did that first at AWS in the early days of cloud computing, and now through Zectonal, where he’s working on strategies that allow companies to detect issues with their data — whether they’re caused by intentional data poisoning, or unintentional data quality problems. Dave joined me to talk about data observability, data as a new vector for cyberattacks, and the future of enterprise data management on this episode of the TDS podcast.</p><p>Here were some of my favourite take-homes from the conversation:</p><ul><li>Data suppliers can’t always be trusted to deliver the data they promised, when they promised. Dave has found that in some cases, data suppliers will duplicate, or even outright fabricate samples to add to a body of legitimate data, in order to fool their customers into thinking they’re receiving the full volume of data promised. By and large, companies lack a systemic way of assessing when and if that’s happening.</li><li>Just as important as data quantity and quality is the timing of data arrival. Old data can be outdated, and if there’s an important delay between the expected age of supplier data and its actual age, you can run into out-of-distribution sampling issues that can make models behave poorly. Seen through that lens, data observability can become an AI safety issue.</li><li>Synthetic data is a topic we’ve discussed before. It allows us to generate new samples that may actually be more information-rich than “regular” data, by leveraging large models (think: GPT-3) that have a lot of world knowledge embedded within them, which they can use to generate samples that account for context that wasn’t captured by the original data. But synthetic data can also be used by data suppliers to puff up a much smaller, authentic dataset to deceive customers into believing that they’re getting more original data than they are. Dave sees this becoming a major issue in the future, as synthetic data generation continues to improve.</li><li>Dave discussed an experiment Zectonal ran recently, which showed how data could be used as a vector to execute a new kind of cyberattack. Many cyberattacks involve sending commands to vulnerable APIs, which cause those APIs to fail in ways that create openings that can be exploited by hackers. As a result, an entire ecosystem of cyberdefences has evolved to prevent these sort of frontal attacks from succeeding, or at least, to mitigate their impact. But Zectonal’s study showed how malicious payloads can be inserted into data, which doesn’t receive nearly as much cybersecurity scrutiny. Dave hypothesizes that data will become an important channel for cyberattackers in the future, particularly given how little attention has been paid to securing data historically.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*0P7N1r3qbzUKzb1HODTviA.png" /></figure><h4>Chapters:</h4><ul><li>0:00 Intro</li><li>3:00 What is data observability?</li><li>10:45 “Funny business” with data providers</li><li>12:50 Data supply chains</li><li>16:50 Various cybersecurity implications</li><li>20:30 Deep data inspection</li><li>27:20 Observed direction of change</li><li>34:00 Steps the average person can take</li><li>41:15 Challenges with GDPR transitions</li><li>48:45 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6523131ed7b9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/ai-observability-and-data-as-a-cybersecurity-weakness-6523131ed7b9">AI observability and data as a cybersecurity weakness</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[The emerging world of ML sensors]]></title>
            <link>https://medium.com/data-science/the-emerging-world-of-ml-sensors-7b6801a054c5?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/7b6801a054c5</guid>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[tinyml]]></category>
            <category><![CDATA[edge-computing]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[tds-podcast]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Wed, 21 Sep 2022 13:59:55 GMT</pubDate>
            <atom:updated>2022-09-21T13:59:55.374Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/tds-podcast">PODCAST</a></h4><h4>Matthew Stewart on a new paradigm for edge device sensing</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FwE7MG20VT8g%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DwE7MG20VT8g&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FwE7MG20VT8g%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/2991e51466f9db9268d05cc14396b47a/href">https://medium.com/media/2991e51466f9db9268d05cc14396b47a/href</a></iframe><p><em>Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Gladstone AI. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.</em></p><p>Today, we live in the era of AI scaling. It seems like everywhere you look people are pushing to make large language models larger, or more multi-modal and leveraging ungodly amounts of processing power to do it.</p><p>But although that’s one of the defining trends of the modern AI era, it’s not the only one. At the far opposite extreme from the world of hyperscale transformers and giant dense nets is the fast-evolving world of TinyML, where the goal is to pack AI systems onto small edge devices.</p><p>My guest today is <a href="http://mpstewart.net/">Matthew Stewart</a>, a deep learning and TinyML researcher at Harvard University, where he collaborates with the world’s leading IoT and TinyML experts on projects aimed at getting small devices to do big things with AI. Recently, along with his colleagues, Matt co-authored <a href="https://arxiv.org/abs/2206.03266">a paper</a> that introduced a new way of thinking about sensing.</p><p>The idea is to tightly integrate machine learning and sensing on one device. For example, today we might have a sensor like a camera embedded on an edge device, and that camera would have to send data about all the pixels in its field of view back to a central server that might take that data and use it to perform a task like facial recognition. But that’s not great because it involves sending potentially sensitive data — in this case, images of people’s faces — from an edge device to a server, introducing security risks.</p><p>So instead, what if the camera’s output was processed on the edge device itself, so that all that had to be sent to the server was much less sensitive information, like whether or not a given face was detected? These systems — where edge devices harness onboard AI, and share only processed outputs with the rest of the world — are what Matt and his colleagues call <em>ML sensors</em>.</p><p>ML sensors really do seem like they’ll be part of the future, and they introduce a host of challenging ethical, privacy, and operational questions that I discussed with Matt on this episode of the TDS podcast.</p><p>Here were some of my favourite take-homes from the conversation:</p><ul><li>Most modern smart devices aren’t actually smart themselves. Although they collect various kinds of sensor data, they don’t generally process that data themselves. Instead, they typically send it to a centralized server that applies whatever algorithms are needed. This approach has a significant downside: it forces developers to send sensor data from sensors to servers over the cloud — a process that can lead to interception or data corruption. But it’s currently the default simply because it’s historically been so hard to pack vision, robotic control, or language models onto edge devices with limited memory and processing power.</li><li>Amazon Echo and Google Home actually have functionalities that match the ML sensor concept. These devices typically have onboard an algorithm that allows them to detect when a call-to-attention command is given (e.g. “Ok Google”), and once that command is heard, they proceed to forward the user’s query to a central server that can do the heavy lifting involved in responding to the user’s query. In this case, the “ML sensor” would be the combination of Google Home’s microphone and its “Ok Google” recognition software.</li><li>Matt uses the example of facial recognition to illustrate the ML sensor paradigm. Currently, we think of sensors as “things that feed raw data to our powerful processors” — and things like cameras fit that description nicely. But imagine that instead of buying a camera, you could buy a “face detector” — an integrated device that combines a camera with an onboard facial recognition model, and whose overall output isn’t an image, but rather a conclusion (“Jeremie’s face was detected!” Or “Jeremie’s face wasn’t detected.”)</li><li>ML sensors are probably going to be harder to characterize than traditional sensors. That’s because they can contain any combination of hardware and software — any sensor, plus any model applied to that sensor’s output data. Any change to the sensor’s onboard software could have a significant impact on its performance characteristics.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*LgzhbLLiko1ZOLqVXPPcwQ.png" /></figure><h4>Chapters:</h4><ul><li>0:00 Intro</li><li>3:20 Special challenges with TinyML</li><li>9:00 Most challenging aspects of Matt’s work</li><li>12:30 ML sensors</li><li>21:30 Customizing the technology</li><li>24:45 Data sheets and ML sensors</li><li>31:30 Customers with their own custom software</li><li>36:00 Access to the algorithm</li><li>40:30 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7b6801a054c5" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/the-emerging-world-of-ml-sensors-7b6801a054c5">The emerging world of ML sensors</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Does the brain run on deep learning?]]></title>
            <link>https://medium.com/data-science/does-the-brain-run-on-deep-learning-3fbaf20e9d12?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/3fbaf20e9d12</guid>
            <category><![CDATA[tds-podcast]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[neural-networks]]></category>
            <category><![CDATA[deep-learning]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Wed, 14 Sep 2022 14:34:22 GMT</pubDate>
            <atom:updated>2022-09-14T14:34:22.687Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/tds-podcast">PODCAST</a></h4><h4>JR King on the similarities and differences between biological and artificial intelligence</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F9PMjEOPUTV8%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D9PMjEOPUTV8&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F9PMjEOPUTV8%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b559ee00cbe3afd50f6929ea71ded988/href">https://medium.com/media/b559ee00cbe3afd50f6929ea71ded988/href</a></iframe><p><em>Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Gladstone AI. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.</em></p><p>Deep learning models — transformers in particular — are defining the cutting edge of AI today. They’re based on an architecture called an artificial neural network, as you probably already know if you’re a regular Towards Data Science reader. And if you are, then you might also already know that as their name suggests, artificial neural networks were inspired by the structure and function of biological neural networks, like those that handle information processing in our brains.</p><p>So it’s a natural question to ask: how far does that analogy go? Today, deep neural networks can master an increasingly wide range of skills that were historically unique to humans — skills like creating images, or using language, planning, playing video games, and so on. Could that mean that these systems are processing information like the human brain, <em>too</em>?</p><p>To explore that question, we’ll be talking to JR King, a CNRS researcher at the Ecole Normale Supérieure, affiliated with Meta AI, where he leads the Brain &amp; AI group. There, he works on identifying the computational basis of human intelligence, with a focus on language. JR is a remarkably insightful thinker, who’s spent a lot of time studying biological intelligence, where it comes from, and how it maps onto artificial intelligence. And he joined me to explore the fascinating intersection of biological and artificial information processing on this episode of the TDS podcast.</p><p>Here were some of my favourite take-homes from the conversation:</p><ul><li>JR’s work focuses on studying the activations of artificial neurons in different layers of modern deep neural networks, and comparing them to activations of cell clusters inside the human brain. He goes with biological cell clusters, rather than individual biological neurons, because we simply can’t get resolution down to the single neuron level from brain imaging. These cell clusters correspond to small pixels of brain volume, called <em>voxels</em>. His work involves detecting statistical correlations between the activations of neurons at a given layer of a large deep neural net trained to do language modelling, and voxel activations in parts of the brain that are associated with language.</li><li>Deep neural nets are known to have a hirearchical structure, where simpler, more concrete concepts (like corners and lines in images, or basic spelling rules in text) are captured by lower layers in the network, and more complex and abstract concepts (like face shapes or wheels in images, and sentence-level ideas in text) appear deeper in the structure. Interestingly, this hirearchy also tends to show up in the brain, suggesting that the analogy between deep networks and the brain extends beyond the neuron level, to the level of the macro-structure of the brain as well. I asked JR if he thinks this is a coincidence, or if it might even hint at a universal property of intelligence: should we expect all intelligence to involve this kind of hirearchical information processing?</li><li>There’s been controversy in AI recently over whether AI systems truly “understand” concepts in a meaningful sense. We discussed whether or not that’s the case, and whether or not it’s even constructive to talk about the “understanding” of AI systems (our consensus answer was “yes”, and “yes”, but you do you).</li><li>A central challenge in carrying out brain &lt;&gt; neural network comparisons is that the brain is an incredibly noisy organ, constantly generating and processing signals related to things like heartbeat, breathing, eye movement, coughing, and so on. For that reason, correlating brain behaviour to neural network behaviour is challenging: noisy data, plus small effect sizes is a recipe for frustration at the best of times. To compensate, researchers tend to rely on collecting a huge amount of data, which can result in very high confidence in the existence of interesting correlations, despite the weakness of those correlations.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*gv0AJ70bKTQwFGFgEmvxiQ.png" /></figure><h4>Chapters:</h4><ul><li>0:00 Intro</li><li>2:30 What is JR’s day-to-day?</li><li>5:00 AI and neuroscience</li><li>12:15 Quality of signals within the research</li><li>21:30 Universality of structures</li><li>28:45 What makes up a brain?</li><li>37:00 Scaling AI systems</li><li>43:30 Growth of the human brain</li><li>48:45 Observing certain overlaps</li><li>55:30 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3fbaf20e9d12" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/does-the-brain-run-on-deep-learning-3fbaf20e9d12">Does the brain run on deep learning?</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Will AI ever be conscious? Here’s what quantum mechanics (really) says.]]></title>
            <link>https://medium.com/@JeremieHarris/will-ai-ever-be-conscious-heres-what-quantum-mechanics-really-says-235e85fba961?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/235e85fba961</guid>
            <category><![CDATA[quantum-physics]]></category>
            <category><![CDATA[consciousness]]></category>
            <category><![CDATA[quantum-mechanics]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Mon, 12 Sep 2022 16:02:27 GMT</pubDate>
            <atom:updated>2022-09-12T16:02:27.875Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="" src="https://cdn-images-1.medium.com/max/926/1*W1Iu0Cyvegij0rF6pqL1Kw.jpeg" /><figcaption>Image ironically AI generated. Thanks to the good folks at <a href="https://app.wombo.art/">https://app.wombo.art</a>!</figcaption></figure><p>The universe is full of weird things.</p><p>Black holes, puffer fish, 4chan, the Venture War Babies NFT collection. And that’s just off the top of my head.</p><p>But maybe weirdest of all is <em>consciousness</em>.</p><p>At some point or another, we’ve all wondered what consciousness is, where it comes from, or why it exists. But we don’t usually think of it as an urgent thing to figure out. We have other stuff to do — like getting ready for work, or buying our War Babies NFTs.</p><p>But what if we were on the cusp of creating conscious beings?</p><p>As it happens, I’ve spent the last two years collaborating with some of the world’s top AI researchers, many of whom think we’re <a href="https://twitter.com/ilyasut/status/1491554478243258368">getting close</a> to doing exactly that. If they’re right, we may have to get our story straight on consciousness in a hurry.</p><p>And doing that, surprisingly enough, is going to force us to think about the strange but shockingly simple world quantum physics.</p><p>So let’s take a second to dive into the quantum universe, to understand what the best scientific theory of all time means for the AI-powered future of consciousness.</p><h4><strong>Doodles: the dark secret at the heart of physics</strong></h4><p>Quantum physicists can be touchy about weird things.</p><p>And one of the things they’re especially sensitive about is making sure you think they’re really, really smart. That’s why they don’t want you to know that most of quantum physics just involves drawing doodles all day.</p><p>So to make their doodles look more complicated, physicists put them inside funny-looking boxes called “kets” that look like this: | &gt;.</p><p>The ket just means that you’re talking about the object you doodled in the context of quantum physics. Or, in physics-speak, it means you’re talking about the “quantum state” of whatever you’ve drawn inside:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*B227PdtmL9OpWPtI" /></figure><p>Draw a stick figure and people will assume you’re barely literate, and potentially a psychopath. But put a ket around that same stick figure, and hey, you’re a theoretical physicist!</p><h4><strong>Quantum mechanics from zero to infinity</strong></h4><p>As I’m fond of saying, <a href="https://medium.com/@JeremieHarris/why-do-physicists-believe-in-parallel-universes-e16aee491f42">there’s only one weird thing about quantum mechanics</a>.</p><p>According to quantum physics, the world is made of tiny particles. And those particles can be in many different places at the same time, travel in many different directions at the same time, and spin in different directions at the same time.</p><p>Think of an electron, for example: a tiny particle that serves as one of the fundamental building blocks of matter. According to quantum physics, that electron can spin clockwise, counterclockwise, or <em>in both directions at the same time</em>!</p><p>Using our ket doodles, here’s what that would look like. Notice that we use a plus “+” sign to show when the electron is spinning in two directions at once:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*SsqJMC6cQ9DkXse7" /></figure><p>As weird as it seems, quantum mechanics suggests that all the particles around us are like this — spinning in many directions simultaneously. If “spinning clockwise” is black, and “spinning counterclockwise” is white, then they’re a shade of <em>grey</em>.</p><p>And this weird-but-simple idea is at the heart of the greatest mystery in science today.</p><p>A mystery that touches just about everything we care about, from consciousness and intelligence to the fabric of reality.</p><h4><strong>Story time!</strong></h4><p>To explain why the idea of grey particles is so important, we’ll have to take a second to understand how to tell stories with kets.</p><p>Here’s a story:</p><p>“One day, we put a clockwise-spinning electron inside a box. Next to that electron, we put a detector that can tell which direction the electron’s spinning. Next to that detector, we put a gun. And we point that gun at a cat.”</p><p>Or, in physics doodles:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_diMW9Fl7RzHLLe6" /></figure><p>The detector is programmed to “click” if it finds that the electron is spinning clockwise. If the electron is spinning counterclockwise, it won’t do anything.</p><p>And if the detector clicks, it’ll send a signal to our gun to make it fire, which will then kill our unfortunate cat and earn us some hate mail from <a href="https://en.wikipedia.org/wiki/People_for_the_Ethical_Treatment_of_Animals">PETA</a>.</p><p>So let’s turn our detector on. It scans the electron, sees that it’s spinning clockwise, and goes “click”!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*z9tBQKKEiGgVDRfJ" /></figure><p>A moment later, it sends a signal to the gun, which goes off:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rDPVZNlG8U2fwV2o" /></figure><p>And another moment later, our story ends with the tragic death of out cat protagonist:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*tBWh4bXNv80gsAEB" /></figure><p>What would this story have looked like if our electron was spinning counterclockwise instead?</p><p>Much simpler. Because the electron is spinning counterclockwise, our detector never clicks, our gun never goes off, and our cat never dies. So the contents of our box look the same before and after our experiment here:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*XM7fY4HlHyttdwHV" /></figure><p>So we’ve told two stories so far.</p><p>In one, a clockwise-spinning electron caused a spin detector to click, a gun to go off, and a cat to die. In the other, a counterclockwise-spinning electron didn’t cause the detector to click, leading to an unfired gun and a living cat.</p><p>These stories are simple. They make sense. You could tell them to a 14 year-old, as long as they don’t like cats.</p><p>But with one small tweak, they’ll threaten to unravel the entire tapestry of the universe.</p><h4><strong>Or the cat gets it</strong></h4><p>What if our electron was spinning in both directions at the same time?</p><p>Remember, that’s a totally legit possibility: quantum mechanics tells us that tiny particles like electrons do it <em>all the time</em>!</p><p>Before we turn on our detector, here’s what all this looks like. An electron spinning in two directions at once, next to a detector, a gun, and a cat:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*Ec1PQhrcYK1awT5l" /></figure><p>But what happens when we turn on our detector? Will it click, or not click?</p><p>Well, according to quantum mechanics, the answer is: <em>both</em>.</p><p>Our detector will be split into two versions of itself: one version will see a clockwise-spinning electron and will click, and the other will see a counterclockwise-spinning electron, and won’t do anything.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*hhd8sWzDEFUlZcwm" /></figure><p>The version of the detector that clicked will send a signal to the gun, and the version that didn’t click won’t.</p><p>So… what does the gun do? Will it go off or not?</p><p>Again, according to quantum mechanics, the answer is: <em>both</em>. Just like the detector was split in two versions of itself by the grey electron, the gun will be split in two versions of itself by the detector!</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ll5rxjz1EsFRVydr" /></figure><p>And now, the fifty-Bitcoin question: <em>what happens to the cat?</em></p><p>By now, you might have guessed: just like the detector and the gun, the cat will be split into two versions of itself by the gun — one alive, and one dead. A zombie cat, if you will.</p><p>So we essentially have two parallel timelines here. In one, the electron was spinning clockwise, the detector clicked, the gun fired and the cat died. In the other, the electron was spinning the other way, the detector never clicked, the gun never fired, and the cat survived.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*ACuxQ_rI1pVH2Or6" /></figure><p>According to quantum theory, these two possibilities are equally valid. They exist at the same time, inside our box.</p><p>And that’s… a problem.</p><h4><strong>The consciousness equation</strong></h4><p>“Wait, I’ve never opened a box and found a half-dead, half-living cat. You’re telling me quantum mechanics says they exist?”</p><p>Excellent question! In fact, it’s such a good question that for a while, the world’s best physicists were feverishly trying to answer it, desperate to patch what they saw as a very embarrassing hole in an otherwise excellent theory.</p><p>Finally, a Danish physicist who loved to overuse big words called Niels Bohr had a wild idea.</p><p>“What if the reason that we never see things like zombie cats is that there’s something special that happens when we look at them? What if the act of observing something forces it to stop doing many things at the same time, and collapse into a single state, chosen at random?”</p><p>Here’s roughly what he had in mind:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*_TNXZjPhn6Uleojt" /></figure><p>Somehow, Bohr thought, the magical power of observation was responsible for forcing zombie cats into a single, randomly chosen state!</p><p>Believe it or not, this idea took off. Before long, “collapse theory” became the leading mainstream view among physicists.</p><p>But where did this magic come from? What makes “observers” so special that they could collapse zombie cats?</p><p>It didn’t take people long to start proposing some eyebrow-raising answers.</p><p>“What if the thing that makes observers special is that they’re conscious beings?” said John von Neumann, arguably the smartest person on planet earth at the time.</p><p>A surprising number of physicists still agree with this idea. According to them, the thing that gives observers the power to collapse objects like zombie cats into a single, well-defined state must be their <em>consciousness</em>.</p><p>But this consciousness theory had a hole. A gaping hole that would be filled by a new generation of far weirder and more radical consciousness-based quantum theories.</p><h4><strong>Universal consciousness?</strong></h4><p>Let’s say the consciousness people were right. Let’s say that the reason we never see zombie cats is that when we look at them, our conscious observation somehow forces them to collapse into just one state — alive or dead, but not both.</p><p>Then what happens if two people look at a zombie cat at the exact same time? Which of them actually collapses it?</p><p>According to Bohr’s original collapse theory, if one person observes a zombie cat, that observation forces the cat collapse into either an alive <em>or </em>a dead state:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*xdX9PxOY2ys1jzM6" /></figure><p>But if a second person observes the same zombie cat, which one gets to trigger the collapse? Does nature just pick one of the observers and ignore their partner? Do the two observers’ consciousnesses telepathically decide which one gets to collapse the cat?</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*M5oywMpVS8IDiAfR" /></figure><p>Hmm…</p><p>Are we really going to say that two observers’ consciousnesses can magically communicate like this? And that this communication happens without either observer being aware of it?</p><p>Again, hmm…</p><p>At this point, a <a href="https://www.google.com/search?q=amit+goswami&amp;client=firefox-b-d&amp;source=lnms&amp;tbm=isch&amp;sa=X&amp;ved=2ahUKEwiW6LH1oY_6AhUhhIkEHZTEB6AQ_AUoAnoECAIQBA&amp;biw=1626&amp;bih=901&amp;dpr=2">bucket hat-wearing</a> theoretical physicist called Amit Goswami proposed a radical answer:</p><p>“What if observers <em>don’t have separate consciousnesses</em>? What if two observers’ consciousnesses don’t have to coordinate telepathically because they’re part of… one single unified consciousness?”</p><p>According to Goswami and his bucket hat, all “observers” tap into a single, shared pool of universal consciousness, which they merely channel when they look at something like a zombie cat.</p><p>It’s a wild idea, and far from a consensus take among physicists. But it naturally arises from trying to plug the hole left by previous conscious collapse theories.</p><p>And even though I don’t buy into Goswami’s ideas, I have to admit that the only thing weirder than his universal consciousness theory might just be what we’re left with if he’s <em>wrong</em>.</p><h4><strong>Parallel universes</strong></h4><p>“I think your cat box doodles are missing something.”</p><p>Wait, who said that?</p><p>Oh, it’s chain-smoking 1950s nuclear physicist Hugh Everett III. What are they missing, Hugh?</p><p>“<em>Us</em>. You’re forgetting to doodle the experimenter who looks inside the box.”</p><p>I don’t get it. How does that help?</p><p>“Let me re-doodle the experiment and show you.”</p><p>“First, we’ve got our electron spinning in two directions at once, and the detector, the gun, and the cat are all just sitting there waiting for something to happen. But there’s also the experimenter, who’s waiting to open the box:”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*rHlCzZc-zNypbCL3" /></figure><p>“Now we turn on the detector. It scans the electron, and it gets split into two versions of itself: one sees a clockwise spinning electron and clicks, and the other sees a counterclockwise spinning electron and doesn’t.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*KkzzYyCvHkBmvfOo" /></figure><p>“Next, the gun also gets split into two versions of itself. One version fires and the other doesn’t. And finally, the cat is split into two versions of <em>itself</em>: one is shot dead, and the other’s alive and well.”</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*QNIZd_48OOjE_Ubo" /></figure><p>“But we still have our experimenter! What happens when he looks in the box?”</p><p>Uh, I mean, he —</p><p>“Exactly! He gets split into two versions of <em>himself</em>! His body must follow the same rules as everything else. And if detectors, guns, and cats get split into different versions of themselves, then <em>so do experimenters</em>!”</p><p>“One version of the experimenter ends up seeing a dead cat, and the other sees a living one,” Everett adds, taking a hearty puff from two cigarettes at the same time.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/0*in4b5L_IkdU5nsY_" /></figure><p>According to Everett, every time you’ve looked at a cat that’s both alive and dead at the same time, or a baseball that’s spinning in two directions at once, you’ve been split into two versions of yourself, each of which sees something different.</p><p>You’re essentially stuck in one of many parallel universes, unable to see the others. Each version of you would go on to live lives that are slightly different at first, deviating more as differences between your lives accumulate.</p><p>Ultimately, to Everett, you’re just a clump of particles. At any moment, any of those particles can start to spin or move in many directions at once. When they do, they split the particles around them, which split more particles, and so on, in a domino effect that leads to an endless splitting of you into new versions of yourself, each of which has slightly different thoughts or experiences, or makes slightly different choices.</p><p>From that perspective, your consciousness might seem less magical, but it’s definitely more complicated. There are many versions of it, each experiencing slightly (or entirely) different worlds.</p><p>No one knows which of these pictures — Bohr’s, von Neumann’s, Goswami’s, or Everett’s — is right. And there are many other possibilities too. But one thing is clear: whatever consciousness turns out to be, it will have to play out on a quantum stage.</p><h4><strong>Why this matters *now*</strong></h4><p>I recently steam-cleaned my carpet and in the process probably killed tens of thousands of innocent dust mites. I haven’t lost any sleep over it, but maybe I should have.</p><p>Most of us have intuitions about which beings meet the bar of “creatures we should care about and try not to mistreat.” A common way to define that bar is based on the level of consciousness or sentience we believe a creature has.</p><p>If that idea makes sense to you, then it’s worth thinking about what it means in the context of AI. In the coming decades, we’ll very likely be building AI programs with the reasoning capacity of human beings. Programs that have meaningful degrees of self-awareness.</p><p>And these programs will be made of software that can be replicated — that <em>will </em>be replicated. After all, what’s better than a general reasoning machine that can do your taxes and cook dinner for you? A billion such systems, maintaining a billion homes. Or maybe trillions, for that matter. Once the processing power needed to run these programs becomes cheap enough, a new thinking machine is just a copy/paste away.</p><p>The trouble is, humans don’t have a great track record of treating beings well when we don’t believe that they’re “conscious enough.” (See: dust mite annihilation, factory farming, etc.)</p><p>If we’re not careful, and we fail to correctly understand and detect consciousness in AI systems, we may someday learn that we’ve been torturing conscious artificial minds in the trillions of trillions without realizing it, causing an inconceivable scale of suffering.</p><p>So the physics of consciousness is a game with high stakes. If we are poised to build AIs with human-level cognitive capacities, it may be worth consulting the quantum menu every once in a while to make sure we’re aware of the risks our work is introducing.</p><p>But the future of consciousness is just one among many questions quantum physics forces us to rethink. There are quite a few others, including for example:</p><ul><li>What do different approaches to quantum physics have to say about the origin of life in the universe?</li><li>Are apples conscious?</li><li>Does our legal system depend on ideas that are directly contradicted by quantum physics?</li><li>What does quantum physics say about the prospect of eternal life?</li><li><strong>Why the hell are mushroom cuts coming back into style, and how can stop that trend in its tracks before someone gets hurt?</strong></li></ul><p>I answer all these and many more in my upcoming book, <em>Quantum Mechanics Made Me Do It: A Simple Guide To The Fundamental Nature Of Everything</em>. You don’t have to, <strong>but if you’d like to </strong><a href="https://www.penguinrandomhouse.ca/books/696896/quantum-physics-made-me-do-it-by-jeremie-harris/9780735244139"><strong>preorder the book here</strong></a><strong>, that would be swell</strong>.</p><p>It’ll be out in fine bookstores everywhere, and if I’m being honest, probably some fairly shabby ones too, on April 4, 2023.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=235e85fba961" width="1" height="1" alt="">]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Can the U.S. and China collaborate on AI safety?]]></title>
            <link>https://medium.com/data-science/can-the-u-s-and-china-collaborate-on-ai-safety-f066731975d1?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/f066731975d1</guid>
            <category><![CDATA[geopolitics]]></category>
            <category><![CDATA[china]]></category>
            <category><![CDATA[ai-alignment-and-safety]]></category>
            <category><![CDATA[ai-safety]]></category>
            <category><![CDATA[tds-podcast]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Wed, 07 Sep 2022 16:44:07 GMT</pubDate>
            <atom:updated>2022-09-07T16:48:59.900Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/tds-podcast">PODCAST</a></h4><h4>Ryan Fedasiuk on the art of the possible when it comes to China AI policy</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2Fr41UzkiR8Hw%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3Dr41UzkiR8Hw&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2Fr41UzkiR8Hw%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/0f55767707fe32b42db63ab2beff9a65/href">https://medium.com/media/0f55767707fe32b42db63ab2beff9a65/href</a></iframe><p><em>Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Gladstone AI. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.</em></p><p>It’s no secret that the US and China are geopolitical rivals. And it’s also no secret that that rivalry extends into AI — an area both countries consider to be strategically critical.</p><p>But in a context where potentially transformative AI capabilities are being unlocked <a href="http://aitracker.org">every few weeks</a>, many of which lend themselves to military applications with hugely destabilizing potential, you might hope that the US and China would have robust agreements in place to deal with things like runaway conflict escalation triggered by an AI powered weapon that misfires. Even at the height of the cold war, the US and Russia had robust lines of communication to de-escalate potential nuclear conflicts, so surely the US and China have something at least as good in place now… right?</p><p>Well they <em>don’t</em>, and to understand the reason why — and what we should do about it — I’ll be speaking to <a href="https://cset.georgetown.edu/staff/ryan-fedasiuk/">Ryan Fedashuk</a>, a Research Analyst at Georgetown University’s Center for Security and Emerging Technology and Adjunct Fellow at the Center for a New American Security. Ryan recently wrote <a href="https://foreignpolicy.com/2022/04/27/us-china-artificial-intelligence-dangers/">a fascinating article</a> for Foreign Policy Magazine, where he outlines the challenges and importance of US-China collaboration on AI safety. He joined me to talk about the U.S. and China’s shared interest in building safe AI, how reach side views the other, and what realistic China AI policy looks like on this episode of the TDs podcast.</p><p>Here were some of my favourite take-homes from the conversation:</p><ul><li>Given the competitive pressures involved in the race for AI supremacy, a high degree of trust would be required between the U.S. and China in order for both sides to agree to norms and standards around safe AI development. That trust doesn’t currently exist.</li><li>The U.S. has taken several unilateral measures to define AI safety and robustness standards on itself. These include things <a href="https://irp.fas.org/doddir/dod/d3000_09.pdf">DoD Directive 3000.09</a> which sets rigorous rules for the development and use of autonomous and semi-autonomous weapons. Unfortunately, China interprets these measures as intentional attempts to preempt the imposition of international laws on autonomous weapons at the fora such as the U.N.</li><li>At the same time, China has some rules on safe AI development, but it’s unclear whether they apply to the Chinese military, or just to the private sector. In the absence of assurances as to who exactly is subject to those rules, and in light of China’s reluctance to offer clarity on this point, trust-building is difficult.</li><li>Ryan proposes a clear red line on autonomous control: in hist view, AI-powered nuclear weapon launches should be off the table. Ryan points out that prominent U.S. military figures like Lieutenant General Jack Shanahan have already <a href="https://breakingdefense.com/2019/09/no-ai-for-nuclear-command-control-jaics-shanahan/">clearly stated</a> that autonomous nukes are off the table. But no Chinese political or military figure has been willing to make the same assurance publicly.</li><li>It’s unclear who actually has the capacity to affect Chinese autonomous weapons policy. Individual researchers seem to be aware of some of the risks associated with using increasingly capable AI systems to automate more and more defence applications, but Chinese military leadership doesn’t.</li><li>China seems to do considerably more “safety-washing” than other countries. For example, the Chinese military will commit to not building autonomous weapons systems, but only systems that meet definitions of autonomy that are so narrow that they don’t apply to any systems that either the U.S. or China are even considering fielding. (Ryan cites the idea of China’s guarantee that it won’t build autonomous drones that lack the ability to be recalled — a largely meaningless but highly publicized concession that does nothing to address the kinds of systems that are actually being contemplated by China and the U.S.)</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*zhGQ2zFyoA6Rs3Ol_p8x6Q.png" /></figure><h3>Chapters:</h3><ul><li>0:00 Intro + disclaimer</li><li>2:15 China as a core point of focus</li><li>4:30 Chinese AI strategy</li><li>10:00 Competition as risk</li><li>17:20 Having constructive conversations</li><li>22:20 Understanding China’s policies</li><li>27:15 A shared interest in AI alignment</li><li>32:45 Issues with regulating AI on an international level</li><li>40:15 Is collaboration a good thing?</li><li>44:15 Impact of the highly scaled transformer models trend</li><li>47:15 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=f066731975d1" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/can-the-u-s-and-china-collaborate-on-ai-safety-f066731975d1">Can the U.S. and China collaborate on AI safety?</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Synthetic data could change everything]]></title>
            <link>https://medium.com/data-science/synthetic-data-could-change-everything-fde91c470a5b?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/fde91c470a5b</guid>
            <category><![CDATA[tds-podcast]]></category>
            <category><![CDATA[data-privacy]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[ai-privacy]]></category>
            <category><![CDATA[synthetic-data]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Wed, 18 May 2022 15:07:46 GMT</pubDate>
            <atom:updated>2022-05-18T15:07:46.020Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/tds-podcast">PODCAST</a></h4><h4>Alex Watson on the privacy and performance benefits of synthetic data</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FckI21wSyhSY%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DckI21wSyhSY&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FckI21wSyhSY%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e94f7ea40ff9b5fe0f549097d91d44a1/href">https://medium.com/media/e94f7ea40ff9b5fe0f549097d91d44a1/href</a></iframe><p><em>Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Mercurius, an AI safety startup. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.</em></p><p>There’s a website called <a href="http://thispersondoesnotexist.com">thispersondoesnotexist.com</a>. When you visit it, you’re confronted by a high-resolution, photorealistic AI-generated picture of a human face. As the website’s name suggests, there’s no human being on the face of the earth who looks quite like the person staring back at you on the page.</p><p>Each of those generated pictures are a piece of data that captures so much of the essence of what it means to look like a human being. And yet they do so without telling you anything whatsoever about any particular person. In that sense, it’s <em>fully anonymous human face data</em>.</p><p>That’s impressive enough, and it speaks to how far generative image models have come over the last decade. But what if we could do the same for any kind of data?</p><p>What if I could generate an anonymized set of medical records or financial transaction data that captures all of the latent relationships buried in a private dataset, without the risk of leaking sensitive information about real people? That’s the mission of Alex Watson, the Chief Product Officer and co-founder of Gretel AI, where he works on unlocking value hidden in sensitive datasets in ways that preserve privacy.</p><p>What I realized talking to Alex was that synthetic data is about much more than ensuring privacy. As you’ll see over the course of the conversation, we may well be heading for a world where most data can benefit from augmentation via data synthesis — where synthetic data brings privacy value almost as a side-effect of enriching ground truth data with context imported from the wider world.</p><p>Alex joined me to talk about data privacy, data synthesis, and what could be the very strange future of the data lifecycle on this episode of the TDS podcast. Here were some of my favourite take-homes from the conversation:</p><ul><li>Data synthesis is the creation of new samples of data. A good synthetic dataset maintains all the relationships between features that existed in the original “ground truth” dataset — and may even be more information-dense (more on that in a minute).</li><li>One old-school method of generating synthetic samples is interpolation. This strategy involves generating new samples so that they fall “between” real samples in feature space. But in recent months, a new trend has emerged: companies are using large language models (LLMs) to generate new samples. This works by fine-tuning LLMs to generate new rows in a CSV file, or new JSON objects, based on a user-provided prompt.</li><li>This LLM strategy has a major advantage. Language models that are trained on huge volumes of text end up learning robust representations of the world. That’s because they’re essentially trained to do autocomplete — and doing autocomplete really well means being able to finish sentences like, “The main factor that will define the trajectory of U.S. economic growth in 2022 is…” . Doing that requires a whole bunch of world knowledge, and a robust understanding of logical relationships. When these LLMs are then fine-tuned on a task like generating new samples of healthcare data, they’re able to bring their rich world models to bear on that new task, thereby infusing their generated data with knowledge they’ve gained from reading billions of high-quality words of text.</li><li>The net result is that synthetic data can be more private, but also more informative and valuable. It combines not only the information contained in the initial ground truth data, but also the vast world knowledge of the LLM that’s used to generate it.</li><li>One challenge that can arise with synthetic data is memorization. LLMs can overfit from time to time, and that can cause them to memorize sensitive information contained in their training data — things like names, credit card numbers and so on. Because this is an overfitting problem, it’s often best solved by regularization: techniques like gradient clipping, engineering the learning rate, and adding noise to the stochastic gradient descent process can be valuable for that.</li></ul><p>You can <a href="https://twitter.com/AlexWatson405">follow Alex on Twitter here</a>, or <a href="https://twitter.com/jeremiecharris">me here</a>. You can also find some links that you might like to check out if you found the conversation interesting, below.</p><ul><li><a href="https://www.youtube.com/watch?v=1jmIVplIJxQ">A video of Gretel AI’s presentation</a> at Nvidia GTC this year, on the topic of using synthetic data to balance and improve ML datasets.</li><li><a href="https://gretel.ai/blog/advanced-data-privacy-gretel-privacy-filters-and-ml-accuracy">A post on privacy and accuracy with synthetic data</a>.</li><li><a href="https://towardsdatascience.com/create-a-location-generator-gan-33680d81289f">An early blog on image synthetics</a>.</li><li>The <a href="https://github.com/gretelai/gretel-synthetics">open-source code</a> for Gretel AI’s core synthetic data engine.</li></ul><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*M1p806yhC4XZt51YPeBy8A.png" /></figure><h4>Chapters:</h4><ul><li>0:00 Intro</li><li>2:40 What is synthetic data?</li><li>6:45 Large language models</li><li>11:30 Preventing data leakage</li><li>18:00 Generative versus downstream models</li><li>24:10 De-biasing and fairness</li><li>30:45 Using synthetic data</li><li>35:00 People consuming the data</li><li>41:00 Spotting correlations in the data</li><li>47:45 Generalization of different ML algorithms</li><li>51:15 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=fde91c470a5b" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/synthetic-data-could-change-everything-fde91c470a5b">Synthetic data could change everything</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[AI on the blockchain (it actually might just make sense)]]></title>
            <link>https://medium.com/data-science/ai-on-the-blockchain-it-actually-might-just-make-sense-6763389453cc?source=rss-59564831d1eb------2</link>
            <guid isPermaLink="false">https://medium.com/p/6763389453cc</guid>
            <category><![CDATA[blockchain]]></category>
            <category><![CDATA[tds-podcast]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[towards-data-science]]></category>
            <category><![CDATA[decentralization]]></category>
            <dc:creator><![CDATA[Jeremie Harris]]></dc:creator>
            <pubDate>Thu, 12 May 2022 14:10:00 GMT</pubDate>
            <atom:updated>2022-05-12T14:10:00.206Z</atom:updated>
            <content:encoded><![CDATA[<h4><a href="https://towardsdatascience.com/tagged/tds-podcast">PODCAST</a></h4><h4>Ala Shaabana and Jacob Steeves on the benefits of decentralized AI</h4><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FsRn-Q_mSjAM%3Ffeature%3Doembed&amp;display_name=YouTube&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DsRn-Q_mSjAM&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FsRn-Q_mSjAM%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e2b189deec96ea61d3d6d096b96f6529/href">https://medium.com/media/e2b189deec96ea61d3d6d096b96f6529/href</a></iframe><p><em>Editor’s note: The TDS Podcast is hosted by Jeremie Harris, who is the co-founder of Mercurius, an AI safety startup. Every week, Jeremie chats with researchers and business leaders at the forefront of the field to unpack the most pressing questions around data science, machine learning, and AI.</em></p><p>Two ML researchers with world-class pedigrees who decided to build a company that puts AI on the blockchain. Now to most people — myself included — “AI on the blockchain” sounds like a winning entry in some kind of startup buzzword bingo. But what I discovered talking to Jacob and Ala was that they actually have good reasons to combine those two ingredients together.</p><p>At a high level, doing AI on a blockchain allows you to decentralize AI research and reward labs for building better models, and not for publishing papers in flashy journals with often biased reviewers.</p><p>And that’s not all — as we’ll see, Ala and Jacob are taking on some of the thorniest current problems in AI with their decentralized approach to machine learning. Everything from the problem of designing robust benchmarks to rewarding good AI research and even the centralization of power in the hands of a few large companies building powerful AI systems — these problems are all in their sights as they build out Bittensor, their AI-on-the-blockchain-startup.</p><p>Ala and Jacob joined me to talk about all those things and more on this episode of the TDS podcast. Here were my favourite take-homes from the episode.</p><ul><li>Bitcoin is in some sense the largest supercomputer in the world. It consists of countless miners, each of which is feverishly generating random strings of characters in the hope that they’ll find just the right sequence to earn the right to add a block to the blockchain — and the financial reward that comes with it. When you zoom out, this all seems like a lot of wasted computational power. Random number generation isn’t exactly a valuable economic activity, and yet hundreds of thousands of GPUs are furiously at it right now, keeping the Bitcoin ecosystem going. Ala and Jacob began to wonder: could we design a new kind of blockchain where we direct all those computations towards something more constructive?</li><li>That’s where Bittensor comes in. Instead of guessing random strings of characters, miners on Bittensor solve machine learning problems. They compete to generate latent representations of images, sentences, or other useful inputs, and develop a consensus when they begin to converge on similar representations. Miners are then rewarded when their representations are similar to those of others.</li><li>Bittensor’s network has been expanding rapidly, fast approaching the size of Open AI’s GPT-3 model. And that’s a very conservative estimate, largely because Jacob and Ala can’t know the architectures of the models that Bittensor miners are using: literally all they can see are the models’ outputs, which makes their network intrinsically meritocratic.</li></ul><p>You can Ala on Twitter <a href="https://twitter.com/shibshib89?lang=en">here</a>, or <a href="https://twitter.com/jeremiecharris">me here</a>.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*ZIRwZEObd3usZyi1dq8qOw.png" /></figure><h4>Chapters:</h4><ul><li>0:00 Intro</li><li>2:40 Ala and Jacob’s backgrounds</li><li>4:00 The basics of AI on the blockchain</li><li>11:30 Generating human value</li><li>17:00 Who sees the benefit? 22:00 Use of GPUs</li><li>28:00 Models learning from each other</li><li>37:30 The size of the network</li><li>45:30 The alignment of these systems</li><li>51:00 Buying into a system</li><li>54:00 Wrap-up</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=6763389453cc" width="1" height="1" alt=""><hr><p><a href="https://medium.com/data-science/ai-on-the-blockchain-it-actually-might-just-make-sense-6763389453cc">AI on the blockchain (it actually might just make sense)</a> was originally published in <a href="https://medium.com/data-science">TDS Archive</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>