<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0" xmlns:cc="http://cyber.law.harvard.edu/rss/creativeCommonsRssModule.html">
    <channel>
        <title><![CDATA[Stories by TensorFlow on Medium]]></title>
        <description><![CDATA[Stories by TensorFlow on Medium]]></description>
        <link>https://medium.com/@tensorflow?source=rss-b1d410cb9700------2</link>
        <image>
            <url>https://cdn-images-1.medium.com/fit/c/150/150/1*iDQvKoz7gGHc6YXqvqWWZQ.png</url>
            <title>Stories by TensorFlow on Medium</title>
            <link>https://medium.com/@tensorflow?source=rss-b1d410cb9700------2</link>
        </image>
        <generator>Medium</generator>
        <lastBuildDate>Wed, 18 Nov 2026 06:38:50 GMT</lastBuildDate>
        <atom:link href="https://medium.com/@tensorflow/feed" rel="self" type="application/rss+xml"/>
        <webMaster><![CDATA[yourfriends@medium.com]]></webMaster>
        <atom:link href="http://medium.superfeedr.com" rel="hub"/>
        <item>
            <title><![CDATA[Our blog has moved]]></title>
            <link>https://medium.com/tensorflow/the-tensorflow-blog-has-moved-3ec67b671229?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/3ec67b671229</guid>
            <category><![CDATA[announcements]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Tue, 05 Nov 2019 18:36:50 GMT</pubDate>
            <atom:updated>2019-11-06T18:06:28.286Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Posted by </em><a href="https://twitter.com/random_forests"><em>Josh Gordon</em></a><em> for the TensorFlow team.</em></p><p>Dear readers,</p><p>The TensorFlow blog has moved to <a href="https://blog.tensorflow.org">https://blog.tensorflow.org</a>. Please visit us there.</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*n_V0WobqWAJUUeVVZX9gkg.png" /><figcaption>Visit our new blog at https://blog.tensorflow.org</figcaption></figure><p>Thank you to our readers, and to our many guest authors who have created an incredible collection of content representing a diverse body of work. Thanks also to Medium for a great couple of years.</p><p>Although we’ve decided to move our blog, articles from the global TensorFlow community — including students, researchers, and companies large and small — will remain an important part of it.</p><p>In the future, new articles will be published on <a href="https://blog.tensorflow.org">https://blog.tensorflow.org</a>. The existing articles here on Medium will remain unchanged.</p><p>Over the next month, we plan to gradually republish our collection of articles on Medium to the new blog, so you’ll be able to find everything in one place. We’ve republished a few so far, with many more to go. With their permission, we plan to republish guest articles on the new blog in the same time frame.</p><p>Thank you again, and see you at <a href="https://blog.tensorflow.org">https://blog.tensorflow.org</a>.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=3ec67b671229" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/the-tensorflow-blog-has-moved-3ec67b671229">Our blog has moved</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Visual Wake Words with TensorFlow Lite Micro]]></title>
            <link>https://medium.com/tensorflow/visual-wake-words-with-tensorflow-lite-micro-8578e59ea6f9?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/8578e59ea6f9</guid>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[tensorflow-lite-micro]]></category>
            <category><![CDATA[wake-word]]></category>
            <category><![CDATA[tinyml]]></category>
            <category><![CDATA[microcontrollers]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Thu, 31 Oct 2019 18:25:40 GMT</pubDate>
            <atom:updated>2019-11-04T23:14:04.693Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Posted by Aakanksha Chowdhery, Software Engineer</em></p><p>Why does “Yo Google” not work with Google Assistant? After all, it’s only one word different from the phrase “Ok Google”. It’s because Google Assistant is listening for two specific words — or Wake Words. Wake Words are critical to the design of low-power machine learning to process data with a computationally inexpensive model to “wake up” the device for full processing. Audio wake words, such as “Ok Google”, are widely used to wake up AI assistant devices before they process speech using more computationally expensive machine learning models.</p><p>With the availability of low-power cameras, a popular application includes using a vision sensor with a microcontroller to classify when an image frame contains a person (or any object of interest). We refer to this application as Visual Wake Words because it enables a device to wake up when a human is present, analogous to how audio wake words are used in speech recognition.</p><p>Machine learning at the edge refers to models that run on-device, without connectivity to the cloud. As users deploy models on audio sensors or low-power cameras, microcontrollers provide a good compromise as a low-cost computational platform. However, available machine learning models hardly fit the device constraints (in terms of power consumption and processing power) of microcontrollers as detailed in this <a href="https://arxiv.org/pdf/1906.05721.pdf">paper</a>.</p><p>At <a href="http://cvpr2019.thecvf.com/">CVPR 2019</a>, Google organized a new challenge, called the “Visual Wake Words Challenge”, soliciting submissions of tiny vision models for microcontrollers. The challenge was to classify images to two classes (person/not-person) that serves a popular use-case for microcontrollers. Google open-sourced the <a href="https://github.com/tensorflow/models/blob/master/research/slim/datasets/build_visualwakewords_data.py">Visual WakeWords Dataset</a> derived from the <a href="http://cocodataset.org/">COCO dataset</a>: the label 1 corresponded to at least one person (or object of interest) being present in the image, and the label 0 corresponded to the image not containing any objects from person class.</p><figure><img alt="Visual Wake Words model to detect the presence of a person." src="https://cdn-images-1.medium.com/max/1000/1*QQashdLCaDDCBEF-WyB1Tw.png" /><figcaption>Visual Wake Words model to detect the presence of a person.</figcaption></figure><p>Machine learning for microcontroller devices requires re-thinking the model design, accounting for new tradeoffs between memory footprint, accuracy, and compute costs. Typical microcontrollers have extremely limited on-chip memory (100–320 KB SRAM) and flash storage (256 KB–1 MB). The entire neural network model with its weight parameters and code has to fit within the small memory budget of flash storage. Further, the temporary memory buffer required to store the input and output activations during computation must not exceed the on-chip memory. In Visual Wake Words Challenge, researchers designed models with highest accuracy that fit the microcontroller device constraints: model size is less than 250 KB, peak memory usage is less than 250 KB, inference cost is less than 60 million multiply-adds per inference. The talk from the <a href="https://rebootingcomputing.ieee.org/lpirc/2019">workshop</a> is available on <a href="http://ieeetv.ieee.org/conference-highlights/visual-wake-words-challenge-aakanksha-chowdhery-lpirc-2018?rf=series%7C3">IEEEtv</a>.</p><figure><img alt="Typical microcontrollers have extremely limited on-chip memory and flash storage." src="https://cdn-images-1.medium.com/max/1024/1*OHOSns-ab4U-PtGwUuXRlQ.png" /><figcaption>Typical microcontrollers have extremely limited on-chip memory and flash storage. For example, <a href="https://www.sparkfun.com/products/15170">SparkFun Edge</a> development board has 384KB RAM and 1MB Flash.</figcaption></figure><p>The submissions used optimization techniques such as model pruning and quantization (available from <a href="https://www.tensorflow.org/model_optimization">Tensorflow Model Optimization toolkit</a>) and neural architecture search algorithms to design tiny models that fit microcontroller device constraints. To deploy their model on device, users leverage <a href="https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro/">Tensorflow Lite Micro</a>, the ML framework for microcontrollers from TensorFlow team.</p><p>The challenge generated a lot of excitement in the research community and received submissions from ARM, Samsung, Qualcomm, MIT, Berkeley, University of Oxford etc. On the Visual WakeWords dataset, top-scoring entries from MIT and Qualcomm achieved a classification accuracy of 94.5% and 95% in two categories: deployable today and deployable in next-generation ML frameworks.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2F7-beBCKVpFE%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3D7-beBCKVpFE&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2F7-beBCKVpFE%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/e7ad8384482f0fdbe6a9aeccedc3617b/href">https://medium.com/media/e7ad8384482f0fdbe6a9aeccedc3617b/href</a></iframe><p>To download the Visual Wake Words dataset and train a model yourself, you can walk through the following <a href="https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/experimental/micro/examples/person_detection/training_a_model.md">tutorial</a>.</p><p><em>Thank you to everyone who worked on this release: Aakanksha Chowdhery, Daniel Situnayake, Pete Warden as well as the following colleagues for their guidance and advice: Jon Shlens, Andrew Howard, Rocky Rhodes, Nat Jeffres, Bo Chen, Mark Sandler, Meghna Natraj, Andrew Selle, Jared Duke.</em></p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8578e59ea6f9" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/visual-wake-words-with-tensorflow-lite-micro-8578e59ea6f9">Visual Wake Words with TensorFlow Lite Micro</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Build and deploy TensorFlow.js models with the power of AutoML]]></title>
            <link>https://medium.com/tensorflow/build-and-deploy-tensorflow-js-models-with-the-power-of-automl-1985b8985083?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/1985b8985083</guid>
            <category><![CDATA[tensorflowjs]]></category>
            <category><![CDATA[web]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[javascript]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Thu, 24 Oct 2019 17:08:12 GMT</pubDate>
            <atom:updated>2019-10-24T17:08:12.565Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Posted by Daniel Smilkov, Sandeep Gupta, and Vishy Tirumalashetty</em></p><p><a href="http://tensorflow.org/js">TensorFlow.js</a> is an open-source library that lets you define, train, and run machine learning models in Javascript. The library has empowered a new set of developers from the extensive JavaScript community to build and deploy machine learning models and enabled new classes of on-device computation. TensorFlow.js runs in all major browsers, server-side in Node.js, and as of recently, in <a href="https://github.com/tensorflow/tfjs-wechat">WeChat</a> and <a href="https://github.com/tensorflow/tfjs/tree/master/tfjs-react-native">React Native</a> providing hybrid-apps access to ML without having to leave the JS ecosystem. Today, we are excited to bring a new way for JavaScript developers to easily train a custom model on your own data and deploy it in your JavaScript application using the GCP AutoML service.</p><p>One of the main goals behind TensorFlow.js is to empower JS developers to use the benefits of ML without having to create their own models. There are two existing ways to bring the power of ML to your JavaScript applications: use one of our pre-packaged models, or fine-tune a model on your own data.</p><p>For example, our pre-packaged <a href="https://medium.com/tensorflow/real-time-human-pose-estimation-in-the-browser-with-tensorflow-js-7dd0bc881cd5">Pose estimation</a> and <a href="https://medium.com/tensorflow/introducing-bodypix-real-time-person-segmentation-in-the-browser-with-tensorflow-js-f1948126c2a0">Body segmentation</a> models allow you to build <a href="https://twitter.com/teropa/status/1001151524351959041">new forms of user interaction</a> and can form the basis for new <a href="https://experiments.withgoogle.com/collection/creatability">accessibility tools</a>. All of these models are published on NPM and you can use them with a few lines of code (no ML knowledge required!):</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/59ef70f67a812addee794523ec1faa84/href">https://medium.com/media/59ef70f67a812addee794523ec1faa84/href</a></iframe><p>If you have your own data, you can start with a pre-trained model and fine-tune it. The pre-trained models and the training itself can live in Python or JavaScript. If the training is done in Python, you can seamlessly convert the SavedModel or Keras model to TensorFlow.js using our <a href="https://github.com/tensorflow/tfjs/tree/master/tfjs-converter#getting-started">converter</a>. If the model is already in TensorFlow.js, you don’t need to do any conversion. <a href="https://codelabs.developers.google.com/codelabs/tensorflowjs-teachablemachine-codelab/index.html#0">This codelab</a> shows you how to take MobileNet, an image classification model optimized for edge devices, and fine-tune it on your own image dataset, all in JavaScript.</p><h4><strong>TensorFlow.js + AutoML</strong></h4><p>We are excited to announce the integration with <a href="https://cloud.google.com/vision/automl/docs/">Cloud AutoML Vision</a>, a Cloud service that enables developers to train a custom model on their labelled data. If your use-case is related to image classification or object detection, you can use the UI to upload your data, train an edge model and directly export it to a TensorFlow.js model, without any coding. Note that custom models can be directly trained using the open-source TensorFlow.js library (or Python TensorFlow), but AutoML provides an easy-to-use service that automates this process for you.</p><figure><img alt="TensorFlow.js model" src="https://cdn-images-1.medium.com/max/1024/0*NyEo0SUJOJKE26c6" /></figure><p>To make it easy to deploy and run the models efficiently in TensorFlow.js, we published a <a href="https://www.npmjs.com/package/@tensorflow/tfjs-automl">@tensorflow/tfjs-automl</a> library on NPM. Once you export the model, you can follow our step-by-step guides for <a href="https://cloud.google.com/vision/automl/docs/tensorflow-js-tutorial">image classification</a> and <a href="https://cloud.google.com/vision/automl/object-detection/docs/tensorflow-js-tutorial">object detection</a>. The guides use the NPM library and show you how to load the pre-trained model in the browser and make a prediction on an image.</p><p>If you already have a custom model, AutoML can still be incredibly useful. AutoML Vision Edge models are optimized to a small memory footprint and offer low latency while delivering high accuracy. We worked with several customers and partners and got some amazing early results and feedback. <a href="https://www.cvpcorp.com/">CVP</a>, a business and technology consulting company, has been evaluating some vision scenarios to help with workplace safety on edge-based models used in remote locations. Here is what Cal Zemelman, Director of Data science at CVP said on their results:</p><p><em>“We are working on a smartphone app for a Federal agency that performs image classification to help prevent accidents and improve public safety. The app is mainly written in JavaScript as a Progressive Web App to allow cross-platform usage, so we were training a Keras model and converting it afterwards to TensorFlow.js. With the new “Export to TensorFlow.js” feature, we were able to test out using AutoML to greatly streamline our model creation workflow. Previously, our best model was a retrained ResNet50 which got to about 91% accuracy. The middle of the road balanced performance model from AutoML Image Classification hit ~99% accuracy in 5 node hours. It is also 1/5 the size of the previous model and we saw a 60% improvement in the speed of inference.”</em></p><p>We are very excited about the early results our customers and partners are seeing and look forward to hearing more with this release.</p><p>You can train an AutoML Vision Edge model today. Once you train the model, you can export the TensorFlow.js model to your client-side application. With AutoML Vision Edge, you only pay for training for any usage beyond 15 node hours you get for free. There are no charges for exporting or client side predictions.</p><h4><strong>Next steps</strong></h4><ul><li>To learn more about AutoML, <a href="https://cloud.google.com/automl/">click here</a>.</li><li>For object detection, follow the step-by-step guides <a href="https://cloud.google.com/vision/automl/object-detection/docs/edge-quickstart">on training</a> and <a href="https://cloud.google.com/vision/automl/object-detection/docs/tensorflow-js-tutorial">deploying</a> to TensorFlow.js</li><li>For image classification, follow the step-by-step guides <a href="https://cloud.google.com/vision/automl/docs/edge-quickstart">on training</a> and <a href="https://cloud.google.com/vision/automl/docs/tensorflow-js-tutorial">deploying</a> to TensorFlow.js</li><li>Learn more about the AutoML NPM library via the <a href="https://www.npmjs.com/package/@tensorflow/tfjs-automl">official documentation</a>.</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=1985b8985083" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/build-and-deploy-tensorflow-js-models-with-the-power-of-automl-1985b8985083">Build and deploy TensorFlow.js models with the power of AutoML</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[“Mission to TensorFlow World” Demo]]></title>
            <link>https://medium.com/tensorflow/mission-to-tensorflow-world-demo-9dcc4371dce6?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/9dcc4371dce6</guid>
            <category><![CDATA[tensorflow-world]]></category>
            <category><![CDATA[conference]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[demo]]></category>
            <category><![CDATA[machine-learning]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Thu, 24 Oct 2019 17:07:10 GMT</pubDate>
            <atom:updated>2019-11-05T01:03:58.618Z</atom:updated>
            <content:encoded><![CDATA[<figure><img alt="Illustration of rocket ship" src="https://cdn-images-1.medium.com/max/1024/0*LBLOQuZKziKOWETT" /></figure><p><em>Whether you’re en route to TensorFlow World or you’re unable to make it, learn more below about our new demo experience and explore new GitHub code repos. Be sure to follow #MissionToTensorFlowWorld on Twitter to see this experience in action!</em></p><p>Celebrating the 50th anniversary of the Apollo 11 mission, the first manned mission to land on the Moon, <em>Mission to TensorFlow World</em> is TensorFlow’s latest space game experience that leverages the entire product ecosystem: TensorFlow Core, TensorFlow.js, TensorFlow Lite, and TensorFlow Extended (TFX) to control a spacecraft traveling through space. Each station uses different versions of TensorFlow on different platforms to power mini demos that influence a central visualization.</p><h4><strong>Pose Shield (TensorFlow.js)</strong></h4><p>On its <em>Mission to TensorFlow World</em>, the spacecraft must pass through a dangerous asteroid field. Using PoseNet and TensorFlow.js, you can enable and disable shield emitters around the periphery of a spacecraft using your arms attempting to block incoming asteroids. Are you quick enough and accurate enough to save the ship from damage? Play the game at <a href="http://g.co/PoseShield">g.co/PoseShield</a>.</p><figure><img alt="Pose Shield" src="https://cdn-images-1.medium.com/max/1024/0*vFuzUgCTisoJ6Zrn" /></figure><h4><strong>Emojicules (TensorFlow Lite)</strong></h4><p>Mechanical failures are inevitable on any space journey, and, unfortunately, the spacecraft has experienced a loss of cabin pressure. Using Emotion Classification and TensorFlow Lite, you must match your facial expression to emoji faces — representing the molecules that make up breathable air (Nitrogen, Oxygen, Carbon Dioxide, and Argon) — to capture those molecules and repressurize the cabin. <em>Repo coming soon!</em></p><figure><img alt="Emojicules demonstration" src="https://cdn-images-1.medium.com/max/1024/0*hyWYjob9-wx92ZFI" /></figure><h4><strong>Startographer (TensorFlow Core, TFX)</strong></h4><p>As the spacecraft enters new star systems on its <em>Mission to TensorFlow World</em>, charting the most direct course is challenging. Using XLNet and TensorFlow Core, planets are assigned conceptual phrases that must be matched with a synonym in order to pass. Startographer begins with a short animation describing the purpose and usage of TFX as a critical validation tool for any TensorFlow implementation. Within the gameplay itself, a TFX calibration animation displays a new planet group referencing TFX’s ability to identify and discard bad data. How many star systems can you travel through before time runs out? <em>Repo coming soon!</em></p><figure><img alt="Startographer demonstration" src="https://cdn-images-1.medium.com/max/1024/0*afRmrHqNOcpDzRmn" /></figure><p>For those who will be at TensorFlow world, we will see you soon! For those who can’t make it, be sure to watch the <a href="http://g.co/dev/tfworld">livestream</a> on Wednesday (10/30).</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=9dcc4371dce6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/mission-to-tensorflow-world-demo-9dcc4371dce6">“Mission to TensorFlow World” Demo</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[How-to Get Started with Machine Learning on Arduino]]></title>
            <link>https://medium.com/tensorflow/how-to-get-started-with-machine-learning-on-arduino-7daf95b4157?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/7daf95b4157</guid>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[arduino]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Tue, 15 Oct 2019 17:11:10 GMT</pubDate>
            <atom:updated>2019-10-16T17:06:36.074Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A guest post by Sandeep Mistry &amp; Dominic Pajak of the Arduino team</em></p><p><a href="https://www.arduino.cc">Arduino</a> is on a mission to make Machine Learning simple enough for anyone to use. We’ve been working with the TensorFlow Lite team over the past few months and are excited to show you what we’ve been up to together: bringing TensorFlow Lite Micro to the <a href="https://store.arduino.cc/nano-33-ble-sense">Arduino Nano 33 BLE Sense</a>. In this article, we’ll show you how to install and run several new <a href="https://www.tensorflow.org/lite/microcontrollers/overview">TensorFlow Lite Micro</a> examples that are now available in the <a href="https://www.arduino.cc/en/guide/libraries#toc1">Arduino Library Manager</a>.</p><p>The first tutorial below shows you how to install a neural network on your Arduino board to recognize simple voice commands.</p><figure><img alt="Running the pre-trained micro_speech inference example." src="https://cdn-images-1.medium.com/max/480/0*53gfZrrBtT1OZ-eH" /><figcaption><em>Example 1: Running the pre-trained micro_speech inference example.</em></figcaption></figure><p>Next, we’ll introduce a more in-depth tutorial you can use to train your own custom gesture recognition model for Arduino using TensorFlow in Colab. This material is based on a practical workshop held by Sandeep Mistry and Don Coleman, an updated version of which is <a href="https://github.com/arduino/ArduinoTensorFlowLiteTutorials/tree/master/GestureToEmoji">now online</a>.</p><figure><img alt="Training your own gesture classification model." src="https://cdn-images-1.medium.com/max/480/0*tTPJtxyn3ozkb4aO" /><figcaption><em>Example 2: Training your own gesture classification model.</em></figcaption></figure><p>If you have previous experience with Arduino, you may be able to get these tutorials working within a couple of hours. If you’re entirely new to microcontrollers, it may take a bit longer.</p><p>We’re excited to share some of the first examples and tutorials, and to see what you will build from here. Let’s get started!</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FHzCRZsGJLbI%3Ffeature%3Doembed&amp;url=https%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DHzCRZsGJLbI&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FHzCRZsGJLbI%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/b5f85ab7179553edce820a7a39a26968/href">https://medium.com/media/b5f85ab7179553edce820a7a39a26968/href</a></iframe><p><em>Note: the following projects are based on TensorFlow Lite for Microcontrollers which is currently experimental within the </em><a href="https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro"><em>TensorFlow repo</em></a><em>. This is still a new and emerging field!</em></p><h4>Microcontrollers and TinyML</h4><p>Microcontrollers, such as those used on Arduino boards, are low-cost, single chip, self-contained computer systems. They’re the invisible computers embedded inside<em> billions</em> of everyday gadgets like wearables, drones, 3D printers, toys, rice cookers, smart plugs, e-scooters, washing machines. The trend to connect these devices is part of what is referred to as the Internet of Things.</p><p>Arduino is an open-source platform and community focused on making microcontroller application development accessible to <a href="https://create.arduino.cc/projecthub">everyone</a>. The <a href="https://store.arduino.cc/usa/nano-33-ble-sense">board</a> we’re using here has an Arm Cortex-M4 microcontroller running at 64 MHz with 1MB Flash memory and 256 KB of RAM. This is tiny in comparison to Cloud, PC, or Mobile but reasonable by microcontroller standards.</p><figure><img alt="Arduino Nano 33 BLE Sense board" src="https://cdn-images-1.medium.com/max/1024/1*NSwaqMt3NSalIRr2p9LnCg.png" /><figcaption><em>Arduino Nano 33 BLE Sense board is smaller than a stick of gum</em></figcaption></figure><p>There are practical reasons you might want to squeeze ML on microcontrollers, including:</p><ul><li>Function — wanting a smart device to act quickly and locally (independent of the Internet).</li><li>Cost — accomplishing this with simple, lower cost hardware.</li><li>Privacy — not wanting to share all sensor data externally.</li><li>Efficiency — smaller device form-factor, energy-harvesting or longer battery life.</li></ul><p>There’s a final goal which we’re building towards that is very important:</p><ul><li>Machine learning can make microcontrollers accessible to developers who don’t have a background in embedded development</li></ul><p>On the machine learning side, there are techniques you can use to fit neural network models into memory constrained devices like microcontrollers. One of the key steps is the <a href="https://petewarden.com/2016/05/03/how-to-quantize-neural-networks-with-tensorflow/">quantization of the weights</a> from floating point to 8-bit integers. This also has the effect of making inference quicker to calculate and more applicable to lower clock-rate devices.</p><p>TinyML is an emerging field and there is still work to do — but what’s exciting is there’s a vast unexplored application space out there. Billions of microcontrollers combined with all sorts of sensors in all sorts of places which can lead to some seriously creative and valuable Tiny ML applications in the future.</p><h3>What you need to get started</h3><ul><li>An <a href="https://store.arduino.cc/nano-33-ble-sense">Arduino Nano 33 BLE Sense</a> board</li><li>A Micro USB cable to connect the Arduino board to your desktop machine</li><li>To program your board, you can use the <a href="https://www.arduino.cc/en/main/software">Arduino Web Editor</a> or install the <a href="https://www.arduino.cc/en/main/software">Arduino IDE</a>. We’ll give you more details on how to set these up in the following section</li></ul><p>The Arduino Nano 33 BLE Sense has a variety of onboard sensors meaning potential for some cool Tiny ML applications:</p><ul><li>Voice — digital microphone</li><li>Motion — 9-axis IMU (accelerometer, gyroscope, magnetometer)</li><li>Environmental — temperature, humidity and pressure</li><li>Light — brightness, color and object proximity</li></ul><p>Unlike classic Arduino Uno, the board combines a microcontroller with onboard sensors which means you can address many use cases without additional hardware or wiring. The board is also small enough to be used in end applications like wearables. As the name suggests, it has Bluetooth LE connectivity so you can send data (or inference results) to a laptop, mobile app or other BLE boards and peripherals.</p><p><strong>Tip: Sensors on a USB stick</strong></p><p>Connecting the BLE Sense board over USB is an easy way to capture data and add multiple sensors to single board computers without the need for additional wiring or hardware — a nice addition to a Raspberry Pi, for example.</p><h3>TensorFlow Lite for Microcontrollers examples</h3><p>The inference examples for TensorFlow Lite for Microcontrollers are now packaged and available through the Arduino Library manager making it possible to include and run them on Arduino in a few clicks. In this section we’ll show you how to run them. The examples are:</p><ul><li>micro_speech — speech recognition using the onboard microphone</li><li>magic_wand — gesture recognition using the onboard IMU</li><li>person_detection — person detection using an external ArduCam camera</li></ul><p>For more background on the examples you can take a look at the source in the <a href="https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/experimental/micro">TensorFlow repository</a>. The models in these examples were previously trained. The tutorials below show you how to deploy and run them on an Arduino. In the next section, we’ll discuss training.</p><h4>How to run the examples using Arduino Create web editor</h4><p><em>Once you connect your Arduino Nano 33 BLE Sense to your desktop machine with a USB cable you will be able to compile and run the following TensorFlow examples on the board by using the </em><a href="https://create.arduino.cc/editor"><em>Arduino Create</em></a><em> web editor:</em></p><figure><img alt="Compiling an example from the Arduino_TensorFlowLite library" src="https://cdn-images-1.medium.com/max/480/0*iWKllZhC0NtCmOYh" /><figcaption><em>Compiling an example from the Arduino_TensorFlowLite library</em></figcaption></figure><h4>Focus on the speech recognition example: micro_speech</h4><p>One of the first steps with an Arduino board is getting the LED to flash. Here, we’ll do it with a twist by using TensorFlow Lite Micro to recognise voice keywords. It has a simple vocabulary of “yes” and “no”. Remember this model is running locally on a microcontroller with only 256KB of RAM, so don’t expect commercial ‘voice assistant’ level accuracy — it has no Internet connection and on the order of 2000x less local RAM available.</p><p>Note the board can be battery powered as well. As the Arduino can be connected to motors, actuators and more, this offers the potential for voice controlled projects.</p><figure><img alt="Running the micro_speech example" src="https://cdn-images-1.medium.com/max/480/0*PmzGHGpJAwmeZtox" /><figcaption><em>Running the micro_speech example</em></figcaption></figure><h4>How to run the examples using the Arduino IDE</h4><p>Alternatively, you can use try the same inference examples using Arduino IDE application.</p><p>First, follow the instructions in the next section <em>Setting up the Arduino IDE.</em></p><p>In the Arduino IDE, you will see the examples available via the <strong>File &gt; Examples</strong> &gt; <strong>Arduino_TensorFlowLite </strong>menu in the ArduinoIDE.</p><p>Select an example and the sketch will open. To compile, upload, and run the examples on the board, and click the arrow icon:</p><figure><img alt="" src="https://cdn-images-1.medium.com/max/616/0*HwEj4ZsTv4n8RR7W" /></figure><p>For advanced users who prefer a command line, there is also the <a href="https://github.com/arduino/arduino-cli#arduino-cli">arduino-cli</a>.</p><h3>Training a TensorFlow Lite Micro model for Arduino</h3><figure><img alt="Gesture classification on Arduino BLE 33 Nano Sense, output as Emojis" src="https://cdn-images-1.medium.com/max/480/0*dniZrqwbvHQcYxOn" /><figcaption><em>Gesture classification on Arduino BLE 33 Nano Sense, output as Emojis</em></figcaption></figure><p>Next, we will use ML to enable the Arduino board to recognise gestures. We’ll capture motion data from the Arduino Nano 33 BLE Sense board, import it into TensorFlow to train a model, and deploy the resulting classifier onto the board.</p><p>The idea for this tutorial was based on Charlie Gerard’s awesome <a href="https://medium.com/@devdevcharlie/play-street-fighter-with-body-movements-using-arduino-and-tensorflow-js-6b0e4734e118"><em>Play Street Fighter with body movements using Arduino and Tensorflow.js</em></a><em>. </em>In Charlie’s example, the board is streaming all sensor data from the Arduino to another machine which performs the gesture classification in Tensorflow.js. We take this further and “TinyML-ify” it by performing gesture classification on the Arduino board itself. This is made easier in our case as the Arduino Nano 33 BLE Sense board we’re using has a more powerful Arm Cortex-M4 processor, and an on-board IMU.</p><p><em>We’ve adapted the tutorial below, so no additional hardware is needed — the sampling starts on detecting movement of the board. The original version of the tutorial adds a breadboard and a hardware button to press to trigger sampling. If you want to get into a little hardware, you can follow that version </em><a href="https://github.com/sandeepmistry/aimldevfest-workshop-2019"><em>instead</em></a><em>.</em></p><h4>Setting up the Arduino IDE</h4><p>Following the steps below sets up the Arduino IDE application used to both upload inference models to your board and download training data from it in the next section. There are a few more steps involved than using Arduino Create web editor because we will need to download and install the specific board and libraries in the Arduino IDE.</p><figure><img alt="Download and install the specific board and libraries in the Arduino IDE." src="https://cdn-images-1.medium.com/max/1024/0*D_EYhyU8LAEpwoxU" /></figure><ul><li>Download and install the Arduino IDE from<strong> </strong><a href="https://arduino.cc/downloads"><strong>https://arduino.cc/downloads</strong></a></li><li>Open the Arduino application you just installed</li><li>In the Arduino IDE menu select<strong> Tools &gt; Board &gt; Boards Manager…</strong></li><li>Search for “Nano BLE” and press install on the board</li><li>It will take several minutes to install</li><li>When it’s done close the Boards Manager window</li></ul><figure><img alt="Boards Manager window" src="https://cdn-images-1.medium.com/max/1024/0*iO86dmc2axTkGOWe" /></figure><ul><li>Now go to the Library Manager <strong>Tools &gt; Manage Libraries…</strong></li><li>Search for and install the <strong>Arduino_TensorFlowLite</strong> library</li><li>Next search for and install the <strong>Arduino_LSM9DS1</strong> library:</li></ul><figure><img alt="Library manager" src="https://cdn-images-1.medium.com/max/1024/0*mVnKwyO_dVf36-YG" /></figure><ul><li>Finally, plug the micro USB cable into the board and your computer</li><li>Choose the board<strong> Tools &gt; Board &gt; Arduino Nano 33 BLE</strong></li><li>Choose the port <strong>Tools &gt; Port &gt; COM5 (Arduino Nano 33 BLE)</strong></li><li>Note that the actual port name may be different on your computer</li></ul><p>There are more detailed <a href="https://www.arduino.cc/en/Guide/NANO33BLESense">Getting Started</a> and <a href="https://www.arduino.cc/en/Guide/Troubleshooting">Troubleshooting</a> guides on the Arduino site if you need help.</p><h4>Streaming sensor data from the Arduino board</h4><p>First, we need to capture some training data. You can capture sensor data logs from the Arduino board over the same USB cable you use to program the board with your laptop or PC.</p><p>Arduino boards run small applications (also called <em>sketches) </em>which are compiled from <em>.ino </em>format Arduino source code, and programmed onto the board using the Arduino IDE or Arduino Create.</p><p>We’ll be using a pre-made sketch <strong>IMU_Capture.ino</strong> which does the following:</p><ul><li>Monitor the board’s accelerometer and gyroscope</li><li>Trigger a sample window on detecting significant linear acceleration of the board</li><li>Sample for one second at 119Hz, outputting CSV format data over USB</li><li>Loop back and monitor for the next gesture</li></ul><p><em>The sensors we choose to read from the board, the sample rate, the trigger threshold, and whether we stream data output as CSV, JSON, binary or some other format are all customizable in the sketch running on the Arduino. There is also scope to perform signal preprocessing and filtering on the device before the data is output to the log — this we can cover in another blog. For now, you can just upload the sketch and get to sampling.</em></p><p>To program the board with this sketch in the Arduino IDE:</p><ul><li>Download <a href="https://github.com/arduino/ArduinoTensorFlowLiteTutorials/blob/master/GestureToEmoji/ArduinoSketches/IMU_Capture/IMU_Capture.ino">IMU_Capture.ino</a> and open it in the Arduino IDE</li><li>Compile and upload it to the board with <strong>Sketch &gt; Upload</strong></li></ul><h4>Visualizing live sensor data log from the Arduino board</h4><p>With that done, we can now visualize the data coming off the board. We’re not capturing data yet — this is just to give you a feel for how the sensor data capture is triggered and how long a sample window is. This will help when it comes to collecting training samples.</p><ul><li>In the Arduino IDE, open the Serial Plotter <strong>Tools &gt; Serial Plotter</strong></li><li>If you get an error that the board is not available, reselect the port:</li><li><strong>Tools &gt; Port &gt; portname (Arduino Nano 33 BLE)</strong></li><li>Pick up the board and practice your punch and flex gestures</li><li>You’ll see it only sample for a one second window, then wait for the next gesture</li><li>You should see a live graph of the sensor data capture (see GIF below)</li></ul><figure><img alt="Arduino IDE Serial Plotter will show a live graph of CSV data output from your board" src="https://cdn-images-1.medium.com/max/480/0*ja9oGdgoQY4YUn0U" /><figcaption><em>Arduino IDE Serial Plotter will show a live graph of CSV data output from your board</em></figcaption></figure><p><em>When you’re done be sure to close the Serial Plotter window — this is important as the next step won’t work otherwise.</em></p><h4>Capturing gesture training data</h4><p>To capture data as a CSV log to upload to TensorFlow, you can use <strong>Arduino IDE &gt; Tools &gt; Serial Monitor</strong> to view the data and export it to your desktop machine:</p><ul><li>Reset the board by pressing the small white button on the top</li><li>Pick up the board in one hand (picking it up later will trigger sampling)</li><li>In the Arduino IDE, open the Serial Monitor <strong>Tools &gt; Serial Monitor</strong></li><li>If you get an error that the board is not available, reselect the port:</li><li><strong>Tools &gt; Port &gt; portname (Arduino Nano 33 BLE)</strong></li><li>Make a punch gesture with the board in your hand (Be careful whilst doing this!)</li><li>Make the outward punch quickly enough to trigger the capture</li><li>Return to a neutral position slowly so as not to trigger the capture again</li><li>Repeat the gesture capture step 10 or more times to gather more data</li><li>Copy and paste the data from the Serial Console to new text file called punch.csv</li><li>Clear the console window output and repeat all the steps above, this time with a flex gesture in a file called flex.csv</li><li>Make the inward flex fast enough to trigger capture returning slowly each time</li></ul><p>Note the first line of your two csv files should contain the fields aX,aY,aZ,gX,gY,gZ</p><figure><img alt="csv file" src="https://cdn-images-1.medium.com/max/954/0*tNTcbfV2w1t0IUIC" /></figure><p><strong>Linux tip: </strong>if you prefer you can redirect the sensor log output from the Arduino straight to a .csv file on the command line. With the Serial Plotter / Serial Monitor windows closed use:</p><pre>$ cat /dev/cu.usbmodem[nnnnn] &gt; sensorlog.csv</pre><h4>Training in TensorFlow</h4><p>We’re going to use <a href="https://colab.research.google.com/">Google Colab</a> to train our machine learning model using the data we collected from the Arduino board in the previous section. Colab provides a Jupyter notebook that allows us to run our TensorFlow training in a web browser.</p><figure><img alt="Arduino gesture recognition training colab" src="https://cdn-images-1.medium.com/max/1024/0*Zgcr73GyPxCwRocn" /><figcaption><em>Arduino gesture recognition training colab</em></figcaption></figure><p>The Colab will step you through the following:</p><ul><li>Setup Python Environment</li><li>Upload the punch.csv and flex.csv data</li><li>Parse and prepare the data</li><li>Build &amp; Train the Model</li><li>Convert the Trained Model to TensorFlow Lite</li><li>Encode the Model in an Arduino Header File</li></ul><p>The final step of the colab is generates the model.h file to download and include in our Arduino IDE gesture classifier project in the next section:</p><figure><img alt="Notebook in Colab" src="https://cdn-images-1.medium.com/max/1024/0*ut8uOXqF06HUASyF" /></figure><p>Let’s open the notebook in Colab and run through the steps in the cells — <a href="https://colab.research.google.com/github/arduino/ArduinoTensorFlowLiteTutorials/blob/master/GestureToEmoji/arduino_tinyml_workshop.ipynb">arduino_tinyml_workshop.ipynb</a></p><h3>Classifying IMU Data</h3><p>Next we will use model.h file we just trained and downloaded from Colab in the previous section in our Arduino IDE project:</p><ol><li>Open <a href="https://github.com/arduino/ArduinoTensorFlowLiteTutorials/blob/master/GestureToEmoji/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino">IMU_Classifier.ino</a> in the Arduino IDE.</li><li>Create a new tab in the IDE. When asked name it model.h</li></ol><figure><img alt="Open IMU_Classifier.ino in the Arduino IDE" src="https://cdn-images-1.medium.com/max/1024/0*y2ZO4B667K8PZI2P" /></figure><p>3. Open the model.h tab and paste in the version you downloaded from Colab</p><p>4. Upload the sketch: <strong>Sketch &gt; Upload</strong></p><p>5. Open the Serial Monitor: <strong>Tools &gt; Serial Monitor</strong></p><p>6. Perform some gestures</p><p>7. The confidence of each gesture will be printed to the Serial Monitor (0 = low confidence, 1 = high confidence)</p><p>Congratulations, you’ve just trained your first ML application for Arduino!</p><figure><img alt="Congratulations you’ve just trained your first ML application for Arduino!" src="https://cdn-images-1.medium.com/max/1024/0*lc93bfEHNJ3nzTgn" /></figure><p>For added fun, the <a href="https://github.com/arduino/ArduinoTensorFlowLiteTutorials/blob/master/GestureToEmoji/ArduinoSketches/Emoji_Button/Emoji_Button.ino">Emoji_Button.ino</a> example shows how to create a USB keyboard that prints an emoji character in Linux and macOS. Try combining the <a href="https://github.com/arduino/ArduinoTensorFlowLiteTutorials/blob/master/GestureToEmoji/ArduinoSketches/Emoji_Button/Emoji_Button.ino">Emoji_Button.ino</a> example with the <a href="https://github.com/arduino/ArduinoTensorFlowLiteTutorials/blob/master/GestureToEmoji/ArduinoSketches/IMU_Classifier/IMU_Classifier.ino">IMU_Classifier.ino</a> sketch to create a gesture controlled emoji keyboard 👊.</p><h3>Conclusion</h3><p>It’s an exciting time with a lot to learn and explore in Tiny ML. We hope this blog has given you some idea of the potential and a starting point to start applying it in your own projects. Be sure to let us know what you build and <a href="https://create.arduino.cc/projecthub">share it</a> with the Arduino community.</p><figure><img alt="TinyML: Machine Learning with TensorFlow on Arduino and Ultra-Low Power Microcontrollers" src="https://cdn-images-1.medium.com/max/1024/1*bi54jstC0uhmAENcgX340w.png" /></figure><p>For a comprehensive background on TinyML and the example applications in this article, we recommend Pete Warden and Daniel Situnayake’s new O’Reilly book “TinyML: Machine Learning with TensorFlow on Arduino and Ultra-Low Power Microcontrollers”</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=7daf95b4157" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/how-to-get-started-with-machine-learning-on-arduino-7daf95b4157">How-to Get Started with Machine Learning on Arduino</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[TensorFlow 2.0 is now available!]]></title>
            <link>https://medium.com/tensorflow/tensorflow-2-0-is-now-available-57d706c2a9ab?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/57d706c2a9ab</guid>
            <category><![CDATA[announcements]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Mon, 30 Sep 2019 19:16:54 GMT</pubDate>
            <atom:updated>2019-09-30T19:16:54.238Z</atom:updated>
            <content:encoded><![CDATA[<p>Earlier this year, we announced TensorFlow 2.0 in alpha at the <a href="https://www.tensorflow.org/dev-summit">TensorFlow Dev Summit</a>. Today, we’re delighted to announce that the final release of TensorFlow 2.0 is now available! Learn how to install it <a href="https://www.tensorflow.org/install">here</a>.</p><p>TensorFlow 2.0 is driven by the community telling us they want an easy-to-use platform that is both flexible and powerful, and which supports deployment to any platform. TensorFlow 2.0 provides a comprehensive ecosystem of tools for developers, enterprises, and researchers who want to push the state-of-the-art in machine learning and build scalable ML-powered applications.</p><iframe src="https://cdn.embedly.com/widgets/media.html?src=https%3A%2F%2Fwww.youtube.com%2Fembed%2FEqWsPO8DVXk%3Ffeature%3Doembed&amp;url=http%3A%2F%2Fwww.youtube.com%2Fwatch%3Fv%3DEqWsPO8DVXk&amp;image=https%3A%2F%2Fi.ytimg.com%2Fvi%2FEqWsPO8DVXk%2Fhqdefault.jpg&amp;key=a19fcc184b9711e1b4764040d3dc5c07&amp;type=text%2Fhtml&amp;schema=youtube" width="854" height="480" frameborder="0" scrolling="no"><a href="https://medium.com/media/669dccf5c26ce313693cde3b7bf0bbd8/href">https://medium.com/media/669dccf5c26ce313693cde3b7bf0bbd8/href</a></iframe><h4><strong>Coding with TensorFlow 2.0</strong></h4><p>TensorFlow 2.0 makes development of ML applications much easier. With tight integration of <a href="https://www.tensorflow.org/guide/keras/overview">Keras into TensorFlow</a>, <a href="https://www.tensorflow.org/guide/eager">eager execution</a> by default, and Pythonic function execution, TensorFlow 2.0 makes the experience of developing applications as familiar as possible for Python developers. For researchers pushing the boundaries of ML, we have invested heavily in TensorFlow’s low-level API: We now export all ops that are used internally, and we provide inheritable interfaces for crucial concepts such as variables and checkpoints. This allows you to build onto the internals of TensorFlow without having to rebuild TensorFlow.</p><figure><img alt="Training and deployment diagram" src="https://cdn-images-1.medium.com/max/960/0*C7GCWYlsMrhUYRYi" /></figure><p>To be able to run models on a variety of runtimes, including the cloud, web, browser, Node.js, mobile and embedded systems, we have standardized on the <a href="https://www.tensorflow.org/guide/saved_model">SavedModel</a> file format. This allows you to run your models with TensorFlow, deploy them with <a href="https://www.tensorflow.org/tfx/guide/serving">TensorFlow Serving</a>, use them on mobile and embedded systems with <a href="https://www.tensorflow.org/lite">TensorFlow Lite</a>, and train and run in the browser or Node.js with <a href="https://www.tensorflow.org/js">TensorFlow.js</a>.</p><p>For high-performance training scenarios, you can use the <a href="https://www.tensorflow.org/guide/distributed_training">Distribution Strategy API</a> to distribute training with minimal code changes and attain great out-of-the-box performance. It supports distributed training with Keras’ Model.fit and also supports custom training loops. <a href="https://www.tensorflow.org/guide/gpu">Multi-GPU support</a> is now available, and you can learn more about using GPUs on Google Cloud <a href="https://cloud.google.com/compute/docs/gpus/">here</a>. Cloud TPU support is coming in a future release. Check out the distributed training <a href="https://www.tensorflow.org/guide/distributed_training">guide</a> for more details.</p><p>TensorFlow 2.0 offers many performance improvements on GPUs. TensorFlow 2.0 delivers up to 3x faster training performance using mixed precision on Volta and Turing GPUs with a few lines of code, used for example in <a href="https://github.com/tensorflow/models/tree/master/official/resnet">ResNet-50</a> and <a href="https://github.com/tensorflow/models/tree/master/official/nlp/bert">BERT</a>. TensorFlow 2.0 is tightly integrated with TensorRT and uses <a href="https://github.com/tensorflow/tensorflow/blob/r2.0/tensorflow/python/compiler/tensorrt/trt_convert.py#L766-L859">an improved API</a> to deliver better usability and high performance during inference on <a href="https://cloud.google.com/compute/docs/tutorials/ml-inference-t4">NVIDIA T4 Cloud GPUs</a> on Google Cloud.</p><p>“Machine learning on NVIDIA GPUs and systems allows developers to solve problems that seemed impossible just a few years ago,” said Kari Briski, Senior Director of Accelerated Computing Software Product Management at NVIDIA. “TensorFlow 2.0 is packed with many great GPU acceleration features, and we can’t wait to see the amazing AI applications the community will create with these updated tools.”</p><p>Effective access for training and validation data is paramount to being effective when building models in TensorFlow. We introduced <a href="https://www.tensorflow.org/guide/data">TensorFlow Datasets</a>, giving a standard interface to a plethora of datasets containing a variety of data types such as images, text, video, and more.</p><p>While the traditional Session-based programming model is still maintained, we recommend using regular Python development with eager execution. The <a href="https://www.tensorflow.org/tutorials/customization/performance">tf.function</a> decorator can be used to convert your code into graphs which can be executed remotely, serialized, and optimized for performance. This is complemented by <a href="https://www.tensorflow.org/guide/function">Autograph</a>, which can convert regular Python control flow directly into TensorFlow control flow.</p><p>And of course, if you have used TensorFlow 1.x and are looking for a migration guide to 2.0, we have published one <a href="https://www.tensorflow.org/guide/migrate">here</a>. The TensorFlow 2.0 release also includes an automatic conversion script to help get you started.</p><p>We have worked with many users inside Google and from the TensorFlow community to test TensorFlow 2.0 features and have been thrilled with the feedback. As an example, the Google News team launched a BERT-based language understanding model in TensorFlow 2.0 that <a href="https://www.blog.google/products/news/new-google-news-ai-meets-human-intelligence/">significantly improved story coverage</a>. TensorFlow 2.0 provides easy-to-use APIs with the flexibility to implement new ideas quickly. Model training and serving was seamlessly integrated into the existing infrastructure.</p><p>Also, ML isn’t just for Python developers — using <a href="https://www.tensorflow.org/js">TensorFlow.js,</a> training and inference is available to JavaScript developers, and we continue to invest in Swift as a language for building models with the <a href="https://www.tensorflow.org/swift">Swift for TensorFlow</a> library.</p><p>There’s a lot to unpack here, so to assist, we’ve created a handy guide on how to be <a href="https://www.tensorflow.org/guide/effective_tf2">effective with all that’s new in TensorFlow 2.0</a>. To make it even easier to get started with TensorFlow 2.0, we are releasing reference implementations of several commonly used ML models using the 2.0 API <a href="https://www.tensorflow.org/resources/models-datasets">here</a>.</p><p>Additionally, to learn how to build applications using TensorFlow 2.0, check out the online courses we have created together with <a href="https://www.coursera.org/learn/introduction-tensorflow">deeplearning.ai</a> and <a href="https://www.udacity.com/course/intro-to-tensorflow-for-deep-learning--ud187">Udacity</a>.</p><p>To get started quickly try <a href="https://cloud.google.com/deep-learning-vm/">Google Cloud’s Deep Learning VM Images</a> — no setup required, pre-configured virtual machines to help you build your TensorFlow 2.0 deep learning projects.</p><h4><strong>Learn More</strong></h4><p>Head over to <a href="https://www.tensorflow.org/">tensorflow.org</a> for more about TensorFlow 2.0, including how to download and get started with coding ML applications. Finally, for more exciting announcements, talks, and hands-on training and presentations about TensorFlow 2.0 and its ecosystem, please join us in person at <a href="http://tensorflow.world/">TensorFlow World</a> on Oct 28–31 in Santa Clara, CA. We hope to see you there!</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=57d706c2a9ab" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/tensorflow-2-0-is-now-available-57d706c2a9ab">TensorFlow 2.0 is now available!</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Using TensorFlow to predict product weight and dimensions]]></title>
            <link>https://medium.com/tensorflow/using-tensorflow-to-predict-product-weight-and-dimensions-8e6af3ab3466?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/8e6af3ab3466</guid>
            <category><![CDATA[tensorflow-core]]></category>
            <category><![CDATA[developer-stories]]></category>
            <category><![CDATA[keras]]></category>
            <category><![CDATA[nlp]]></category>
            <category><![CDATA[regression]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Wed, 25 Sep 2019 17:01:01 GMT</pubDate>
            <atom:updated>2019-09-25T22:35:54.720Z</atom:updated>
            <content:encoded><![CDATA[<p><em>A Guest Post by Rodolfo Bonnin from the Mercado Libre Applied Machine Learning team</em></p><figure><img alt="Mercado libre banner" src="https://cdn-images-1.medium.com/max/640/0*K5llZP4xxf_GmvZF" /></figure><h4><strong>Introduction</strong></h4><p><a href="https://www.mercadolibre.com/">Mercado Libre</a> is the leading marketplace platform in Latin America, reaching millions of users selling and buying tens of millions of different items a day. From a shipping perspective, one of the most important pieces of information for an item is dimensions and weight since they are used for predicting costs and forecasting the occupancy of the fulfillment centers. As a user-driven marketplace, this information is not always available, and we have found that we can optimize our pipeline by predicting it in advance.</p><figure><img alt="Initial listing submission screen" src="https://cdn-images-1.medium.com/max/1024/0*1xHTR47-aUL8AzjS" /><figcaption><em>Our initial listing submission screen does not ask for measurements or weight</em></figcaption></figure><p>Mercado Libre is a general marketplace, so an item could be either a new cellphone (between 7 - 14 ounces, and approximately 5″ x 5″ x 2″), or a used dishwasher weighing 175 pounds and measuring 24″ x 24″ x 35″.</p><p>Initially, we implemented a lookup mechanism, which consisted of determining measure and weight based on previous measurements of the item in our fulfillment, cross-docking, and validating third party carriers, and after an outlier filtering process. Of course, this process had low coverage (the percentage of items already measured was low).</p><p>As a second measure, we built simple models based on statistical summaries of single product features, like category and brand/model, and used product similarity metrics to map new items to existing ones. These methods had poor accuracy in general, and there were a large number of cases where no prediction was possible because there weren’t adequate samples.</p><p>The need arose for a regression model, able to infer measurements from information like the item title, description, category, and a few other attributes. Importantly, these attributes include both structured and unstructured data, which are difficult to model with traditional statistical methods. The model predicts and saves 3k predictions a minute (4M a day), and the API serves (mostly pre-saved) 200k requests a minute (288M a day).</p><h4><strong>Data and features</strong></h4><p>When preparing the next iteration, after reviewing the listing publication process, we had the certainty that at least the listing title, item description (free text), along with the category and the product brand were likely to be key features for the model. Of course, the data had to be cleaned up and prepared to be useful for our project. The software stack for the ETL part of the project, was based on scipy, pandas, numpy and scikit-learn. Once we had decided on features, it was time to vectorize the data.</p><figure><img alt="Composition of the model’s feature union" src="https://cdn-images-1.medium.com/max/811/0*AB4xb8BV4Cs4fsoX" /><figcaption><em>Composition of the model’s feature union</em></figcaption></figure><p>To encode the two main text based features, we tokenized and lower cased all the words. Next, we needed to find a suitable vector representation. For this first iteration of this project, we decided to experiment with a method using keywords and TF-idf, rather than an LSTM which we will explore in the future. TF-idf is a vectorization technique, applied to text data, which converts tokenized documents into a matrix.</p><h4><strong>The Model</strong></h4><p>The model we used was a Multi-Layer Perceptron, implemented in keras with the Tensorflow backend. Hyperparameters and architecture details were chosen by performing a grid search on learning rate, dropout, number of layers, number of kernels and kernel dimensions (we now know that random search is a much better approach).</p><p>Regarding the loss function, we tried several experiments to see which one was more appropriate:</p><p>The <strong>mean_squared_error</strong> provided a measure that, even when heavily penalizing the errors, overshot to fit outliers. The <strong>mean_absolute_percentage_error</strong> preserved especially the smaller items zone from outliers (with the lower 1Kg covering 50% of the items). The <strong>mean_absolute_error</strong> was useful for general and broad optimization, across all item classes.</p><figure><img alt="Weight error distribution for the different types of loss function" src="https://cdn-images-1.medium.com/max/1024/0*fmR6yNOnsIhSkUIL" /><figcaption><em>Weight error distribution for the different types of loss function</em></figcaption></figure><figure><img alt="The model" src="https://cdn-images-1.medium.com/max/1024/0*ftsbXZfZSXKyDyr_" /><figcaption>General architecture of the neural network model</figcaption></figure><h4><strong>Serving predictions</strong></h4><p>The model is currently being used in the product submission page and the fulfillment center packaging calculator, to forecast the occupancy of the shelves and the right envelope to use at the final product packaging stage.</p><p>The API currently serves 3k predictions a minute or 4M a day. The inference time for our model is 30ms and we serve it on a 64GB RAM + 6 cores instance with 13 ~900MB workers.</p><p>Here is an example Json response:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/9cf5bc56d31bb2bd025591d04654423a/href">https://medium.com/media/9cf5bc56d31bb2bd025591d04654423a/href</a></iframe><p>Moving forward, we plan to run additional experiments to optimize model quality and to refine the results based on feedback from the fulfillment center. Working with TensorFlow enabled us to rapidly design, iterate, and deploy models to production.</p><h4><strong><em>Acknowledgments</em></strong>:</h4><ul><li>The whole Pymes Team, especially Mirko Panozzo, Conrado García Berrotarán, Kevin Clemoveki, Diego Piloni, Martín Ciruzzi and Esteban Tundidor.</li><li>Francisco Ingham for helping with the grammar, style and contents.</li><li>Joaquin Verdinelli for coordinating the redaction and communication with Google.</li><li>Constanza Stahl for helping with the graphics.</li><li>Cynthia Bustamante for general coordination help</li></ul><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=8e6af3ab3466" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/using-tensorflow-to-predict-product-weight-and-dimensions-8e6af3ab3466">Using TensorFlow to predict product weight and dimensions</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Body movement recognition in the ‘Smart Baduanjin’ App]]></title>
            <link>https://medium.com/tensorflow/body-movement-recognition-in-the-smart-baduanjin-app-2a4e2d5159c8?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/2a4e2d5159c8</guid>
            <category><![CDATA[olivex]]></category>
            <category><![CDATA[tensorflow]]></category>
            <category><![CDATA[ai]]></category>
            <category><![CDATA[posenet]]></category>
            <category><![CDATA[tflite]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Mon, 23 Sep 2019 17:01:01 GMT</pubDate>
            <atom:updated>2019-09-23T17:01:01.414Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Guest post by Keith Chan and Vincent Zhang from OliveX Engineering team</em></p><h3>Introduction</h3><p><a href="https://www.olivex.ai/">OliveX</a> is a Hong Kong-based company focused on fitness-related software, serving more than 2 million users since we first launched in 2018. Many of our users are elderly and our Baduanjin app helps them practice Baduanjin while minimizing the possibility of injury. To achieve that, we utilize the latest artificial intelligence technology in our app to automatically detect Baduanjin practicing moves and provide corresponding feedback to our users.</p><h3>Goals and requirements</h3><p>Baduanjin is a popular exercise that consists of eight kinds of limb movements and controlled breathing. Precise movements and control are critical for Baduanjin, which improves the health of the body both mentally and physically.</p><figure><img alt="Animated person performing body movements" src="https://cdn-images-1.medium.com/max/1024/0*2LvOTf1WNQ7J0Y33" /></figure><p>By using the “Smart Baduanjin” app, users can determine if they are performing the moves correctly by using AI to track their movements. By leveraging the latest machine learning technology, we hope to replace the traditional learning approach in which users simply follow an exercise video with a more enjoyable interactive experience in which users get feedback on their body movements in real time. We also hope that these features could help the elderly to practice Baduanjin more effectively and reduce the risk of injury.</p><figure><img alt="Stills from the ‘Smart Baduanjin’ app" src="https://cdn-images-1.medium.com/max/1024/1*Us5wKti-jB4ZrcIAQHf2wg.png" /></figure><p>After analyzing and prioritizing our goals, we defined the overall product requirements:</p><ol><li>Our uses usually perform Baduanjin in an outdoor setting, so our product needs to be ‘mobile’</li><li>When a user practices Baduanjin, he/she usually needs to follow a demo video and imitate the movements from a coach, so our product should be able to play videos with sound</li><li>To provide valuable real time feedback to our users, we need to capture user body movements with the front-facing camera</li></ol><p>Specifically regarding the body movement recognition, we want our algorithms to be able to do the following:</p><ol><li>Recognize each Baduanjin movement in the overall sequence</li><li>Give scores based on the correctness of a user’s body movements</li><li>Provide correctional guidance on unsatisfactory movements</li></ol><h3>Technical analysis</h3><h4>ML framework selection</h4><p>Based on the requirements above, we needed to select the right deep learning framework to implement our project. In particular, we were looking for a framework that has:</p><ol><li>Strong support for mobile devices and can run smoothly, even on mid- to low-end smartphones</li><li>A friendly API design and rich debugging tools</li><li>Mature community support and resources available</li></ol><p>After comprehensive investigation and evaluation, we found that TensorFlow Lite satisfied our needs and requirements. In addition, Google open sourced <a href="https://github.com/tensorflow/tfjs-models/tree/master/posenet">PoseNet</a>, an app specifically designed for detecting human body poses, and provided demo code based on TensorFlow.js (EDITOR: we have recently released a <a href="https://medium.com/tensorflow/track-human-poses-in-real-time-on-android-with-tensorflow-lite-e66d0f3e6f9e">PoseNet sample based on TensorFlow Lite</a>). Google not only helped us finish the initial work for human body pose recognition with the help of the open source code, but also convinced us that our action recognition algorithm could work on mobile devices, since the performance on JavaScript was so good.</p><h3>Algorithms</h3><h4>Body movement recognition</h4><p>In the initial stage of our development, we investigated existing body movement recognition algorithms. Currently, mainstream algorithms are mostly based on analyzing sequential video frames. Although these algorithms could meet our requirements, the networks are fairly complicated and running inference on them consumes a lot of computational resources. But since one of our hard requirements was to run the model on mobile devices, we had to make a tradeoff between accuracy and performance.</p><p>Our approach was to first acquire key human body joints through PoseNet, and then recognize specific actions based on the sequence of body joint movement. Since PoseNet tracks only 17 body joints, the amount of computation was significantly reduced, as compared to the full size image. Below we show the workflow of our algorithm: first we used PoseNet to extract body joint data from input video, and then we carried out action classifications based on the body joint data.</p><figure><img alt="How to acquire human body joints using PoseNet" src="https://cdn-images-1.medium.com/max/1024/0*l2-0uiXDWkFT8Ss-" /></figure><h4>Key actions</h4><p>After deciding on the technical approach, we needed to define the key body movements important to recognize for the app. To do this, we transformed the body movement recognition problem to a typical machine learning classification problem. Below is a sample of how we defined the key Baduanjin movements:</p><figure><img alt="Chart of key Baduanjin movements" src="https://cdn-images-1.medium.com/max/1024/0*CFKMKk67EVUmkEQw" /></figure><p>We trained a traditional deep neural network to classify user body movements. After several iterations of hyperparameter tuning and training data optimization/augmentation, our final model had good accuracy and was able to satisfy our product requirements, as shown below.</p><figure><img alt="Accuracy vs. Gesture graph" src="https://cdn-images-1.medium.com/max/1024/0*B7z0TObHpPYW7Rp7" /></figure><h4>Challenges on mobile devices</h4><p>After finishing the deep learning model, our next step was to deploy our models on iOS and Android mobile devices. At first, we tried TensorFlow Mobile. But since we needed to get recognition results in real time, TensorFlow Mobile was not a viable option since its performance did not meet this requirement.</p><p>As we were trying to solve the performance challenge, Google released <a href="https://www.tensorflow.org/lite">TensorFlow Lite</a>, which was a big leap from TensorFlow Mobile in terms of performance. We compare the two offerings below:</p><figure><img alt="Table comparing TensorFlow Mobile and TensorFlow Lite" src="https://cdn-images-1.medium.com/max/601/0*smCg1K44hK5bbFS3" /></figure><p>We also show the initial benchmark results on our model below:</p><figure><img alt="Initial benchmark results" src="https://cdn-images-1.medium.com/max/601/0*P2-6GKD-8JATDM_F" /></figure><p>Based on the benchmark numbers, we concluded that it wasn’t feasible to do body movement recognition in real time on most Android devices based on a 512x512 input size. To solve this problem, we profiled our machine learning models and found that PoseNet was the bottleneck, which consumed 95% of the computational time. So we adjusted PoseNet input size and hyperparameters, and retrained our action recognition algorithm to compensate for the accuracy loss introduced by reduced input size. In the end, we chose 337x337 RGB as the input and 0.5 as the width multiplier for MobileNet on Android.</p><p>Our target users are mostly elderly, so they tend to have lower-end devices. Even though we improved the performance by adjusting PoseNet parameters, it was still not satisfactory. Therefore, we resorted to the ubiquitous accelerator in smartphones: GPU. Google happened to release <a href="https://www.tensorflow.org/lite/performance/gpu">TensorFlow Lite GPU delegate (experimental)</a> around that time, which saved us plenty of engineering resources.</p><p>Mobile GPU considerably accelerated our model execution. Below is our benchmark on several popular devices among our users, with and without GPU delegate.</p><figure><img alt="Benchmark on several popular devices among our users, with and without GPU delegate." src="https://cdn-images-1.medium.com/max/600/0*l3UAaZ3K_DpBWTJ-" /></figure><p>Since Baduanjin moves are relatively slow for older users, our product could run smoothly on most devices after integrating TensorFlow Lite GPU delegate (experimental).</p><figure><img alt="Animated Banduanjin movement" src="https://cdn-images-1.medium.com/max/393/0*3Cec85GK3VvuMLVU" /></figure><h3>Conclusion</h3><p>We successfully finished our Smart Baduanjin product on both iOS and Android, and have received positive feedback from our testing users. By leveraging ML technology and TensorFlow, we provided a ‘tutorial model’ for Baduanjin beginners so that they can follow the demo video and learn the moves. For experienced Baduanjin practitioners, we provided valuable feedback such as scores to help them further improve their skills. Currently, ‘Smart Baduanjin’ is available for free in both the <a href="https://apps.apple.com/hk/app/%E6%99%BA%E8%83%BD%E5%85%AB%E6%AE%B5%E9%8C%A6/id1410056117">App Store</a> and <a href="https://play.google.com/store/apps/details?id=com.familyfit.google.baduanjin&amp;hl=en_US">Google Play</a>.</p><p>Meanwhile, OliveX is actively exploring applying human body pose estimation to other fitness exercises. We found that many other exercise practices are just like Baduanjin, in that the correctness of the practitioners’ body movements matters a lot. Correct body movements not only help people avoid bodily injury, but also increase their exercise efficiency. Thus, we want to transfer the knowledge and skills we obtained from the Baduanjin project to other domains as well.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=2a4e2d5159c8" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/body-movement-recognition-in-the-smart-baduanjin-app-2a4e2d5159c8">Body movement recognition in the ‘Smart Baduanjin’ App</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[Creating a Custom TFX Executor]]></title>
            <link>https://medium.com/tensorflow/creating-a-custom-tfx-executor-28edaa3604f6?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/28edaa3604f6</guid>
            <category><![CDATA[tfx]]></category>
            <category><![CDATA[tensorflow]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Thu, 19 Sep 2019 17:01:01 GMT</pubDate>
            <atom:updated>2019-09-19T17:01:01.373Z</atom:updated>
            <content:encoded><![CDATA[<p><em>Posted by </em><a href="https://www.linkedin.com/in/kevinhaas/"><em>Kevin Haas</em></a><em>, </em><a href="https://www.linkedin.com/in/zhitao-li-5520a821/"><em>Zhitao Li</em></a><em>, and </em><a href="https://twitter.com/robert_crowe"><em>Robert Crowe</em></a><em> on behalf of the TFX team</em></p><p>TensorFlow Extended (TFX) is a platform for creating production-ready ML pipelines. TFX was created by Google and provides the backbone of Google’s ML services and applications, and we’ve been open sourcing TFX for everyone who needs to create production ML pipelines.</p><figure><img alt="TFX pipeline" src="https://cdn-images-1.medium.com/max/948/0*5_LeOofdfAR6IX09" /></figure><p>TFX can be extended and customized in several ways, including developing new components and including them in your pipeline. A TFX pipeline is a series of TFX components, each of which performs a separate task, which are sequenced as a directed acyclic graph (DAG). In this post we’ll present an example to illustrate the process of developing a new <a href="https://www.tensorflow.org/tfx/guide#anatomy_of_a_component">TFX component</a>. Watch for more posts that will discuss additional ways to extend and customize TFX.</p><h3>Scenario</h3><p>We want to replace the behavior of the existing <a href="https://www.tensorflow.org/tfx/guide/trainer">TFX Trainer component</a> with a new component that includes an <em>executor</em> which submits a job to run the same training on the <a href="https://cloud.google.com/">Google Cloud Platform (GCP)</a>. Since the upstream and downstream semantics will remain unchanged, we will reuse the existing Trainer component and replace the behavior of the executor.</p><h3>Anatomy of a Component</h3><figure><img alt="Anatomy of a component" src="https://cdn-images-1.medium.com/max/600/1*9fwUFzc4OSaVgpQuJm8F9g.png" /></figure><p>TFX components consist of three main pieces:</p><ul><li>Driver</li><li>Executor</li><li>Publisher</li></ul><h4>Driver and Publisher</h4><p>The driver supplies metadata to the executor by querying the <a href="https://www.tensorflow.org/tfx/guide/mlmd">ML Metadata (MLMD)</a> store and the publisher takes the results of the executor and updates the metadata store. As a developer, you will typically not need to interact with the driver and publisher directly, but messages logged by the driver and publisher may be <a href="https://www.tensorflow.org/tfx/guide#troubleshooting">useful during debugging</a>.</p><h4>Executor</h4><p>The executor is where a component performs its processing. As a developer you write code which runs in the executor, based on the requirements of the classes which implement the type of component that you’re working with. For example, when you’re working on a Transform component you will need to develop a preprocessing_fn. Executors consume and create artifacts, which are kept in the metadata store.</p><h3>Adding a custom executor</h3><h4>Creating a custom executor</h4><p>To create the custom executor, we start with a copy of the current Trainer <a href="https://github.com/tensorflow/tfx/blob/master/tfx/components/trainer/executor.py">executor</a> and then make the modifications to our <a href="https://github.com/tensorflow/tfx/blob/master/tfx/extensions/google_cloud_ai_platform/trainer/executor.py">custom executor</a> to initiate a training job on <a href="https://cloud.google.com/ai-platform/">Google Cloud AI Platform</a>. Much of the basic executor structure will remain the same as the inputs, outputs, and execution parameters will be the same. The changes will be in how the inputs are processed and the outputs are generated. This is achieved by creating a new Executor class that extends<em> tfx.components.base.base_executor.BaseExecutor</em> and implements <em>Do()</em>.</p><pre>class Executor(base_executor.BaseExecutor):<br>  &quot;&quot;&quot;Start a trainer job on Google Cloud AI Platform.&quot;&quot;&quot;<br><br>  def Do(self, input_dict,<br>         output_dict,<br>         exec_properties):<br>    &quot;&quot;&quot;Starts a trainer job on Google Cloud AI Platform.</pre><p>Don’t forget to test it before moving on to the next step! We have created a convenience <a href="https://github.com/tensorflow/tfx/blob/master/tfx/scripts/run_executor.py">script</a> for you to try out your executor before putting it into production. You should write similar code to exercise unit tests for your code. As with any production software deployment, when developing for TFX you should make sure to have good test coverage and a strong CI/CD framework.</p><h4>Override the executor used by the Trainer component</h4><p>In order to do this, we will replace the <a href="https://github.com/tensorflow/tfx/blob/master/tfx/components/trainer/executor.py">default trainer executor </a>used by TFX with the new <a href="https://github.com/tensorflow/tfx/blob/master/tfx/extensions/google_cloud_ai_platform/trainer/executor.py">custom executor</a> which will create the training job on Google Cloud AI Platform. This is achieved with the optional <em>executor_class </em>component parameter.</p><pre>from tfx.extensions.google_cloud_ai_platform.trainer<br>import executor as ai_platform_trainer_executor<br>...<br>trainer = Trainer(<br>    ...,<br>    executor_class=ai_platform_trainer_executor.Executor,<br>}</pre><p>That’s it! Now when the Trainer component is called by the workflow engine, it will run the custom executor instead of the default executor, while creating and consuming the same <a href="https://www.tensorflow.org/tfx/guide/mlmd">ML Metadata</a> artifacts as the default executor.</p><h4>Pass the component arguments to your trainer</h4><p>TFX executors are self-contained binaries focused on running a single step of the ML pipeline. Custom executors require the same three parameters as all other TFX executors: <em>input_dict, output_dict, exec_properties</em>. More details on the semantics of these parameters can be found in the <a href="https://github.com/tensorflow/tfx/blob/master/tfx/components/base/base_executor.py#L32"><em>BaseExecutor</em></a> class.</p><p>When processing data flowing through your TFX pipeline you will often typically want to read input data from the artifact URIs in your <em>input_dict</em>, and often you might want to write your output to artifact URIs from your <em>output_dict</em>. This may include reading and writing to more than one split, as in the case of processing with train and eval splits.</p><pre>from tfx.types import artifact_utils<br><br>train_input_examples_uri = artifact_utils.get_split_uri(<br>  input_dict[&#39;my_input_data&#39;], &#39;train&#39;)<br>eval_input_examples_uri = artifact_utils.get_split_uri(<br>  input_dict[&#39;my_input_data&#39;], &#39;eval&#39;)<br><br>train_output_examples_uri = artifact_utils.get_split_uri(<br>  output_dict[&#39;my_output_data&#39;], &#39;train&#39;)<br>eval_output_examples_uri = artifact_utils.get_split_uri(<br>  output_dict[‘my_output_data&#39;], &#39;eval&#39;)</pre><p>In the example above the dictionary keys <em>my_input_data </em>and <em>my_output_data </em>are defined in the ComponentSpec for the component that you’re overriding the executor for.</p><pre>class MyComponentSpec(tfx.types.ComponentSpec):<br>  PARAMETERS = {<br>      &lt;...&gt;<br>  }<br>  INPUTS = {<br>      &#39;my_input_data&#39;:   <br>      ChannelParameter(type=standard_artifacts.Examples),<br>  }<br>  OUTPUTS = {<br>      &#39;my_output_data&#39;:<br>      ChannelParameter(type=standard_artifacts.Examples),                                                            <br>  }</pre><p>The splits are defined in the output Channel for the component that you’re overriding the executor for, typically in the constructor:</p><pre>output_data = tfx.types.Channel(<br>  type=standard_artifacts.Examples,<br>    artifacts=[<br>      standard_artifacts.Examples(split=split)<br>      for split in artifact.DEFAULT_EXAMPLE_SPLITS<br>    ])<br>spec = LabelingComponentSpec(<br>  input_data=input_data,<br>  my_output_data=output_data)</pre><p>Additional parameters are passed to your custom trainer executor using a <em>custom_config dict</em>. These can be retrieved by the custom executor using <em>exec_properties.get(‘custom_config’).get(‘your_config_key’).</em> In the example below, all of the additional arguments needed to submit a <a href="https://cloud.google.com/ml-engine/docs/tensorflow/training-overview">Google Cloud AI Platform training job </a>can be found in <em>_ai_platform_training_args</em>.</p><pre>_ai_platform_training_args = {<br>   &#39;pythonModule&#39;: None,  # Will be populated by TFX<br>   &#39;args&#39;: None,  # Will be populated by TFX<br>   &#39;region&#39;: _gcp_region,<br>   &#39;jobDir&#39;: os.path.join(_output_bucket, &#39;tmp&#39;),<br>   &#39;project&#39;: ‘your GCP project id’<br>}<br><br>...<br>trainer = Trainer(<br>    ...,<br>    custom_config={&#39;ai_platform_training_args&#39;:<br>    _ai_platform_training_args})   <br>}</pre><h4>Linking the custom trainer’s output to the expected output artifact</h4><p>Hooking up the custom trainer to emit the expected outputs is essential to the success of downstream components. For the Google Cloud AI Platform custom trainer, we serialize the executor input parameters so they can be transmitted as part of the GCP training job. Because the Google Cloud AI Platform (CAIP) executor is redirecting the default TFX executor to run on Google Cloud AI Platform, both take the same {<em>transformed examples, a transform_fn, </em>and a<em> schema</em>} input parameters to create a TF model. The custom executor used in this example submits a CAIP training job that will invoke (via <a href="https://github.com/tensorflow/tfx/blob/master/tfx/scripts/run_executor.py">run_executor.py</a>) the default TFX trainer as the CAIP python module, effectively opening a conduit from the local workstation to run the TFX trainer on CAIP.</p><pre># Configure Google Cloud AI Platform job<br>    training_inputs = exec_properties.get(&#39;custom_config&#39;,<br>        {}).pop(&#39;ai_platform_training_args&#39;)<br>    executor_class_path = &#39;%s.%s&#39; % <br>        (tfx_trainer_executor.Executor.__module__,<br>        tfx_trainer_executor.Executor.__name__)<br><br>    # Start Google Cloud AI Platform job<br>    return runner.start_cmle_training(input_dict, output_dict,<br>        exec_properties, executor_class_path, training_inputs)</pre><h3>Running the pipeline remotely with your custom executor</h3><p>So far we’ve been assuming that your pipeline is running locally, using code available in your $PYTHONPATH. An upcoming blog post will explain how to execute custom executors packaged in containers, or as PyPI packages.</p><h3>Related topics</h3><p>In addition to Trainer, the <a href="https://github.com/tensorflow/tfx/blob/master/docs/guide/examplegen.md#custom-examplegen">TFX ExampleGen component</a> also supports executor-level customization. ExampleGen provides a generic component and a base executor which apply ML best practices, e.g. data shuffling and consistent/configurable partitions.</p><p>If existing ExampleGen components don’t meet your needs, create a new <a href="https://beam.apache.org/">Apache Beam</a> PTransform for handling the conversion from an input split to TF examples and TFX will do the rest. The <a href="https://www.tensorflow.org/tfx/guide/examplegen">ExampleGen doc</a> has more details.</p><h3>For more information</h3><p>To learn more about TFX check out the <a href="https://www.tensorflow.org/tfx">TFX website</a>, join the <a href="https://groups.google.com/a/tensorflow.org/forum/#!forum/tfx">TFX discussion group</a>, and watch our <a href="https://goo.gle/2xVkwt4">TFX playlist on YouTube</a>, and <a href="https://goo.gle/2WtM7Ak">subscribe</a> to the TensorFlow channel.</p><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=28edaa3604f6" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/creating-a-custom-tfx-executor-28edaa3604f6">Creating a Custom TFX Executor</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
        <item>
            <title><![CDATA[DeepPavlov: an open-source library for end-to-end dialog systems and chatbots]]></title>
            <link>https://medium.com/tensorflow/deeppavlov-an-open-source-library-for-end-to-end-dialog-systems-and-chatbots-31cf26849e37?source=rss-b1d410cb9700------2</link>
            <guid isPermaLink="false">https://medium.com/p/31cf26849e37</guid>
            <category><![CDATA[chatbots]]></category>
            <category><![CDATA[artificial-intelligence]]></category>
            <category><![CDATA[python]]></category>
            <category><![CDATA[machine-learning]]></category>
            <category><![CDATA[programming]]></category>
            <dc:creator><![CDATA[TensorFlow]]></dc:creator>
            <pubDate>Wed, 18 Sep 2019 17:01:01 GMT</pubDate>
            <atom:updated>2019-09-18T20:53:24.201Z</atom:updated>
            <content:encoded><![CDATA[<h3><strong><em>DeepPavlov: An open-source library for end-to-end dialogue systems and chatbots</em></strong></h3><p><em>A guest post by </em><a href="https://twitter.com/_vaskon_"><em>Vasily Konovalov</em></a></p><figure><img alt="DeepPavlov images compilation" src="https://cdn-images-1.medium.com/max/1024/0*_qN8onPn7t-cj8Lj" /></figure><p>Dialogue systems have recently become a standard in human-machine interaction, with chatbots appearing in almost every industry to simplify the interaction between people and computers. They can be integrated into websites, messaging platforms, and devices. Chatbots are on the rise, and companies are choosing to delegate routine tasks to chatbots rather than humans, thus providing huge labor cost savings. Unlike humans, chatbots are capable of processing multiple user requests at a time and are always available.</p><p>However, many companies don’t know where to start when developing a bot to meet their business needs. Historically, chatbots can be divided into two large groups: rule-based and data-driven. The former relies on predefined commands and templates. Each of these commands should be written by a chatbot developer using regular expressions and textual data analysis. By contrast, data-driven chatbots rely on machine learning models pretrained on dialogue data.</p><p>In this article I will explain how to develop chatbots with <a href="https://github.com/deepmipt/DeepPavlov">DeepPavlov</a>, and why <a href="https://tensorflow.org">TensorFlow</a> is an indispensable tool in doing so. DeepPavlov is built and maintained by the Neural Networks and Deep Learning Lab of the <a href="https://mipt.ru/">Moscow Institute of Physics and Technology</a>. DeepPavlov is a winning submission to the <a href="https://devpost.com/software/deeppavlov-eio49m">#PoweredByTF 2.0 Challenge</a>.The code for this article can be accessed on <a href="https://colab.research.google.com/github/deepmipt/dp_notebooks/blob/master/DP_tf.ipynb">Google Colab</a>.</p><h3>Architecture of dialogue systems</h3><p>To keep things simple, let us start with the most basic elements of dialogue systems. First, a chatbot needs to understand utterances in a natural language. The <a href="https://en.wikipedia.org/wiki/Natural-language_understanding">Natural Language Understanding (NLU)</a> module translates a user query from natural language into a labeled semantic representation. For example, the utterance <em>“Please set an alarm for 8am”</em> will be translated into a machine-understandable form like <strong>set_alarm(8 am)</strong>. Then the bot has to decide what is expected of it. The <a href="https://en.wikipedia.org/wiki/Dialog_manager">Dialogue Manager (DM)</a> keeps track of the dialogue state and decides what to answer to the user. At the last stage, the <a href="https://en.wikipedia.org/wiki/Natural-language_generation">Natural Language Generator (NLG)</a> translates a semantic representation back into human language. For example,<strong> rent_price(Atlanta)=3000 USD</strong> translates to <em>“The rent price in Atlanta is around $3,000</em>.<em>”</em> The picture below shows a typical dialogue system architecture.</p><figure><img alt="Typical dialogue system architecture" src="https://cdn-images-1.medium.com/max/666/0*D-3poKw--9Rlppld" /></figure><h3>Building dialogue systems with DeepPavlov</h3><p>The open-source conversational AI framework <a href="https://deeppavlov.ai/?utm_source=medium&amp;utm_medium=article&amp;utm_campaign=tf">DeepPavlov</a> offers a free and easy-to-use solution for building dialogue systems. DeepPavlov comes with several predefined components for solving NLP-related problems. The framework allows you to train and test models, as well as fine-tune hyperparameters. It supports Linux and Windows platforms, Python 3.6, and Python 3.7. You can install DeepPavlov by running:</p><pre>pip install -q deeppavlov</pre><p>The DeepPavlov models are defined in separate configuration files under the config folder. A config file consists of five main sections: <strong>dataset_reader</strong>, <strong>dataset_iterator</strong>, <strong>chainer</strong>, <strong>train</strong>,<strong> </strong>and<strong> metadata</strong>. The <strong>dataset_reader</strong> defines the dataset’s location and format. After loading, the data is split between the train, validation, and test sets according to the <strong>dataset_iterator</strong> settings. The <strong>chainer</strong> section of the configuration files consists of three subsections. The <strong>in</strong> and <strong>out</strong> sections define input and output to the <strong>chainer</strong>, whereas the <strong>pipe</strong> section defines a pipeline of the required components to interact with the models. The <strong>metadata</strong> section describes the model requirements along with the model variables.</p><p>You can <strong>interact</strong> with the models defined in the configuration files via the command-line interface (CLI). However, before using any model you should install all its requirements by running it with the <strong>install</strong> command. The model’s dependencies are defined in the requirements part of the configuration file.</p><pre>python -m deeppavlov install &lt;config_path&gt;</pre><p>Where &lt;config_path&gt; is a path to the chosen model’s config file.</p><p>To get predictions from a model interactively through CLI, run</p><pre>python -m deeppavlov interact &lt;config_path&gt; [-d]</pre><p>Where -d downloads the required data, such as pretrained model files and embeddings.</p><p>You can train a model by running it with the <strong>train</strong> parameter. The model will be trained on the dataset defined in the <strong>dataset_reader</strong> section of the configuration file.</p><pre>python -m deeppavlov train &lt;config_path&gt;</pre><p>The DeepPavlov framework allows you to test all the available models on your data to identify the one that performs best. To test a model, specify the dataset split along with the split fields in the <strong>dataset_iterator</strong> section of the configuration file.</p><pre>python -m deeppavlov test &lt;config_path&gt;</pre><p>In addition, you can run a server with API access to a model by executing DeepPavlov with the <strong>riseapi</strong> command:</p><pre>python -m deeppavlov riseapi &lt;config_path&gt;</pre><p>You can find more running actions in our <a href="https://github.com/deepmipt/DeepPavlov">docs</a>.</p><h3>An indispensable tool</h3><p><a href="https://www.tensorflow.org/">TensorFlow</a> is an end-to-end open-source platform for machine learning. TensorFlow was an indispensable tool when developing DeepPavlov.</p><p>Starting from TensorFlow 1.4.0, <a href="https://keras.io/">Keras</a> has been part of the core API [<a href="https://github.com/tensorflow/tensorflow/releases/tag/v1.4.0">https://github.com/tensorflow/tensorflow/releases/tag/v1.4.0</a>]. Keras is a high-level API that lowered the barrier to getting started with deep learning. Keras provides a high-level abstraction layer over TensorFlow so that we can focus more on the problem and hyperparameter tuning. Most <a href="http://docs.deeppavlov.ai/en/master/features/models/classifiers.html">text classification</a> models of DeepPavlov have been implemented by using Keras abstractions. Keras provides us with fast prototyping techniques for quickly trying various neural network architectures and tuning hyperparameters.</p><p>In addition, the flexibility of TensorFlow allows us to build any neural network architecture we can think of, including, but not limited to, sequence tagging and question answering. Specifically, we use TensorFlow for seamless integration with the BERT-based models. We’ve already implemented BERT-based English and multilingual models for text classification, named entity recognition, and question answering (more on that in the upcoming sections). Moreover, the TensorFlow flexibility enables us to build BERT on our data; this is how we trained BERT on conversational data that led to better performance on social networks input.</p><p>Another great advantage of TensorFlow is TensorBoard. You can use TensorBoard to visualize your TensorFlow graph, plot metrics, and show additional data. TensorBoard allows us to inspect models and make appropriate changes while debugging them. This can be useful for gaining a better understanding of machine learning models.</p><h3>DeepPavlov delivers results</h3><p><a href="https://github.com/deepmipt/DeepPavlov">DeepPavlov</a> comes with several predefined components powered by TensorFlow and Keras for solving NLP-related problems, including text classification, named entity recognition, question answering, and many others. Nowadays, state-of-the-art results in many tasks have been achieved by applying BERT-based models. The release of BERT (Bidirectional Encoder Representations from Transformers) [<a href="https://arxiv.org/abs/1810.04805">research paper</a>] made the year 2018 an inflection point for the Natural Language Processing community. BERT is a transformer-based technique for pretraining language representations. We integrated BERT into three downstream tasks: <a href="https://towardsdatascience.com/the-bert-based-text-classification-models-of-deeppavlov-a85892f14d61">text classification</a>, <a href="https://towardsdatascience.com/19-entities-for-104-languages-a-new-era-of-ner-with-the-deeppavlov-multilingual-bert-1bfa6d413ea6">named entity recognition</a> (and sequence tagging in general), and <a href="https://towardsdatascience.com/bert-based-cross-lingual-question-answering-with-deeppavlov-704242c2ac6f">question answering</a>. As a result, we achieved substantial improvements in all these tasks. In the following sections, I will describe in detail how to use the BERT-based models of DeepPavlov. The code can be accessed on <a href="https://colab.research.google.com/github/deepmipt/dp_notebooks/blob/master/DP_tf.ipynb">Google Colab</a>.</p><h3>BERT for Text Classification</h3><p>Let’s demonstrate the <a href="https://github.com/deepmipt/DeepPavlov">DeepPavlov</a> BERT-based text classification models using the insult detection problem. It involves predicting whether a comment posted during a public discussion is considered insulting to one of the participants. This is a binary classification problem with only two classes: <em>Insult </em>and<em> Not Insult.</em></p><p>Any pretrained model can be used for inference via both the command-line interface (CLI) and Python. Before using the model, make sure that all the required packages are installed using the command:</p><pre>python -m deeppavlov install insults_kaggle_bert</pre><pre>python -m deeppavlov interact insults_kaggle_bert -d</pre><p>You can interact with the model via Python code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/79b43424f5c801dc7a1c78b5e171b51e/href">https://medium.com/media/79b43424f5c801dc7a1c78b5e171b51e/href</a></iframe><p>You can train the BERT-based text classification model on your own data. In order to do so, modify the <strong>data_path</strong> parameter in the <strong>dataset_reader</strong> section of the configuration file.</p><p>Then train the model in CLI:</p><pre>python -m deeppavlov train my_text_classification_config.json</pre><p>Or via Python:</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/4c2dc709745cfad3221c8b8c4845b65b/href">https://medium.com/media/4c2dc709745cfad3221c8b8c4845b65b/href</a></iframe><p>You can read more about BERT-based text classification models <a href="https://towardsdatascience.com/the-bert-based-text-classification-models-of-deeppavlov-a85892f14d61">here</a>, and you can test them in our <a href="https://demo.deeppavlov.ai/#/mu/ner?utm_source=medium&amp;utm_medium=article&amp;utm_campaign=tf">demo</a>.</p><h3>BERT for Named Entity Recognition</h3><p>In addition to the text classification models, DeepPavlov contains BERT-based models for <a href="https://en.wikipedia.org/wiki/Named-entity_recognition">named-entity recognition (NER)</a>. This is one of the most common tasks in NLP and can be formulated as follows: Given a sequence of tokens (words, and possibly punctuation marks), provide a tag from a predefined tag set for each token in the sequence. NER has a variety of business applications. For example, it can extract the important information from resumes to facilitate their evaluation by HR professionals. Moreover, NER can be used to identify the relevant entities in customer requests, such as product specifications, corporate names, or company branch details.</p><p>We trained our NER model on the OntoNotes English-language corpus, which has 19 types in the markup schema, including PER (person), LOC (location), ORG (organization), and many others. In order to <strong>interact</strong> with the model, first <strong>install</strong> its requirements.</p><pre>python -m deeppavlov install ner_ontonotes_bert_mult</pre><pre>python -m deeppavlov interact ner_ontonotes_bert_mult [-d]</pre><p>In addition, you can interact with the model via Python code.</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/e52a292d168c9960cdf458dbb6df4886/href">https://medium.com/media/e52a292d168c9960cdf458dbb6df4886/href</a></iframe><p>The multilingual BERT (M-BERT) model enables zero-shot transfers between languages, which means you can test the model on non-English sentences, even though it was trained on English OntoNotes, for example</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7d73bfb17a7cadc166305e79d6331509/href">https://medium.com/media/7d73bfb17a7cadc166305e79d6331509/href</a></iframe><p>You can read more about NER models in the article. In addition, you can check out the NER models in our <a href="https://demo.deeppavlov.ai/#/mu/ner?utm_source=medium&amp;utm_medium=article&amp;utm_campaign=tf">demo</a>.</p><h3>BERT for Question Answering</h3><p>Context-based question answering is the task of finding an answer to a question over a given context (e.g., a paragraph from Wikipedia), where the answer to each question is a segment of the context. For example, the triple of context, question, and answer below forms a correct triplet for the context-based question answering task.</p><p><strong>Context:</strong></p><p><em>In meteorology, precipitation is any product of the condensation of atmospheric water vapor that falls under gravity. The main forms of precipitation include drizzle, rain, sleet, snow, graupel, and hail. Precipitation forms as smaller droplets coalesce via collision with other raindrops or ice crystals </em><strong><em>within a cloud</em></strong><em>. Short, intense periods of rain in scattered locations are called “showers.”</em></p><p><strong>Question</strong>:</p><p><em>Where do water droplets collide with ice crystals to form precipitation?</em></p><p><strong>Answer:</strong></p><p><em>within a cloud</em></p><p>A question answering system can automate a lot of processes in your business. For example, it can help your employers to get answers based on your internal company documentation. In addition, it helps you to check the reading comprehension ability of your students in tutoring. Recently, the context-based question answering task attracted a lot of attention in academia. One of the major milestones in this field was the release of the Stanford Question Answering Dataset (<a href="https://rajpurkar.github.io/SQuAD-explorer/">SQuAD</a>). It is a new reading comprehension dataset, consisting of questions posed by crowdworkers on a set of Wikipedia articles. The SQuAD dataset has given rise to endless approaches to question answering problems. One of the most successful is the BERT-based question answering model. This model outperforms all the others and currently delivers results bordering on human performance.</p><p>In order to use the BERT-based QA model with DeepPavlov, first install its requirements.</p><pre>python -m deeppavlov install squad_bert</pre><p>Then you can interact with the model as follows:</p><pre>python -m deeppavlov interact squad_bert -d</pre><p>In addition, you can use the model via Python code</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/7b24bc6b0d02c6d4da2a0bbeff0475b2/href">https://medium.com/media/7b24bc6b0d02c6d4da2a0bbeff0475b2/href</a></iframe><p>The Multilingual BERT model enables building a multilingual QA system simply by training it on the English SQuAD dataset. The multilingual QA supports all of the <a href="https://github.com/google-research/bert/blob/master/multilingual.md#list-of-languages">104 languages</a> that were used to train M-BERT. You can use it as follows</p><iframe src="" width="0" height="0" frameborder="0" scrolling="no"><a href="https://medium.com/media/385e8967327a167ffab5de28f5053240/href">https://medium.com/media/385e8967327a167ffab5de28f5053240/href</a></iframe><p>As you can see, we call the model by providing the batch of contexts and the batch of questions, and as an output, the model returns the batch of extracted results from the contexts with their start positions. This code snippet demonstrates that the multilingual QA model, while being trained on an English dataset, is capable of extracting answers from a French context even when the question is asked in a different language.</p><p>The detailed comparison of cross-language transferability of the multilingual QA model can be found in a dedicated <a href="https://towardsdatascience.com/bert-based-cross-lingual-question-answering-with-deeppavlov-704242c2ac6f">article</a>.</p><h3>Conclusion</h3><p>We hope this was helpful and that you’ll be eager to use DeepPavlov for your own natural language understanding use cases. You can read more about us in our official <a href="https://medium.com/deeppavlov">blog</a>. Also, feel free to test our BERT-based models by using our <a href="http://demo.deeppavlov.ai?utm_source=medium&amp;utm_medium=article&amp;utm_campaign=tf">demo</a>. And don’t forget DeepPavlov has a dedicated <a href="https://forum.deeppavlov.ai">forum</a>, where any questions concerning the framework and the models are welcome.</p><figure><img alt="Moscow Institute of Physics and Technology" src="https://cdn-images-1.medium.com/max/801/0*sU2Ntu56v7cuSdaK" /></figure><img src="https://medium.com/_/stat?event=post.clientViewed&referrerSource=full_rss&postId=31cf26849e37" width="1" height="1" alt=""><hr><p><a href="https://medium.com/tensorflow/deeppavlov-an-open-source-library-for-end-to-end-dialog-systems-and-chatbots-31cf26849e37">DeepPavlov: an open-source library for end-to-end dialog systems and chatbots</a> was originally published in <a href="https://medium.com/tensorflow">TensorFlow</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>]]></content:encoded>
        </item>
    </channel>
</rss>