We gave Antigravity CLI one real task: import 81,444 images into FiftyOne and run a full data-curation pipeline on all of them, no shortcuts.

What is Antigravity CLI?

Antigravity is Google’s agent harness for engineering work. It ships as a CLI (agy), an SDK, and an IDE. You type a task in plain English, and the agent plans it, calls tools, runs shell commands, edits files, and keeps going until it’s done, pausing to ask you before doing anything risky.

Three ideas make it more than “a chatbot in a terminal”:

• Skills: small folders with a SKILL.md file describing a workflow: when to use it, and exactly how, step by step. Antigravity finds these automatically and turns each one into something it can use mid-conversation, and into a slash command you can call directly.
• Plugins: a Skill (or a bundle of Skills, subagents, and MCP servers) packaged as one installable unit. A plugin’s manifest is deliberately tiny: just a name and a description. Everything else it needs lives in fixed folders next to it.
• MCP servers: the Model Context Protocol is how Antigravity talks to external tools. A skill library like FiftyOne’s exposes its own operators this way, so the agent can call them directly instead of guessing at a command line.

On top of that, Antigravity supports multiple reasoning models in the same session: you can start on one and switch to another without losing context, useful when a long job runs into a quota limit.

Installing it

curl -fsSL https://antigravity.google/cli/install.sh | bash

That downloads the agy binary to ~/.local/bin/agy:

Then start it:

Adding a skill library

Skills are what give the agent domain expertise it doesn’t have by default. You can write your own, or install an existing library, like FiftyOne Skills: an open-source set of 18 skills covering more than 80 computer vision operators, importing datasets, running models, evaluating predictions, curating data, and more.

curl -sL skil.sh | sh -s - voxel51/fiftyone-skills

The installer asks which agent to target. Antigravity is one option among several:

Once installed, /skills confirms what the agent can now see. In this run, that’s 20 total (18 from the FiftyOne library, plus 2 built into Antigravity itself):

Slash commands, and how to actually use them well

Type / at any point to open the command menu. The full list is in the CLI reference docs; here’s what matters day to day:

• /skills: Browse every loaded skill (local and global)
• /agents: Open the Agent Manager, switch custom agents, watch background subagents
• /mcp: Manage connected MCP servers
• /tasks: See background shell jobs the agent has running
• /model: Choose your reasoning model, persists across sessions
• /usage (alias /quota): Show how much of your model quota you’ve used
• /btw <question>: Ask a quick side question without derailing the main task
• /context: Open the context usage visualization panel
• /config (alias /settings): Open the full settings editor, including safety controls

Good habits when a task involves Skills

Each one of these came up in the run below.

1. Run /skills before you write your prompt, not after. Knowing the exact skill names lets you reference them directly, the way the prompt below does: “Use the fiftyone-import-dataset and fiftyone-dataset-curation skills…” Naming the skill removes any guesswork about which workflow the agent should follow.
2. Check /mcp before kicking off anything long. If a skill depends on an MCP server (like FiftyOne’s own tool server) and it isn’t connected, you want to know that in five seconds, not fifty minutes into a run.
3. Use /tasks to watch long jobs instead of babysitting the chat. A skill-driven job that takes over an hour (like the import below) runs as a background task. /tasks shows you it’s alive without cluttering the conversation.
4. Check /usage before a big autonomous run, and know /model is your escape hatch. Long, skill-heavy sessions burn through quota fast. When this session actually hit a quota wall mid-import, switching models with /model (from Gemini 3.5 Flash to Claude Sonnet 4.6) let it keep going without losing any of its progress or context.
5. Use /btw for side questions instead of interrupting the main task. It runs in the background and doesn’t derail whatever the agent is already doing.
6. /config is where you decide how much autonomy to grant. Tool Permission controls whether the agent asks before every action (request-review, the default), never asks (always-proceed), or something in between:

/model and /usage in practice

This session started on Gemini 3.5 Flash (Medium). Partway through, it hit this:

⚠ Individual quota reached. Please upgrade your subscription to increase your limits. Resets in 2h54m22s.

/usage (or /quota) is what would have flagged this coming. Once it happened, the fix was one command: /model, switch to Claude Sonnet 4.6 (Thinking), keep going. The new model shows up immediately in the status bar, visible on this approval prompt a few minutes later:

That’s the real value of the two commands together: /usage tells you when you’re about to run out of runway, /model lets you change lanes without starting over.

The example: 81,444 images, no hand-holding

Here’s the exact instruction given to the agent:

Use the fiftyone-import-dataset and fiftyone-dataset-curation skills to import the full huggan/wikiart dataset (https://huggingface.co/datasets/huggan/wikiart) into the FiftyOne installation at /Users/ado/Documents/fiftyone/env. Run the complete curation pipeline (no sampling), then use the fiftyone-playwright skills to monitor and validate the process in real time. If any step fails, automatically diagnose, fix, and continue until the pipeline completes successfully.

Before touching anything, it read the three relevant SKILL.md files, checked the local environment, and came back with a three-phase plan (import, curation, Playwright validation) plus two questions: could it install a package, and should it keep a running work log.

Import. Streaming images one at a time was too slow for 81,444 of them. The agent noticed, rewrote its own import script twice, and landed on a batched approach: download 8 parquet files in parallel (~4 GB at a time), extract to JPEGs, import, delete, repeat. This kept it within the ~74 GB of free disk space. It ran for nine batches, reporting real numbers along the way (13,584 samples, then 22,632, then 49,776 at 61%, then 58,824 at 72%), each one checked live in the FiftyOne App via Playwright rather than assumed from a log. After the quota-driven model switch above, it finished:

🎉 Import complete! All 81,444 samples have been imported successfully in 4012 seconds (~67 minutes).

Curation: the actual “diagnose, fix, continue” test. The curation script computed embeddings on the Mac’s GPU, or was supposed to. The model quietly stayed on CPU instead, running at ~2 samples/second, which would have taken over 14 hours. The agent debugged this instead of stopping: it found that a device: “mps” setting wasn’t reaching the underlying PyTorch model, fixed it by loading the model directly and moving it to the GPU manually, hit a second bottleneck (a slow per-image processing pattern), and rewrote the script a third time using batched torchvision calls. That version benchmarked at 60–77 samples/second and, running for real, climbed past 180. At one point, seeing how fast it was going, the user suggested it didn’t need to process every image. The agent correctly stuck to the original “no sampling” instruction, noting the full run would finish in about 7 minutes anyway.

Inside FiftyOne, after the run

The result is something you can actually look at, not just a log saying “done.” All 81,444 samples are loaded with their artist, genre, and style labels intact:

Every painting now also carries a uniqueness score alongside its labels, here 0.87 out of 1:

That score is the real payoff of the curation pipeline: a value near 1 means an image looks unlike almost anything else in the dataset, while a value near 0 means it looks a lot like other images already in there. Sorting by uniqueness is how you’d find near-duplicates worth removing, or find the truly one-of-a-kind pieces, across 81,444 images without opening a single one by hand.

The img_umap embeddings tell an even bigger story about the dataset as a whole. Colored by genre:

The orange covering most of the genre map is Unknown Genre, meaning most paintings in this dataset simply don’t have a genre label. That’s not a bug in the pipeline. It’s a real, useful finding: genre coverage in this dataset is thin, and now you know it in one glance instead of discovering it painting by painting.

Compare that to the same map colored by style instead. There’s no dominant “unknown” blob here. All 27 styles show up as distinct, colorful regions, and visually similar styles (like Impressionism and Post_Impressionism) sit near each other on the map, exactly as you’d expect from an embedding that groups images by visual similarity. Put the two maps side by side and you get a genuine curation insight for free: this dataset’s style labels are far more complete and trustworthy than its genre labels, which tells you exactly where to focus annotation effort next.

Try it yourself

1. Install Antigravity CLI.
2. Install a skill library, for example FiftyOne’s.
3. Run /skills to confirm what’s loaded, then describe your task in plain English, naming the skills you want it to use
4. For anything long-running: keep an eye on /tasks and /usage, and know /model is there if you need it

Sources and further reading:

1. Antigravity CLI documentation
2. Antigravity CLI reference (slash commands, keybindings, settings)
3. FiftyOne Skills on GitHub
4. huggan/wikiart dataset on Hugging Face
5. FiftyOne + natural language interfaces

This guide shows you how to:

• Use the new Google Colab extension inside VS Code
• Run notebooks on Colab GPU/TPU directly from your editor
• Recreate the Qwen3-VL + FiftyOne video analysis demo
• Explore your results interactively inside FiftyOne App

How to Install It

1. Open VS Code
2. Go to Extensions
3. Search “Colab”
4. Install the official Google Colab extension

Connect to a Colab Runtime

1. Open or create a .ipynb notebook
2. Click the kernel selector in the top-right
3. Choose: Colab → New Colab Server
4. Sign in with your Google account
5. Pick GPU (recommended)

Your notebook will now run on Colab hardware while you stay inside VS Code.

Qwen3-VL + FiftyOne Video Analysis

Qwen3-VL is the newest and most powerful vision-language model in the Qwen family. It understands text, images, and video at a very deep level, and it can reason about them together. Compared to earlier versions, it’s better at everything — language, vision, long-context understanding, and complex reasoning.

It comes in different sizes and architectures (Dense and MoE), so you can run it on anything from edge devices to cloud GPUs. There are also special Instruct and Thinking versions depending on whether you need simple instructions or deeper reasoning.

Why Qwen3-VL is powerful

• Visual agent abilities — It can understand computer and mobile screens, recognize UI elements, and even help complete tasks by calling tools.
• Visual coding — It can generate Draw.io diagrams, HTML, CSS, and JavaScript directly from images or videos.
• Stronger spatial reasoning — It understands object positions, viewpoints, occlusions, and can reason in both 2D and 3D.
• Long-context video understanding — Handles up to 256K tokens (and expandable to 1M), meaning it can analyze long videos, books, and large documents with full recall.
• Better multimodal reasoning — Great at math, STEM, causal reasoning, and giving logical, evidence-based answers.
• Upgraded recognition — Trained on broad, high-quality visual data so it can recognize almost anything: people, products, landmarks, animals, and more.
• Expanded OCR — Supports 32 languages, works well even with blur, low light, tilted text, and long documents.
• Text understanding like a pure LLM — It reads, writes, and reasons with text at the same level as top language models.

Under the hood

• Interleaved-MRoPE for better long-horizon video reasoning.
• DeepStack to merge multi-level visual features for sharper image-text alignment.
• Text–timestamp alignment for precise event localization inside videos.

Qwen3-VL understands videos, images, and text extremely well, and reasons across them like a top LLM, making it perfect for advanced video analysis setups like the one in this blog.

Now, we’ll recreate the workflow from the original notebook created by Harpreet Sahota:

• Run Qwen3-VL on a video
• Generate descriptions, events, OCR, analysis
• Compute embeddings
• Explore everything in FiftyOne
• Use custom prompts for structured results

Paste each step into your notebook inside VS Code (connected to Colab).

Step 1 — Install Dependencies

Install the core libraries needed for video loading, multimodal modeling, and dataset exploration.

!pip install -q fiftyone decord qwen-vl-utils transformers torch torchvision

Step 2 — Import FiftyOne + Register the Model Source

Tell FiftyOne where the Qwen3-VL model definitions live so you can load it like any other zoo model.

import fiftyone as fo
import fiftyone.zoo as foz

foz.register_zoo_model_source(
    "https://github.com/harpreetsahota204/qwen3vl_video",
    overwrite=True,
)

Step 3 — Load a Video Dataset

Load your input videos — either from the FiftyOne zoo or your own files, and prepare metadata required for temporal operations.

Use a Zoo QuickStart video Dataset

dataset = foz.load_zoo_dataset("quickstart-video", max_samples=5)

# Compute metadata (required for temporal operations)
dataset.compute_metadata() 

Step 4 — Load Qwen3-VL

Load the Qwen3-VL video understanding model from the model zoo, first download is large, this will take time the first run (If your setup can handle it, feel free to jump to the bigger 8B model).

model = foz.load_zoo_model("Qwen/Qwen3-VL-4B-Instruct")

Step 5 — Basic Video Description

This step uses Qwen3-VL to summarize each video in natural language.
The "description" mode generates a high-level explanation of what is happening in the clip.

model.operation = "description"
dataset.apply_model(model, label_field="description", skip_failures=True)

# View a sample description
sample = dataset.first()
print(sample.description_summary)

Step 6 — Comprehensive Analysis

Beyond basic description, this mode extracts richer, multi-field insights. It may include objects, actions, attributes, and scene details depending on the model’s capabilities.

model.operation = "comprehensive"
dataset.apply_model(model, label_field="analysis", skip_failures=True)

print("Analysis complete!")
print(f"Fields added: {[f for f in dataset.get_field_schema().keys() if f.startswith('analysis')]}")

Step 7 — Temporal Event Detection

Temporal localization identifies segments in the video where certain events occur. This is helpful when the model needs to reason over the timeline instead of analyzing the video as a whole.

model.operation = "temporal_localization"
dataset.apply_model(model, label_field="events", skip_failures=True)

Step 8 — OCR on the Video

In this step, the model extracts text visible in the frames, perfect for signage, subtitles, labels, and any screen text that appears throughout the clip.

model.operation = "ocr"
dataset.apply_model(model, label_field="ocr", skip_failures=True)

print("OCR complete!")

Step 9 — Compute Embeddings

We compute vector embeddings for each video. These embeddings represent the clip at a semantic level and can be used for similarity search, clustering, and visualization.

# Configure pooling strategy
model.pooling_strategy = "mean"

# Compute embeddings
dataset.compute_embeddings(
    model,
    embeddings_field="qwen_embeddings",
    skip_failures=True
)

print("Embeddings computed!")

Step 10 — Similarity Search

This section uses FiftyOne Brain to find videos that are semantically similar. It builds a similarity index using embeddings and then retrieves the closest matches to a query sample.

import fiftyone.brain as fob

# Build similarity index
fob.compute_similarity(
    dataset,
    brain_key="qwen_similarity",
    embeddings="qwen_embeddings"
)

# Find similar videos
query_sample = dataset.first()
similar_view = dataset.sort_by_similarity(
    query_sample,
    k=5,
    brain_key="qwen_similarity"
)

print(f"Found {len(similar_view)} similar videos")

Step 11 — UMAP Visualization

This step projects the high-dimensional embeddings into a 2D space using UMAP. The result is a scatterplot-style visualization that helps reveal clusters and relationships between videos.

# Compute UMAP visualization
results = fob.compute_visualization(
    dataset,
    method="umap",
    brain_key="qwen_viz",
    embeddings="qwen_embeddings",
    num_dims=2
)

print("UMAP visualization computed!")

Step 12 — Launch the FiftyOne App

Finally, we launch the interactive FiftyOne App to explore video clips, predictions, events, OCR, and similarity results visually.

session = fo.launch_app(dataset, auto=False)
session.url

Step 13 — Optional: Custom Prompts

This allows you to define your own multimodal prompt. The model will return structured JSON responses following your schema — great for tagging, metadata extraction, or building custom workflows.

model.operation = "custom"
model.custom_prompt = """Analyze this video and provide:
{
  "content_type": "educational/entertainment/promotional/other",
  "mood": "calm/energetic/dramatic/neutral"
}
"""

dataset.apply_model(model, label_field="custom_analysis", skip_failures=True)

# View results
sample = dataset.first()
print(sample.custom_analysis_result)

{
  "content_type": "educational/entertainment/promotional/other",
  "mood": "calm"
}

Modify the JSON schema to match your needs.

Resources

• Qwen3-VL Docs Fiftyone.
• Create your own pluing in Fiftyone.
• Qwen3-VL Video Demo Notebook (Harpreet).
• Google Colab VS Code Extension.
• FiftyOne Documentation.
• Qwen3-VL Github.
• Qwen3-VL Blog.

A huge thank you to:

• Harpreet Sahota (Voxel51) for creating the original Qwen3-VL + FiftyOne notebook
• The Google Colab engineering team for shipping this incredible VS Code integration
• The Google Developer Experts (GDE) program for supporting the initiative and empowering developers worldwide

This combination of VS Code + Colab + FiftyOne + Qwen3-VL — is a real superpower for anyone working with video AI. 🚀

So, For our Intelligent Agents class with Professor Ali Minai at the University of Cincinnati, we gave Gemini Advanced 1.5 Pro with Deep Research a task: generate a literature review about Embodied Intelligent Agents, Agentic AI, and Agent Models. Our assignment was to prompt the LLM to create a review paper with at least ten citations (since our group had two members), covering the topic area and including a bibliography. We were specifically instructed not to find the papers ourselves and feed them to the LLM; the LLM was supposed to generate the entire paper independently. We were allowed up to three prompts if the first attempt wasn’t satisfactory. We wanted to see if Gemini could really do research and what its limitations were. This blog post documents our experience, including our prompts, Gemini’s responses, and our critique of the final output.

Initial prompt: We are two students conducting a literature review on Embodied Intelligent Agents, Agentic AI, and Agent Models. Generate a comprehensive review paper that includes at least 10 citations from relevant research papers. The paper should follow a structured format with sections including an introduction, background, methodologies, key developments, challenges, future directions, and a conclusion. The citations should be formatted properly, and a bibliography should be included at the end. Ensure that the review reflects a broad and in-depth understanding of the topic.

Interestingly, even though our prompt was fairly simple, Gemini seemed to internally break down the task into smaller steps. It appeared to create a kind of research plan for itself, which included things like finding papers, summarizing concepts, identifying methodologies, and so on. This internal planning process wasn’t explicitly shown to us in the output, but it likely influenced how it structured the final paper.

What Gemini Made:

Gemini made a paper. It had all the parts we asked for: intro, background, methods, the whole shebang. It even found some papers to cite. At first glance, it looked okay. It talked about embodied AI (agents with bodies!), agentic AI (agents that can think for themselves!), and agent models (computer versions of agents). It even mentioned generative agents (agents that can act like humans).

But… There Were Problems:

When we looked closer, things weren’t so great. It used a lot of words, but it didn’t seem to really understand what it was talking about. It was like it just read some stuff online and put it all together without really thinking about it.

The Good Stuff (A Little):

• Structure: It followed the rules and put the paper together in the right way. Good job on that!
• Finding Some Papers: It did find some papers. But… (you knew there was a but, right?)

The Not-So-Good Stuff (A Lot):

• Not Really Understanding: The paper felt like it just put words together without really knowing what they meant. It used big words, but it didn’t show it really got the ideas.
• Citation Problems (Big Time!): This was the biggest issue. Remember, we needed good citations. Here’s the breakdown:
• Good: The first two citations were fine. Numbers 4 and 7 were good too! Yay!
• Kinda Good, But Not Quite: Numbers 3, 6, and 8 looked okay, but the links to the online papers (arXiv) were wrong.
• Wrong Author: Number 5 got the author totally wrong, and the link was bad too.
• Weird: Number 9 was strange. It got the name and author right, but the paper link was for a different version of the paper! It was like finding a recipe but getting the ingredients for a different dish.
• Didn’t Exist: Number 10 just wasn’t a real paper. Oops!
• Almost, But Not Quite: Number 11 was also a no-go. The paper they cited doesn’t exist, but there’s a similar one. They got that one wrong too!

Citation Statistics:

Out of the 11 citations:

• Perfect: 4
• Almost Right (But Link Wrong): 3
• Wrong Author/Link: 1
• Doesn’t Exist: 2
• Similar, But Wrong: 1

So, more than half the citations had some kind of problem!

Another Weird Thing:

Gemini did find a really important paper from way back in 1950 by Alan Turing about computers and thinking. That was cool! But then, it jumped straight to papers from 2021. What happened to all the research in between? It’s like it skipped a whole bunch of important stuff!

And… It Didn’t Use the Papers Much:

Even when it did find okay papers, it didn’t really use them in the text. It was like it just put the names of the papers there without explaining how they related to what it was saying.

What We Learned: AI Research Still Needs Human Help

Gemini’s try at this project showed us some important things about how good AI is at research right now. It’s cool that it could make a paper and find some sources, but it also showed us that there’s a big difference between just putting words together and actually understanding stuff.

Here’s the breakdown:

• Looks Good on the Outside: Gemini can make a paper look like it’s real. It can use fancy words, organize the information, and even put in citations. This can trick you at first. But it’s just on the surface.
• Doesn’t Really Get It: The main problem is that Gemini doesn’t really understand what it’s reading. It can write sentences, but it doesn’t really get the big ideas, how different research connects, or the little details that matter. You can see this when it tries to explain things — it’s all very simple and doesn’t show real understanding.
• Citations Were a Mess: The problems with the citations are a perfect example. It’s not just that some were wrong; it’s how they were wrong. Gemini didn’t seem to care if a source was good or bad, or if it even matched what the paper was saying. It just grabbed stuff without checking. Some of the “sources” were just Wikipedia or other websites that aren’t really good for academic research. It’s like using a children’s encyclopedia for a college paper! Humans, on the other hand, are careful about sources. They check if they’re real, if they’re good, and if they fit the paper. Gemini just doesn’t do that.
• Humans Are Still Important: This whole thing shows us why we still need human researchers. AI can be helpful for finding stuff, but it can’t think like a human. It can’t really understand what it finds, connect ideas, or make sure everything is correct. We need real people to guide the research, check the work, and make sure it’s good.
• AI Can Make Stuff Up: The fact that Gemini made up some citations or got them totally wrong is a big problem. AI can “hallucinate,” which means it can say things that sound real but are totally fake. This is scary, especially in research where everything needs to be true.
• Data Isn’t the Same as Knowing: Gemini shows us that just having lots of information isn’t the same as really knowing something. It can play with words and make sentences, but it doesn’t understand things the way people do. Knowing something means understanding the context, figuring out what it means, and connecting ideas.

The Big Picture:

AI is getting better all the time, but it’s not ready to do real research by itself yet. It’s a good helper, but it can’t replace human researchers. We need people to do the thinking, to make sure things are right, and to really understand what’s going on. So, we don’t have to worry about robot researchers taking over just yet (probably!). The real danger is if we start trusting AI too much without realizing it still makes mistakes and needs our help.

Disclaimer:

The views and opinions expressed in this blog post are solely those of the authors and do not necessarily reflect the official policies or positions of the University of Cincinnati or Professor Ali A. Minai. This analysis is based on our personal experience with the specified LLM for a class assignment and should not be taken as a comprehensive evaluation of all LLMs or their capabilities.

Full Review:

You can find the complete, unedited literature review generated by Gemini (along with all our prompts and Gemini’s responses) in this PDF.

Acknowledgements:

We would like to thank Professor Ali A. Minai for this insightful assignment, which provided valuable hands-on experience with LLMs and their capabilities.

About the Authors:

• Adonai Vera
• Aneesh Deshmukh

Instructor:

• Professor Ali A. Minai

En la búsqueda de optimizar los modelos de IA configurando los servidores para que corran en GPU, descubrí diversos artículos que fueron cruciales para lograr mi objetivo, pero mi gran problema era que no tenia una guía completa para configurar mi máquina virtual, la cual no solo corriera en GPU, sino que corriera OpenCV con el módulo dnn que permite una inferencia de 211–1549% más eficiente.

Por esta razón decidí crear una guía completa “zero to end” del proceso de configuración del servidor, la cual te ayudara a instalar en tu maquina virtual, CUDA 10.0, cuDNN 7.64.4, Opencv ‘dnn’, Ubuntu 18.04 y Chrome Remote Desktop. A continuación encontraras 6 pasos que espero sean de gran utilidad para ti y logres tener tu tecnología en producción y en GPU.

Paso 1: Crea una cuenta en GCP

Para empezar debes tener tu cuenta en GCP y si no la tienes, Google Cloud Platform te permite crear una cuenta totalmente gratuita y después del proceso de verificación te obsequia 300 créditos de bienvenida. Para registrarte ingresa al siguiente link GCP.

Para poder usar GPU es fundamental que tengas habilitadas QUOTAS. Para habilitarlas debes ir a IAM & Admin > Quotas filtras por “Service: Compute Engine API” y “GPU” seleccionas “all regions” y “all quotas”. Según la GPU que desees usar debes solicitar un nuevo limite, escribir un comentario sobre las razones de uso.

Ten en cuenta que entre 1–2 días te habilitaran la GPU para que pueda ser utilizada.

Paso 2: Creación máquina virtual

El primer paso es elegir el proyecto en donde estará alojado el servidor, seguido a esto se procede a crear el servidor. Para esto dirígete a “Compute engine” > “VM Instance”.

En este paso debes seleccionar el tipo de máquinas, la imagen y la GPU. En nuestro caso usamos una máquina “n1-standard-8' CPU” y una “NVIDIA Tesla T4”, el sistema operativo fue Ubuntu 18.04 LTS. Por último, se habilitó el tráfico HTTP y HTTPS, y se dio clic en “crear”.

Se recomienda generar políticas de seguridad para evitar posibles hackeos. Habilitar puertos de tráfico como HTTP y HTTPS permite generar puertas de vulnerabilidad que deberán ser analizadas al poner el sistema en producción.

La máquina virtual tarde entre 5–10 minutos en configurarse. Seguido a esto, damos clic en SSH para iniciar un terminal.

Recuerda que el costo por uso de GPU es elevado, si excedes los 300 créditos gratuitos, se cargaran a tu tarjeta de crédito. Se recomienda apagar la instancia si no está en uso.

Configuración Chrome Remote Desktop (Opcional)

Para muchos configurar todo el sistema desde el terminal, puede representar un proceso tedioso, es por eso que agregue esta configuración opcional para configurar Chrome Remote Desktop y acceder al servidor desde Chrome.

Antes de comenzar, se recomienda configurar una clave con el comando sudo passwd

Desde el terminal SSH, se debe instalar y configurar Chrome Remote Desktop con los siguientes comandos:

Luego de instalarlo correctamente, debes configurar el acceso al servidor, paro eso ingresa en Remote Desktop y sigue las instrucciones de configuración, al final encontraras un comando que debe ser ejecutado en el servidor:

Seguido a esto, ya podrás acceder a la máquina virtual desde “Acceso remoto”.

Paso 3: Instalación de CUDA

En este caso se instalará la versión 10.0 para Ubuntu 18.04. Sin embargo, si quieres instalar otra versión, en el siguiente link puedes obtener todas las versiones de CUDA.

Para verificar la correcta instalación de los drivers y de CUDA utilizamos el comando “nvidia-smi”. Si los drivers fueron instalados correctamente, se mostrara la siguiente imagen:

Paso 4: Instalación de CuDNN

La versión de CuDNN que se instalara es “CuDNN v7.6.4 para CUDA 10.0”, que se encuentra en el link de NVIDIA.

Primero debes crear una cuenta y seleccionar la versión “cuDNN v7.6.4 (September 27, 2019), for CUDA 10.0”, el tipo de arquitectura es “cuDNN Library for Linux”.

Seguido a esto navegamos hasta la carpeta en donde se encuentra el archivo “cudnn-10.0-linux-x64-v7.6.4.38.tgz” y realizamos la instalación de cuDDN.

Para verificar la correcta instalación de los drivers y de CUDA utilizamos el comando “nvcc -V”, en la siguiente imagen se observa la versión de CUDA:

Configuración de entornos virtual en Python (Opcional)

Es recomendable configurar entornos virtuales en Python, ya que te permite utilizar diferentes versiones de librerías dentro del mismo servidor, cada una de los entornos virtuales funciona de manera independiente.

Todos los paquetes de Python que instales dentro del entorno virtual estarán aislados de los paquetes instalados en toda la máquina.

# Ejemplo de instalación de numpy con PIP.
pip install numpy

Paso 5: Instalación de OpenCV y dependencias de GPU

El primer paso es instalar todas las dependencias necesarias para ejecutar OpenCV “dnn” con NVIDIA GPU. Es importante instalarlas por fuera del entorno virtual debido a que son globales para toda la máquina.

En caso de recibir el siguiente error:

/usr/local/cuda-10.0/targets/x86_64-linux/lib/libcudnn.so.7 is not a symbolic link”

Sigue los siguientes pasos:

Este error se presenta debido a que libcudnn.so.5 y libcudnn.so no son enlaces simbólicos. Para mayor información les dejo el siguiente foro.

Descargamos código fuente de OpenCV

Este paso es el más importante, ya que instalar OpenCV desde PIP no permitirá que se ejecuten los modelos desde la GPU. Para descargar el código fuente debes seguir los siguientes comandos en el terminal. (Recuerda que debes estar por fuera del entorno virtual de Python.)

Versión de arquitectura CUDA

Determinar la versión de la arquitectura de CUDA es fundamental, seleccionar una versión que no sea la de la GPU instalada representará que el módulo dnn de OpenCV no funcione correctamente, para determinar la versión accede a Developer Nvidia, ahí debes buscar la GPU que tienes y encontrar la versión de la arquitectura, en este caso instalamos una “NVIDIA T4” y la versión es “7.5”

Después de conocer la versión, se procede a compilar OpenCV, es importante que se lea a detalle los códigos para que se configure según la máquina virtual. En cada uno de las líneas de código agregué comentarios para que conozcas en donde debes cambiar a tu código.

Paso 6: Verificación de OpenCv “dnn” con GPU

Para verificar la correcta instalación abrimos un terminal de Python dentro de nuestro entorno virtual e importamos opencv.

Para configurar GPU desde Python es muy sencillo, lo único que debes agregar son las siguientes dos líneas de código.

Y de esta forma terminamos esta guía de configuración de una máquina virtual con NVIDIA GPU, OpenCV “dnn” en Google Cloud Platform.

Agradecimientos a Shruti , Andrey Senin y Adrian Rosebrock, por guiarme en el proceso de instalación y configuración del sistema.

¡Espero que esta información les ayude en su proceso de optimizar sus modelos con GPU, éxitos!