Create transfers

This page shows you how to create and start transfer jobs.

To see if your source and destination (also known as a sink) are supported by Storage Transfer Service, refer to Supported sources and sinks.

Agents and agent pools

Depending on your source and destination, you may need to create and configure an agent pool and install agents on a machine with access to your source or destination.

  • Transfers from Amazon S3, Microsoft Azure, URL lists, or Cloud Storage to Cloud Storage do not require agents and agent pools.

  • Transfers whose source and/or destination is a file system, or from S3-compatible storage, do require agents and agent pools. See Manage agent pools for instructions.

Before you begin

Before configuring your transfers, make sure you have configured access:

If you're using gcloud commands, install the gcloud CLI.

Create a transfer

Don't include sensitive information such as personally identifiable information (PII) or security data in your transfer job name. Resource names may be propagated to the names of other Google Cloud resources and may be exposed to Google-internal systems outside of your project.

Google Cloud console

  1. Go to the Storage Transfer Service page in the Google Cloud console.

    Go to Storage Transfer Service

  2. Click Create transfer job. The Create a transfer job page is displayed.

  3. Choose a source:

    Cloud Storage

    Your user account must have storage.buckets.get permission to select source and destination buckets. Alternatively, you can type the name of the bucket directly. For more information, see Troubleshooting access.

    1. Under Source type, select Cloud Storage.

    2. Select your Destination type.

    3. If your destination is Cloud Storage, select your Scheduling mode. Batch transfers execute on a one-time or scheduled basis. Event-driven transfers continuously monitor the source and transfer data when it's added or modified.

      To configure an event-driven transfer, follow the instructions at Event-driven transfers.

    4. Click Next step.

    5. Select a bucket and (optionally) a folder in that bucket by doing one of the following:

      • Enter an existing Cloud Storage bucket name and path in the Bucket or folder field without the prefix gs://. For example, my-test-bucket/path/to/files. To specify a Cloud Storage bucket from another project, type the name exactly into the Bucket name field.

      • Select a list of existing buckets in your projects by clicking Browse, then selecting a bucket.

        When you click Browse, you can select buckets in other projects by clicking the Project ID, then selecting the new Project ID and bucket.

      • To create a new bucket, click Create new bucket.

      If you specify a path within the folder, see Specifying source and destination paths for important details.

    6. If this is an event-driven transfer, enter the Pub/Sub subscription name, which takes the following format:

      projects/PROJECT_NAME/subscriptions/SUBSCRIPTION_ID
    7. Optionally, choose to filter objects by prefix or by last modified date. If you specified a folder as your source location, prefix filters are relative to that folder. For example, if your source is my-test-bucket/path/, an include filter of file includes all files starting with my-test-bucket/path/file.

    8. Click Next step.

    Amazon S3

    See Transfer from Amazon S3 to Cloud Storage.

    S3-compatible storage

    See Transfer from S3-compatible storage to Cloud Storage.