Object storage

GitLab supports using an object storage service for holding numerous types of data. It’s recommended over NFS and in general it’s better in larger setups as object storage is typically much more performant, reliable, and scalable.

To configure the object storage, you have two options:

• Recommended. Configure a single storage connection for all object types: A single credential is shared by all supported object types. This is called the consolidated form.

• Configure each object type to define its own storage connection: Every object defines its own object storage connection and configuration. This is called the storage-specific form.

  If you already use the storage-specific form, see how to transition to the consolidated form.

If you store data locally, see how to migrate to object storage.

Object storage provider support

GitLab uses the Fog library for object storage and supports the following three connection types. Other Fog providers are not supported.

If your object storage service is compatible with one of these connection types, configure it using the corresponding connection settings below. The choice of provider is yours.

Providers with active test coverage

GitLab actively tests the following providers:

• Amazon S3 - AWS connection type. Object Lock is not supported. For more information, see issue 335775.
• Google Cloud Storage - Google connection type.
• Azure Blob Storage - AzureRM connection type.

Community-documented providers

The following providers have been documented by the community. GitLab does not test these providers. Configuration examples are provided as a convenience. If you use one of these providers and encounter issues, GitLab Support might be unable to help.

• Digital Ocean Spaces. S3-compatible, see provider-specific configuration examples.
• Oracle Cloud Infrastructure. S3-compatible, see provider-specific configuration examples.
• OpenStack Swift (S3-compatible mode).
• Storj Gateway. S3-compatible, see provider-specific configuration examples.
• Ceph RGW. S3-compatible, see provider-specific configuration examples
• Hitachi Vantara HCP. S3-compatible, see provider-specific configuration examples.
• On-premises hardware and appliances that expose an S3-compatible API.

Configure a single storage connection for all object types (consolidated form)

Most types of objects, such as CI artifacts, LFS files, and upload attachments can be saved in object storage by specifying a single credential for object storage with multiple buckets.

Configuring the object storage using the consolidated form has a number of advantages:

• It can simplify your GitLab configuration because the connection details are shared across object types.
• It enables the use of encrypted S3 buckets.
• It uploads files to S3 with proper Content-MD5 headers.

When the consolidated form is used, direct upload is enabled automatically. Thus, only the following providers can be used:

• S3-compatible providers
• Google Cloud Storage
• Azure Blob storage

The consolidated form configuration can’t be used for backups or Mattermost. Backups can be configured with server side encryption separately. See the table for a complete list of supported object storage types.

Enabling the consolidated form enables object storage for all object types. If not all buckets are specified, you may see an error like:

Object storage for <object type> must have a bucket specified

If you want to use local storage for specific object types, you can disable object storage for specific features.

Configure the common parameters

In the consolidated form, the object_store section defines a common set of parameters.

For an example, see how to use the consolidated form and Amazon S3.

Configure the parameters of each object

Each object type must at least define the bucket name where it will be stored.

The following table lists the valid objects that can be used:

Within each object type, four parameters can be defined:

For an example, see how to use the consolidated form and Amazon S3.

Disable object storage for specific features

As seen previously, object storage can be disabled for specific types by setting the enabled flag to false. For example, to disable object storage for CI artifacts:

gitlab_rails['object_store']['objects']['artifacts']['enabled'] = false

A bucket is not needed if the feature is disabled entirely. For example, no bucket is needed if CI artifacts are disabled with this setting:

gitlab_rails['artifacts_enabled'] = false

Configure each object type to define its own storage connection (storage-specific form)

With the storage-specific form, every object defines its own object storage connection and configuration. You should use the consolidated form instead, except for the storage types not supported by the consolidated form. When working with the GitLab Helm charts, refer to how the charts handle consolidated form for object storage.

The use of encrypted S3 buckets with non-consolidated form is not supported. You may get ETag mismatch errors if you use it.

For storage types not supported by the consolidated form, refer to the following guides:

Configure the connection settings

Both consolidated and storage-specific form must configure a connection. The following sections describe parameters that can be used in the connection setting.

S3-compatible providers

These settings apply to Amazon S3 and any S3-compatible service using the AWS connection type. When not using AWS directly, set endpoint to your provider’s URL.

S3-compatible services vary in how closely they implement the AWS S3 API. GitLab uses specific S3 behaviors, including pre-signed URLs, multipart uploads, and optionally chunked signature streaming, that not every S3-compatible implementation supports identically. If a provider works in other tools but not in GitLab, the settings that most likely need adjustment are:

• aws_signature_version.
• enable_signature_v4_streaming.

The connection settings match those provided by fog-aws:

S3 compatibility and known failure modes

Claiming S3 compatibility does not guarantee a provider works correctly with GitLab. If you encounter errors with an S3-compatible provider, try the following adjustments before raising a support request:

• Signature streaming: Some providers reject chunked transfer encoding used by AWS Signature Version 4 streaming. Set enable_signature_v4_streaming: false.
• Signature version: Some providers do not fully support Signature Version 4. Set aws_signature_version: 2.
• Path-style URLs: Some providers require path-style bucket addressing. Set path_style: true.
• ETag validation: Some providers return ETags that do not match the MD5 of the uploaded object, which GitLab validates. See ETag mismatch.

GitLab Support can help troubleshoot configuration, but cannot guarantee resolution for issues specific to providers not in the tested set.

Use Amazon instance profiles

Instead of supplying AWS access and secret keys in object storage configuration, you can configure GitLab to use Amazon Identity Access and Management (IAM) roles to set up an Amazon instance profile. When this is used, GitLab fetches temporary credentials each time an S3 bucket is accessed, so no hard-coded values are needed in the configuration.

Prerequisites:

• GitLab must be able to connect to the instance metadata endpoint.
• If GitLab is configured to use an internet proxy, the endpoint IP address must be added to the no_proxy list.
• For IMDS v2 access, ensure the hop limit is sufficient. If GitLab is running in a container, you may need to raise the limit from 1 to 2.

To set up an instance profile:

1. Create an IAM role with the necessary permissions. The following example is a role for an S3 bucket named test-bucket:

   json

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Effect": "Allow",
               "Action": [
                   "s3:PutObject",
                   "s3:GetObject",
                   "s3:DeleteObject"
               ],
               "Resource": "arn:aws:s3:::test-bucket/*"
           },
           {
               "Effect": "Allow",
               "Action": [
                   "s3:ListBucket"
               ],
               "Resource": "arn:aws:s3:::test-bucket"
           }
       ]
   }

   If an external process tags objects in your S3 buckets (for example, AWS GuardDuty Malware Protection), add s3:GetObjectTagging to the object-level Action list for source buckets and s3:PutObjectTagging for destination buckets. Without these permissions, GitLab CopyObject operations fail with AccessDenied when copying tagged objects.

2. Attach this role to the EC2 instance hosting your GitLab instance.

3. Set the use_iam_profile GitLab configuration option to true.

Encrypted S3 buckets

When configured either with an instance profile or with the consolidated form, GitLab Workhorse properly uploads files to S3 buckets that have SSE-S3 or SSE-KMS encryption enabled by default. AWS KMS keys and SSE-C encryption are not supported because this requires sending the encryption keys in every request.

Server-side encryption headers

Setting a default encryption on an S3 bucket is the easiest way to enable encryption, but you may want to set a bucket policy to ensure only encrypted objects are uploaded. To do this, you must configure GitLab to send the proper encryption headers in the storage_options configuration section:

As with the case for default encryption, these options only work when the Workhorse S3 client is enabled. One of the following two conditions must be fulfilled:

• use_iam_profile is true in the connection settings.
• Consolidated form is in use.

ETag mismatch errors occur if server side encryption headers are used without enabling the Workhorse S3 client.

Google Cloud Storage (GCS)

Here are the valid connection parameters for GCS:

GitLab reads the value of google_json_key_location, then google_json_key_string, and finally, google_application_default. It uses the first of these settings that has a value.

The service account must have permission to access the bucket. For more information, see the Cloud Storage authentication documentation.

Google Cloud Application Default Credentials

Google Cloud Application Default Credentials (ADC) are typically used with GitLab to use the default service account or Workload Identity Federation. Set google_application_default to true and omit google_json_key_location and google_json_key_string.

If you use ADC, be sure that:

• The service account that you use has the iam.serviceAccounts.signBlob permission. Typically this is done by granting the Service Account Token Creator role to the service account.

• If you are using Google Compute virtual machines, ensure they have the correct access scopes to access Google Cloud APIs. If the machines do not have the right scope, the error logs may show:

  shell

  Google::Apis::ClientError (insufficientPermissions: Request had insufficient authentication scopes.)

Azure Blob storage

Although Azure uses the word container to denote a collection of blobs, GitLab standardizes on the term bucket. Be sure to configure Azure container names in the bucket settings.

Azure Blob storage can only be used with the consolidated form because a single set of credentials are used to access multiple containers. The storage-specific form is not supported. For more details, see how to transition to consolidated form.

The following are the valid connection parameters for Azure. For more information, see the Azure Blob Storage documentation.