InspectJobConfig

Controls what and how to inspect for findings.

JSON representation 
{
  "storageConfig": {
    object (StorageConfig)
  },
  "inspectConfig": {
    object (InspectConfig)
  },
  "inspectTemplateName": string,
  "actions": [
    {
      object (Action)
    }
  ]
}
Fields
storageConfig

object (StorageConfig)

The data to scan.
inspectConfig

object (InspectConfig)

How and what to scan for.
inspectTemplateName

string

If provided, will be used as the default for all values in InspectConfig. inspectConfig will be merged into the values persisted as part of the template.
actions[]

object (Action)

Actions to execute at the completion of the job.

StorageConfig

Shared message indicating Cloud storage type.

JSON representation 
{
  "timespanConfig": {
    object (TimespanConfig)
  },

  // The following is a list of mutually exclusive fields. At most one of the
  // fields will be set in a response:
  "datastoreOptions": {
    object (DatastoreOptions)
  },
  "cloudStorageOptions": {
    object (CloudStorageOptions)
  },
  "bigQueryOptions": {
    object (BigQueryOptions)
  },
  "hybridOptions": {
    object (HybridOptions)
  }
  // End of mutually exclusive fields.
}
Fields
timespanConfig

object (TimespanConfig)

Configuration of the timespan of the items to include in scanning.
Type of storage system to inspect. The following is a list of mutually exclusive fields. At most one of the fields will be set in a response:
datastoreOptions

object (DatastoreOptions)

Google Cloud Datastore options.
cloudStorageOptions

object (CloudStorageOptions)

Cloud Storage options.
bigQueryOptions

object (BigQueryOptions)

BigQuery options.
hybridOptions

object (HybridOptions)

Hybrid inspection options.
End of mutually exclusive fields.

DatastoreOptions

Options defining a data set within Google Cloud Datastore.

JSON representation 
{
  "partitionId": {
    object (PartitionId)
  },
  "kind": {
    object (KindExpression)
  }
}
Fields
partitionId

object (PartitionId)

A partition ID identifies a grouping of entities. The grouping is always by project and namespace, however the namespace ID may be empty.
kind

object (KindExpression)

The kind to process.

PartitionId

Datastore partition ID. A partition ID identifies a grouping of entities. The grouping is always by project and namespace, however the namespace ID may be empty.

A partition ID contains several dimensions: project ID and namespace ID.

JSON representation 
{
  "projectId": string,
  "namespaceId": string
}
Fields
projectId

string

The ID of the project to which the entities belong.
namespaceId

string

If not empty, the ID of the namespace to which the entities belong.

KindExpression

A representation of a Datastore kind.

JSON representation 
{
  "name": string
}
Fields
name

string

The name of the kind.

CloudStorageOptions

Options defining a file or a set of files within a Cloud Storage bucket.

JSON representation 
{
  "fileSet": {
    object (FileSet)
  },
  "bytesLimitPerFile": string,
  "bytesLimitPerFilePercent": integer,
  "fileTypes": [
    enum (FileType)
  ],
  "sampleMethod": enum (SampleMethod),
  "filesLimitPercent": integer
}
Fields
fileSet

object (FileSet)

The set of one or more files to scan.
bytesLimitPerFile

string (int64 format)

Max number of bytes to scan from a file. If a scanned file's size is bigger than this value then the rest of the bytes are omitted. Only one of bytesLimitPerFile and bytesLimitPerFilePercent can be specified. This field can't be set if de-identification is requested. For certain file types, setting this field has no effect. For more information, see Limits on bytes scanned per file.
bytesLimitPerFilePercent

integer

Max percentage of bytes to scan from a file. The rest are omitted. The number of bytes scanned is rounded down. Must be between 0 and 100, inclusively. Both 0 and 100 means no limit. Defaults to 0. Only one of bytesLimitPerFile and bytesLimitPerFilePercent can be specified. This field can't be set if de-identification is requested. For certain file types, setting this field has no effect. For more information, see Limits on bytes scanned per file.
fileTypes[]

enum (FileType)

List of file type groups to include in the scan. If empty, all files are scanned and available data format processors are applied. In addition, the binary content of the selected files is always scanned as well. Images are scanned only as binary if the specified region does not support image inspection and no fileTypes were specified. Image inspection is restricted to 'global', 'us', 'asia', and 'europe'.
sampleMethod

enum (SampleMethod)

How to sample the data.
filesLimitPercent

integer

Limits the number of files to scan to this percentage of the input FileSet. Number of files scanned is rounded down. Must be between 0 and 100, inclusively. Both 0 and 100 means no limit. Defaults to 0.

FileSet

Set of files to scan.

JSON representation 
{
  "url": string,
  "regexFileSet": {
    object (CloudStorageRegexFileSet)
  }
}
Fields
url

string

The Cloud Storage url of the file(s) to scan, in the format gs://<bucket>/<path>. Trailing wildcard in the path is allowed.

If the url ends in a trailing slash, the bucket or directory represented by the url will be scanned non-recursively (content in sub-directories will not be scanned). This means that gs://mybucket/ is equivalent to gs://mybucket/*, and gs://mybucket/directory/ is equivalent to gs://mybucket/directory/*.

Exactly one of url or regexFileSet must be set.
regexFileSet

object (CloudStorageRegexFileSet)

The regex-filtered set of files to scan. Exactly one of url or regexFileSet must be set.

CloudStorageRegexFileSet

Message representing a set of files in a Cloud Storage bucket. Regular expressions are used to allow fine-grained control over which files in the bucket to include.

Included files are those that match at least one item in includeRegex and do not match any items in excludeRegex. Note that a file that matches items from both lists will not be included. For a match to occur, the entire file path (i.e., everything in the url after the bucket name) must match the regular expression.

For example, given the input {bucketName: "mybucket", includeRegex: ["directory1/.*"], excludeRegex: ["directory1/excluded.*"]}:

  • gs://mybucket/directory1/myfile will be included
  • gs://mybucket/directory1/directory2/myfile will be included (.* matches across /)
  • gs://mybucket/directory0/directory1/myfile will not be included (the full path doesn't match any items in includeRegex)
  • gs://mybucket/directory1/excludedfile will not be included (the path matches an item in excludeRegex)

If includeRegex is left empty, it will match all files by default (this is equivalent to setting includeRegex: [".*"]).

Some other common use cases:

  • {bucketName: "mybucket", excludeRegex: [".*\.pdf"]} will include all files in mybucket except for .pdf files
  • {bucketName: "mybucket", includeRegex: ["directory/[^/]+"]} will include all files directly under gs://mybucket/directory/, without matching across /
JSON representation 
{
  "bucketName": string,
  "includeRegex": [
    string
  ],
  "excludeRegex": [
    string
  ]
}
Fields
bucketName

string

The name of a Cloud Storage bucket. Required.
includeRegex[]

string

A list of regular expressions matching file paths to include. All files in the bucket that match at least one of these regular expressions will be included in the set of files, except for those that also match an item in excludeRegex. Leaving this field empty will match all files by default (this is equivalent to including .* in the list).

Regular expressions use RE2 syntax; a guide can be found under the google/re2 repository on GitHub.
excludeRegex[]

string

A list of regular expressions matching file paths to exclude. All files in the bucket that match at least one of these regular expressions will be excluded from the scan.

Regular expressions use RE2 syntax; a guide can be found under the google/re2 repository on GitHub.

SampleMethod

How to sample bytes if not all bytes are scanned. Meaningful only when used in conjunction with bytesLimitPerFile. If not specified, scanning would start from the top.

Enums
SAMPLE_METHOD_UNSPECIFIED No sampling.
TOP Scan from the top (default).
RANDOM_START For each file larger than bytesLimitPerFile, randomly pick the offset to start scanning. The scanned bytes are contiguous.

BigQueryOptions

Options defining BigQuery table and row identifiers.

JSON representation 
{
  "tableReference": {
    object (BigQueryTable)
  },
  "identifyingFields": [
    {
      object (FieldId)
    }
  ],
  "rowsLimit": string,
  "rowsLimitPercent": integer,
  "sampleMethod": enum (SampleMethod),
  "excludedFields": [
    {
      object (FieldId)
    }
  ],
  "includedFields": [
    {
      object (FieldId)
    }
  ]
}
Fields
tableReference

object (BigQueryTable)

Complete BigQuery table reference.
identifyingFields[]

object (FieldId)

Table fields that may uniquely identify a row within the table. When actions.saveFindings.outputConfig.table is specified, the values of columns specified here are available in the output table under location.content_locations.record_location.record_key.id_values. Nested fields such as person.birthdate.year are allowed.