Write policy analysis to BigQuery

This page explains how to analyze Identity and Access Management (IAM) policies asynchronously and write results to BigQuery. The process is similar to analyzing IAM policies except the analysis result is written to BigQuery tables.

Before you begin

Enable the Cloud Asset API.

Roles required to enable APIs

To enable APIs, you need the serviceusage.services.enable permission. If you created the project, then you likely already have this permission through the Owner role (roles/owner). Otherwise, you can get this permission through the Service Usage Admin role (roles/serviceusage.serviceUsageAdmin). Learn how to grant roles.

Enable the API

You must enable the API in the project you will use to send the query. This doesn't have to be the same resource that you scope your query to.

Required roles and permissions

The following roles and permissions are required to run a policy analysis and export the results to BigQuery.

Required IAM roles

To get the permissions that you need to analyze a policy and export the results to BigQuery, ask your administrator to grant you the following IAM roles on the project, folder, or organization that you will scope your query to:

BigQuery Data Editor (roles/bigquery.dataEditor)
Cloud Asset Viewer (roles/cloudasset.viewer)
To analyze policies with custom IAM roles: Role Viewer (roles/iam.roleViewer)
To use the Google Cloud CLI to analyze policies: Service Usage Consumer (roles/serviceusage.serviceUsageConsumer)

For more information about granting roles, see Manage access to projects, folders, and organizations.

These predefined roles contain the permissions required to analyze a policy and export the results to BigQuery. To see the exact permissions that are required, expand the Required permissions section:

Required permissions

The following permissions are required to analyze a policy and export the results to BigQuery:

bigquery.datasets.create
bigquery.jobs.create
bigquery.tables.create
bigquery.tables.get
bigquery.tables.updateData
bigquery.tables.update
cloudasset.assets.analyzeIamPolicy
cloudasset.assets.searchAllResources
cloudasset.assets.searchAllIamPolicies
To analyze policies with custom IAM roles: iam.roles.get
To use the Google Cloud CLI to analyze policies: serviceusage.services.use

You might also be able to get these permissions with custom roles or other predefined roles.

Required Google Workspace permissions

If you want to see if a principal has certain roles or permissions as a result of their membership in a Google Workspace group, you need the groups.read Google Workspace permission. This permission is contained in the Groups Reader Admin role, and in more powerful roles such as the Groups Admin or Super Admin roles. See Assign specific admin roles for more information.

Analyze policies and export results

Console

In the Google Cloud console, go to the Policy Analyzer page.

Go to Policy Analyzer
In the Analyze policies section, find the query template you want to use, then click Create query. If you want to create a custom query, click Create custom query.
In the Select query scope field, select the project, folder, or organization that you want to scope the query to. Policy Analyzer will analyze access for that project, folder, or organization, as well as any resources within that project, folder, or organization.
Ensure that your query parameters are set:
- If you're using a query template, confirm the prefilled query parameters.
- If you're creating a custom query, set the resources, principals, roles, and permissions that you want to query for.
For more information about the types of queries you can create, see Analyze IAM policies.
In the pane labeled with the name of the query, click Analyze > Export result only. The Export results pane opens.
In the Set export destination section, enter the following information:
- Project: The project where your BigQuery dataset is located.
- Dataset: The BigQuery dataset that you want to export results to.
- Table: The prefix of the BigQuery tables to which the analysis results will be written. If a table with the specified prefix doesn't exist, BigQuery creates a new table.
Click Continue.
Optional: In the Configure additional settings section, select the options you want:
- Partitioning: Whether to partition the table. To learn more about partitioned tables, see Introduction to partitioned tables.
- Write preference: Specifies the action that occurs if the destination table or partition already exists. By default, if the table or partition already exists, BigQuery appends the data to the table or the latest partition.
Click Export.

Policy Analyzer runs your query and exports the results to the specified table.

gcloud

The AnalyzeIamPolicyLongrunning method allows you to issue an analysis request and get results in the specified BigQuery destination.

Before using any of the command data below, make the following replacements:

RESOURCE_TYPE: The type of the resource that you want to scope your search to. Only IAM allow policies attached to this resource and to its descendants will be analyzed. Use the value project, folder, or organization.
RESOURCE_ID: The ID of the Google Cloud project, folder, or organization that you want to scope your search to. Only IAM allow policies attached to this resource and to its descendants will be analyzed. Project IDs are alphanumeric strings, like my-project. Folder and organization IDs are numeric, like 123456789012.
PRINCIPAL: The principal whose access you want to analyze, in the form PRINCIPAL_TYPE:ID—for example, user:my-user@example.com. For a full list of the principal types, see Principal identifiers.
PERMISSIONS: A comma-separated list of the permissions that you want to check for—for example, compute.instances.get,compute.instances.start. If you list multiple permissions, Policy Analyzer will check for any of the permissions listed.
DATASET: The BigQuery dataset in the form projects/PROJECT_ID/datasets/DATASET_ID, where PROJECT_ID is the alphanumeric ID of your Google Cloud project and DATASET_ID is the ID of your dataset.
TABLE_PREFIX: The prefix of the BigQuery tables to which the analysis results will be written. If a table with the specified prefix doesn't exist, BigQuery creates a new table.
PARTITION_KEY: Optional. The partition key for BigQuery partitioned table. Policy Analyzer only supports REQUEST_TIME partition keys.
WRITE_DISPOSITION: Optional. Specifies the action that occurs if the destination table or partition already exists. For a list of possible values, see writeDisposition. By default, if the table or partition already exists, BigQuery appends the data to the table or the latest partition.

Execute the gcloud asset analyze-iam-policy-longrunning command:

Linux, macOS, or Cloud Shell

gcloud asset analyze-iam-policy-longrunning --RESOURCE_TYPE=RESOURCE_ID \
    --full-resource-name=FULL_RESOURCE_NAME \
    --identity=PRINCIPAL \
    --permissions='PERMISSIONS' \
    --bigquery-dataset=DATASET \
    --bigquery-table-prefix=TABLE_PREFIX \
    --bigquery-partition-key=PARTITION_KEY \
    --bigquery-write-disposition=WRITE_DISPOSITION