BigQuery automatically encrypts all data before it is written to disk. The data is automatically decrypted when read by an authorized user. By default, Google manages the key encryption keys used to protect your data. You can also use customer-managed encryption keys, and encrypt individual values within a table.
By default, Google manages the cryptographic keys on your behalf using the same hardened key management systems that we use for our own encrypted data. These systems including strict key access controls and auditing.
If you want to manage the key encryption keys used for your data at rest, instead of having Google manage the keys, use Cloud Key Management Service to manage your keys. This scenario is known as customer-managed encryption keys (CMEK).
By default, BigQuery encrypts customer content stored at rest. BigQuery handles and manages this default encryption for you without any additional actions on your part. First, data in a BigQuery table is encrypted using a data encryption key. Then, those data encryption keys are encrypted with key encryption keys, which is known as envelope encryption. Key encryption keys do not directly encrypt your data but are used to encrypt the data encryption keys that Google uses to encrypt your data.
If you want to control encryption yourself, you can use customer-managed encryption keys (CMEK) for BigQuery. Instead of Google managing the key encryption keys that protect your data, you control and manage key encryption keys in Cloud KMS.
Cloud KMS keys used to protect your data in BigQuery are AES-256 keys. These keys are used as key encryption keys in BigQuery, in that they encrypt the data encryption keys that encrypt your data.
Use the Google Cloud Console to determine the BigQuery service account ID, and provide the service account with the appropriate role to encrypt and decrypt using Cloud KMS.
Storing and encrypting data at Google's scale requires using a central cryptographic key management service with multiple layers of keys for the encrypted data. An example of multiple layer of keys is envelope encryption, which is the process of encrypting a key with another key.
Asymmetric encryption uses the public key portion of the asymmetric key and decryption uses the private key portion of the key. Cloud Key Management Service provides functionality to retrieve the public key and functionality to decrypt ciphertext that was encrypted with the public key. Cloud KMS does not allow direct access to the private key.
This topic shows how to use Cloud Key Management Service to protect sensitive application data on the client, before transmitting it across a network. Sensitive data is also called secret data or a secret.
BigQuery Encryption Functions — Part I: Data deletion/retention with Crypto Shredding
Use this pattern for crypto-shredding only: While the implemented logic leverages encryption functions to achieve crypto-shredding, due to some of the limitations that I’ll discuss further, it will NOT satisfy use-cases that require encryption for privacy and security. To make this more concrete, BigQuery’s AEAD functions can only accept encryption/decryption keys in the form of String or Binary. That poses a challenge where the key stored in the external mapping storage must eventually be transformed to String or Binary. This means that we CANNOT encrypt our encryption keys in Bigtable. In other words, in our example, Bigtable is storing the AEAD encryption/decryption keys in a plain-text JSON format. And while one can protect Bigtable access using GCP IAM policy, to satisfy encryption and privacy use-cases, the key should be stored encrypted. An ideal solution that would satisfy both crypto-shredding and encryption for privacy/security would involve storing the encryption key in a KMS system such as Cloud KMS. In addition to that, BigQuery should allow retrieval of the key from Cloud KMS instead of Binary or String format.