Onboarding Google Cloud Platform - Threat Detection

Step by step guide

  1. Login to the GCP console using an email account with the Organization AdministratorOrganization Role AdministratorLogging Admin roles.
  2. Create a new project in your organization and use it to host the integrations's resources. (This ensures that quota limits on APIs are controlled in a project separate from production workloads). You will be automatically redirected to the project’s view (you can tell from the dropdown on the top).
  3. Ensure that the project has a billing account.
    1. Click the menu icon on the top-left corner and navigate to "Billing".
    2. If the project has no billing account, click on “Link A Billing Account”.
    3. Select the billing account you would like to associate the project with from the dropdown box.
  4. Navigate to “IAM & Admin” > “Service Accounts”. Create a service-account:
    1. Click on CREATE SERVICE ACCOUNT.
    2. Under Service account details, enter Warden GCP Log Ingestion as the Service account name, then enter GCP Log Ingestion API Access as the Service account description.
    3. Click on CREATE AND CONTINUE
    4. Grant the service account the following roles:
      1. Pub/Sub Subscriber
      2. Pub/Sub Viewer
  5. Note down the email address of the service account you just created (Warden GCP Log Ingestion). Click on the menu icon (3 dots) under the action column for the service account that was just created and click on “Create Key”.
  6. Leave the default JSON selected and click “Create”.
  7. Save the provided JSON file.
  8. Enable the necessary APIs.
    1. Either by using Google Cloud Shell:gcloud services enable iam.googleapis.com monitoring.googleapis.com logging.googleapis.com cloudresourcemanager.googleapis.com pubsub.googleapis.com cloudbilling.googleapis.com
    2. Or manually via the Google Cloud Console API Library, search for and enable the following APIs
  9. From the dropdown menu on the top, switch to the organization’s view. You must be at the organization’s view to set up organization-wide permissions for the service-account.
  10. Paste the saved service account credentials json in the API Credentials field below.
  11. Navigate to Pub/Sub > Topics. Create a new topic:
    1. Click on CREATE TOPIC.
    2. Enter topic details:
      1. Topic ID: Enter “horangi-warden-gcp-logs-integration-topic”.
      2. Uncheck “Add a default subscription”
      3. Set message retention duration to 7 days
    3. Click on CREATE TOPIC once again.
  12. Create an aggregated log sink:
    1. Note: Organization sinks can’t be created from the Google cloud console, so please use the gcloud command-line tool.
    2. Log in to GCP through your command line console using the command `gcloud auth login`
    3. Create a log sink:
      1. gcloud logging sinks create [sink-name] pubsub.googleapis.com/projects/[project-name]/topics/[topic-id] --include-children --organization=organizations/[org-id] --log-filter="protoPayload.@type=\"type.googleapis.com/google.cloud.audit.AuditLog\""
    4. Use the following field values:
      1. sink-name: horangi-warden-logs-sink
      2. project-name: [name of your created project]
      3. topic-id: horangi-warden-gcp-logs-integration-topic
      4. org-id: [Enter your organization ID]
    5. After the successful creation of the sink, remember to grant the GCP SA Logging Account with the Pub/Sub Publisher role.
    6. Note: Your environment may generate a lot of audit logs, and Pub/Sub throughputs are subject to quota limits. If you are running into those limits, you can split your logs over several topics to break up throughput.
  13. In the organization view, navigate to Logging > Logs Router. Select the Log Sink you have created above with the name: `horangi-warden-logs-sink`.
    1. Click EDIT SINK
    2. Set sink destination service to Cloud Pub/Sub topic
    3. Set sink destination to `pubsub.googleapis.com/projects/[PROJECT_ID]/topics/horangi-warden-gcp-logs-integration-topic`
    4. Click UPDATE SINK
  14. Go back to your created project then navigate to Pub/Sub Subscriptions. Create a new subscription:
    1. Click on CREATE SUBSCRIPTION.
    2. Enter subscription details:
      1. Subscription ID: Enter “horangi-warden-gcp-logs-integration-subscription”.
      2. Select a Cloud Pub/Sub topic: Select the topic created earlier.
      3. Delivery type: Select “Pull”.
      4. Acknowledgement Deadline: 600 seconds