SaveOn.cloud

Manage your Cloud Infrastructure Spend

Eliminate Abandoned Resources

in  , ,

You’re probably paying for unused and forgotten cloud resources.

These pesky resources linger without purpose. They hide under the radar, contributing to wasteful spend.

Let’s find and delete these abandoned resources.

What Are Abandoned Resources

Also referred to as orphaned, abandoned resources became unused but were not de-provisioned.

An example is a persistent disk. When you delete a Google Compute Engine (GCE) instance, depending on configuration, its disks may not be deleted. Sometimes this is good because you want to keep its data, but it is easily forgotten.

An abandoned resource provides no value. Detached from a service, they are inoperable, and not needed for any reason. They can be identified by the cloud provider…if you know where to look. You can see that an External IP address is not in use, or that a GCE instance is turned off.

Abandoned Resource Types

The following resources may become abandoned and should be monitored:

  • Compute Engine Instances that are turned off
  • Persistent Disks not attached to a Compute Engine Instance
  • Disk snapshots beyond our data retention needs or for disks that no longer exist
  • External IP addresses that are unused
  • Cloud SQL database instances that are turned off

Abandoned: Different than Unneeded

Not all unneeded resources are abandoned. An example is a storage bucket that is operational but no longer supporting anything. The cloud platform cannot tell us this resource is unneeded, because it cannot judge that it is doing nothing of value. It is still ready to respond to requests.

Unneeded resources are also a source of waste, but, they need a different monitoring process. All operating resources in your environment should be re-evaluated periodically. They should be reviewed by a human to understand their purpose and ongoing need. This is a process of lifecycle management, and will be discussed in a future article.

Detecting Abandoned Resources

The cloud platform can recognize when a resource becomes abandoned. It provides several methods for their detection.

In a small environment, it may be good enough to look for these resources once or twice a year. But at larger scale, where many engineers are making changes on a regular basis, it becomes important to automate this monitoring. Even with good provisioning practices it is easy for resources to be left behind.

We do not recommend deleting resources until you confirm they are not needed. There are situations where detached/unused resources are needed. For example a persistent disk supporting a Google Kubernetes Engine (GKE) PersistentVolumeClaim. These resources may be attached and used only when needed for scale or to bring up a user or job specific component.

Here are some of the methods you may use to detect abandoned resource on Google Cloud Platform (GCP).

CLI and GUI Detection

Both the web based Google Cloud Console as well as the CLI Cloud SDK gcloud tool can reveal unused resources.

These tools may be suitable for a small environments. The CLI tool can be expanded with scripts for more streamlined detection and reporting.

Example CLI use for listing powered off GCE instances:

gcloud compute instances list [--project projectID] | grep TERMINATED

Depending on the resource type, there may be helpful metadata you can use to confirm suitability for deletion. An example is the timestamp of when a persistent disk was detached from a Compute Engine instance. Helpful metadata might not be available by the web interface, but only available by the CLI or APIs.

For larger environments, on-going monitoring is recommended. Additionally, more sophisticated detection and reporting is needed to make the review easier. Using the provider APIs directly may be more appropriate for supporting this. We will look at their use next.

API Detection / Python Example

The following example is a simple Python script that will pull a list of all disks in a project that are not attached to a GCE instance. It uses Google’s API via Python libraries to retrieve a list of all disks across all locations. It then filters down to only those with no ‘users’ attribute, which means it is not attached to a GCE instance.

from pprint import pprint
from googleapiclient import discovery
from oauth2client.client import GoogleCredentials

yourProjectID = "##Project ID Here##"
gcp_credentials = GoogleCredentials.get_application_default()
service = discovery.build('compute', 'v1', credentials=gcp_credentials)
request = service.disks().aggregatedList(project=yourProjectID)
projectDisks = {}
while request is not None:
    try:
        response = request.execute()
    except:
        print("Error: ", sys.exc_info()[1])
    else:
        for location in response['items']:
            if 'disks' in response['items'][location]:
                if not 'users' in response['items'][location]['disks'][0]:
                        projectDisks.update({location: [response['items'][location]['disks']]})

    request = service.disks().list_next(previous_request=request, previous_response=response)

pprint(projectDisks)

API Detection / CloudCostMgmt Toolkit

We have provided a Python scripts similar to the previous example in our CloudCostMgmt Toolkit. You may use these script as-is, or as a starting point to customize for your needs.

This script looks through every project in your environment and reports abandoned resources. Use the output to review and verify the disks can be deleted.

cloudcostmgmt % python3 gcp/unattached-disk-list.py 
projectId,location,name,creationTimestamp,projectCostCenter,diskCostCenter,sizeGb,lastDetachTimestamp,type,consoleUrl
proj-id123,zones/us-central1-c,disk-import-boot-disk-scratch-bm190,2020-03-03T14:33:17.499-08:00,productteam,,14,2020-03-03T15:09:13.471-08:00,pd-ssd,https://console.cloud.google.com/compute/disksDetail/zones/us-central1-c/disks/disk-import-boot-disk-scratch-bm190?project=proj-id123
proj-id123,zones/us-central1-c,disk-import-boot-disk-scratch-rn8nx,2020-03-03T18:07:25.342-08:00,productteam,,14,2020-03-03T18:43:21.234-08:00,pd-ssd,https://console.cloud.google.com/compute/disksDetail/zones/us-central1-c/disks/disk-import-boot-disk-scratch-rn8nx?project=proj-id123
proj-id123,zones/us-central1-c,disk-import-boot-disk-scratch-w72xn,2020-03-04T08:51:36.606-08:00,productteam,,14,2020-03-04T09:27:44.102-08:00,pd-ssd,https://console.cloud.google.com/compute/disksDetail/zones/us-central1-c/disks/disk-import-boot-disk-scratch-w72xn?project=proj-id123
proj-id123,zones/us-central1-c,disk-importer-import-ovf-import-boot-disk-bm190,2020-03-03T14:33:17.445-08:00,productteam,,10,2020-03-03T15:09:13.471-08:00,pd-ssd,https://console.cloud.google.com/compute/disksDetail/zones/us-central1-c/disks/disk-importer-import-ovf-import-boot-disk-bm190?project=proj-id123
proj-id123,zones/us-central1-c,disk-importer-import-ovf-import-boot-disk-rn8nx,2020-03-03T18:07:25.472-08:00,productteam,,10,2020-03-03T18:43:21.234-08:00,pd-ssd,https://console.cloud.google.com/compute/disksDetail/zones/us-central1-c/disks/disk-importer-import-ovf-import-boot-disk-rn8nx?project=proj-id123
testproj-175620,zones/us-central1-a,test-www,2017-09-26T12:29:06.672-07:00,testcenter,,100,2020-05-15T09:16:39.277-07:00,pd-standard,https://console.cloud.google.com/compute/disksDetail/zones/us-central1-a/disks/daap-www?project=testproj-175620
testproj-175620,zones/us-central1-a,eevee-tableau-data,2017-11-17T09:35:31.342-08:00,testcenter,,500,2019-12-20T13:53:43.223-08:00,pd-standard,https://console.cloud.google.com/compute/disksDetail/zones/us-central1-a/disks/eevee-tableau-data?project=testproj-175620
testproj-175620,zones/us-central1-c,test-tableau,2017-11-16T11:26:28.474-08:00,testcenter,,100,2017-11-17T11:01:05.144-08:00,pd-ssd,https://console.cloud.google.com/compute/disksDetail/zones/us-central1-c/disks/daap-tableau?project=testproj-175620

We can take this CSV and re-format as needed to send to a team to confirm whether the disks are needed any longer.

Automating Abandoned Resource Detection

A one-time or periodic clean-up may be enough for small environments. But for large environments, where the scope is large and frequency of change is high, it will be worthwhile to automate this.

Accomplishing this automation will be a topic of a future article. But a high-level overview on how to do this will be as follows:

Cloud Scheduler daily trigger a Cloud Function which executes the CloudCostMgmt tool. This then reports via email/slack for review.

Through automation, you can reduce the ongoing effort of detecting abandoned resources. Additionally, you can detect them more rapidly to minimize their impact on your bill.

Summary

Eliminating abandoned resources is important in managing cloud costs and minimizing waste. Consider the examples provided, and adopt an approach that makes sense for your environment.

Share