SaveOn.cloud

Labeling cloud resources allows you to group costs and create a manageable, business-relevant break-down of your expenses.

In this article we look at a strategy for applying labels consistently across your environment for support of cost management practices.

We look at how labels are used to support billing break-down, and an example labeling standard you may build on for your organization.

The terminology and implementation described in this article are specific to Google Cloud Platform (GCP). The concepts may apply to other platforms but some modification may be necessary.

Getting started with resource labeling doesn’t have to be a daunting task. The methods provided allow you to achieve 100% cost allocations without having to apply labels to 100% of your deployed resources. This significantly reduces initial and ongoing effort of applying labels, and allows you to gain cost optimization insights right away.

Labels - An Overview

Labels are key:value formatted metadata that can be applied to GCP resources. That means they are data that describe the resource. A label allows you to, as an example, describe a cost-center to associate the resource for billing purposes.

Labels can be used for purposes beyond billing management including operational and automation, but our focus is on cost management and bill reporting for this article.

A label, when applied to a resource or a project, gets maintained in the billing audit detail for that resource. This allows us to use the label when analyzing and reporting on billing data, giving us much better detail on our expenses than what the cloud provider bill provides by default.

Here is an example of a billing audit entry that includes label data (some fields omitted for brevity):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

[
  {
    "billing_account_id": "123456-123456-123456",
    "service": {
      "id": "5490-F7B7-8DF6",
      "description": "Stackdriver Logging"
    },
    "sku": {
      "id": "143F-A1B0-E0BE",
      "description": "Log Volume"
    },
    "usage_start_time": "2020-05-18 20:00:00 UTC",
    "usage_end_time": "2020-05-18 21:00:00 UTC",
    "project": {
      "id": "sample-project-id",
      "name": "sample-project-name",
      "labels": [
        {
          "key": "cost-center",
          "value": "samplecostcenter"
        },
        {
          "key": "environment",
          "value": "prod"
        }
      ],
    },
    "labels": [
      {
        "key": "cost-center",
        "value": "samplecostcenter"
      },
      {
        "key": "component",
        "value": "front-end"
    ],
    "location": {
      "location": "us",
      "country": "US",
      "region": null,
      "zone": null
    },
    "export_time": "2020-05-18 23:39:41.495 UTC",
    "cost": "0.0",
    "usage": {
      "amount": "951.0",
      "unit": "bytes",
      "amount_in_pricing_units": "8.856877684593201E-7",
      "pricing_unit": "gibibyte"
    },
    "invoice": {
      "month": "202005"
    },
    "cost_type": "regular"
  }
]

Lines #17-26 depict the label as applied to a GCP Project.

Lines #28-36 depict the label as applied to an individual GCP resource.

GCP Billing Accounts can export this audit data into a BigQuery table for you to use a Business Intelligence tool, such as Google’s DataStudio, for analysis. Accomplishing this will be covered in future articles.

Label Requirements and Limitations

Not all resources can be labeled. The list of resources on GCP that can be labeled are listed here. We can still achieve 100% cost allocation by inheriting labels from the project because all GCP resources live inside of a project. More on this below.

Format of labels must be as follows¹:

• Each resource can have multiple labels, up to a maximum of 64.
• Each label must be a key-value pair.
• Keys have a minimum length of -character and a maximum length of 63 characters, and cannot be empty. Values can be empty, and have a maximum length of 63 characters.
• Keys and values can contain only lowercase letters, numeric characters, underscores, and dashes. All characters must use UTF-8 encoding, and international characters are allowed.
• The key portion of a label must be unique. However, you can use the same key with multiple resources.
• Keys must start with a lowercase letter or international character.

What Labels Aren’t

GCE: Labels are different than network tags which applied to Google Compute Engine (GCE) instances. This can be misleading because other cloud providers use the term ‘tag’ for the same concept GCP uses “label”. In GCP, a GCE tag is used to manage firewall and routing logic within a VPC network.

GKE: GCP Labels are also independent from Kubernetes labels. If you use Google Kubernetes Engine (GKE), you can apply labels to elements within the Kubernetes engine for the purposes of management and automation within the GKE cluster. These labels do not have relevance outside of the GKE cluster, and do not inform billing data.

Define a Labeling Standard

Before you apply labels, you need to define a Cloud Infrastructure Labeling Standard for your organization. Without a standard for all users of the platform to follow you end up with inconsistent and unhelpful reporting.

As previously stated, this article’s primary concern is cost management. Resource labeling has uses beyond cost management, so your labeling standard will likely need to be extended to support those needs as well.

A simple labeling standard example:

cost-center is a logical grouping of costs that has meaning to the business. Review this article for guidance on how to define this.

environment defines the production state of the environment, e.g. prod or dev. Aside from the operational importance of this designation, this is helpful from a cost break-down to give insights into costs to develop and maintain infrastructure for non-production purposes. In many companies this is important to track for tax purposes because you may receive tax benefits related to R&D expenses.

service and component are examples of further break-down that may be helpful from a cost management standpoint, depending on the size and scale of the infrastructure being managed.

Depending on the complexity and scale of your environment, you may simplify or extend this standard further. An example might be the addition of a business-unit label.

Collaborate

Collaboration is a key element of every cost-optimization initiative. Only through collaboration and buy-in will you be successful building cost-consciousness into the culture, where engineers and product teams are considering cost of their designs on an ongoing basis.

Definition of your labeling standard is another opportunity to share with a broad audience what you seek to accomplish any why, and allow others to have a part in building the solution.

Develop this standard collaboratively involving engineering, accounting, product, and finance leaders who will be affected by this. Obtaining their input on its definition will help ensure it fits well within existing organizational language and structure, and will help them understand the purpose and need for the standard which will help obtain buy-in for organization-wide adoption.

Project Labeling / Resource Inheritance

You may be wondering why the cost-center label is optional except for Projects. We recommend a practice where resources inherit the cost-center project label designation of their GCP. This reduces engineering effort and greatly simplifies the adoption of labeling which in turn expedites the process of getting actionable cost-saving insights. This is also practically the only way to achieve 100% labeling coverage.

This inheritance is depicted as follows:

Note: Project-level label inheritance will not be reflected in Google’s built-in bill report viewer, nor will it work as a break-down for the budget alerting mechanism. We will instead use a Business Intelligence tool such as Google DataStudio to view the data and to support our cost-management practices. We will describe how to make this work in a future article when we work with the billing data exported to BigQuery.

Shared Projects

Most companies have at least some GCP projects where multiple cost-centers have resources within that project. In these instances we recommend designating and assigning a shared cost-center to the project, and then applying more specific cost-center labels to the resources. Resource-level labels will override Project-level labels for our cost reporting purposes. You may not get 100% of shared costs split out, but that is usually not material.

This may not be less helpful to you if you have one or few projects with many different cost-centers consuming resources within that project. In this case you will have to do more work in assigning labels to the individual resources to achieve cost break-down.

Conclusion

Once your labeling standard is defined, publish this standard somewhere people will expect it to be. e.g. an organization wiki. Share it as necessary to ensure knowledge of the standard to so that engineers can start using it.

Next steps on your cost-management journey include: technical application of labels to resources, billing data export, and then analysis of the data using Business Intelligence tools. We will cover each of these in upcoming articles.