Tutorials

Use the ProfitBricks REST API with Python Part 1

ProfitBricks not only offers an easy-to-use GUI – the Data Center Designer (DCD) – we also provide an API that is as capable as the DCD. Via this API, you can easily automate tasks such as the resource management or the provisioning of entire data centers.

The ProfitBricks REST API is maintained and enhanced on a regular basis to make sure it is up-to-date. Furthermore, there are SDKs for a number of programming languages that make it easy to use the API.

This article explains how to use the ProfitBricks REST API for retrieving information about a virtual data center and its components.

Part 1 will focus on the following:

  • Available virtual data centers (VDC) and location
  • Available images and snapshots
  • Reserved IP blocks
  • Servers and storage devices of a VDC
  • A VDC’s networks

Retrieving Global Resources

The ProfitBricks SDK for Python makes it easier to use the ProfitBricks API because the request/response protocol is encapsulated in HTTPS. As a consequence, you only need to call simple methods in order to retrieve data. In case of an error, an exception is thrown and this contains the HTTP status code. The further error handling is then up to the user.

Global resources are independent of a particular virtual data center. Global resources include a list of existing virtual data centers and resources that are available to all VDCs, e.g., customer and ProfitBricks images, snapshots, and reserved IP blocks.

You can retrieve global resources with only a few lines of code:

from profitbricks.client import ProfitBricksService

# login to ProfitBricks
pbclient = ProfitBricksService("user", "password")

# get all my data centers
datacenters = pbclient.list_datacenters()

# get all accessible images
images = pbclient.list_images()

# get all my snapshots
snapshots = pbclient.list_snapshots()

# get all my reserved IP blocks
ipblocks = pbclient.list_ipblocks()

The requested entities are returned in JSON format. You can manage the extent of what is returned by applying the depth parameter.

pbclient.list_datacenters(0)

returns the ID of all data centers:

{u'items': [
  {u'href': u'https://api.profitbricks.com/rest/datacenters/186c87fc-964b-4f28-8f1d-0cc0d5662647',
   u'type': u'datacenter',
   u'id': u'186c87fc-964b-4f28-8f1d-0cc0d5662647'},
  {u'href': u'https://api.profitbricks.com/rest/datacenters/48094e65-aa30-4943-8471-58b45d60c221',
   u'type': u'datacenter',
   u'id': u'48094e65-aa30-4943-8471-58b45d60c221'}
],
 u'href': u'https://api.profitbricks.com/rest/datacenters',
 u'type': u'collection',
 u'id': u'datacenters'
}

Applying the default value depth=1 to the same statement pbclient.list_datacenters(1) returns information about location, name and status of the data center:

{u'items': [
  {u'id': u'186c87fc-964b-4f28-8f1d-0cc0d5662647',
   u'entities': {
    u'loadbalancers': {
     u'href': u'https://api.profitbricks.com/rest/datacenters/186c87fc-964b-4f28-8f1d-0cc0d5662647/loadbalancers',
     u'type': u'collection',
     u'id': u'186c87fc-964b-4f28-8f1d-0cc0d5662647/loadbalancers'
},
u'lans': {
 u'href': u'https://api.profitbricks.com/rest/datacenters/186c87fc-964b-4f28-8f1d-0cc0d5662647/lans',
 u'type': u'collection',
 u'id': u'186c87fc-964b-4f28-8f1d-0cc0d5662647/lans'
},
u'volumes': {
 u'href': u'https://api.profitbricks.com/rest/datacenters/186c87fc-964b-4f28-8f1d-0cc0d5662647/volumes',
 u'type': u'collection',
 u'id': u'186c87fc-964b-4f28-8f1d-0cc0d5662647/volumes'
},
u'servers': {
 u'href': u'https://api.profitbricks.com/rest/datacenters/186c87fc-964b-4f28-8f1d-0cc0d5662647/servers',
 u'type': u'collection',
 u'id': u'186c87fc-964b-4f28-8f1d-0cc0d5662647/servers'
}
   },
   u'href': u'https://api.profitbricks.com/rest/datacenters/186c87fc-964b-4f28-8f1d-0cc0d5662647',
   u'type': u'datacenter',
   u'properties': {
    u'location': u'de/fra',
    u'name': u'API-Playgroud',
    u'version': 2,
    u'description': u'Try and Test for API script samples '
   },
   u'metadata': {
    u'lastModifiedDate': u'2016-01-15T16:16:25Z',
    u'lastModifiedBy': <username>',
    u'state': u'AVAILABLE',
    u'etag': u'1cdf27d84b4d8274f9c003c8eb50ede2',
    u'createdBy': u'<username>',
    u'createdDate': u'2016-01-15T16:16:25Z'
   }
  },
  ### further data centers ###
 ],
 u'href': u'https://api.profitbricks.com/rest/datacenters',
 u'type': u'collection',
 u'id': u'datacenters'
}

Increasing the depth parameter for the above global statement will not return more data. As is shown in the section “Requesting Server and Storage Inventory”, a single statement at a different entity level, e.g., a server, can return far more data.

The example script pb_datacenter_inventory.py returns the global information of images, snapshots and reserved IP blocks and writes them to the corresponding CSV files.

The statement python pb_datacenter_inventory.py –i writes information about images and snapshots to the CSV file pb_datacenter_images.csv as shown in the following example:

Example CSV images

The statement python pb_datacenter_inventory.py –b writes information about reserved IP blocks to the CSV file pb_datacenter_ipblocks.csv as shown in the following example:

Example CSV ipblocks

Requesting Server and Storage Inventory

This section will focus on requesting detailed information about virtual data centers, their servers and storage devices. The following information is requested and written to a CSV file as an inventory overview:

  • ID and name of a VDC and the location of the data center
  • For all servers of a VDC:
  • State
  • License type (operating system type e.g. Linux/Windows)
  • Cores, RAM, number of NICs, number of attached storage devices
  • Total size of attached storage devices
  • Date of creation and last changes made
  • For all storage devices of a VDC:
  • ID and name
  • Status
  • License type (operating system type e.g. Linux/Windows)
  • Size
  • ID of the server to which the storage device is attached
  • Date of creation and last changes made.

Here is an example of what the result should look like:

Example CSV inventory

In order to retrieve the data, a list of all virtual data centers is created. The default value depth = 1 also returns additional information about the data center, e.g., name and location.

from profitbricks.client import ProfitBricksService

# login to ProfitBricks
pbclient = ProfitBricksService("user", "password")

# get all my data centers
datacenters = pbclient.list_datacenters()
for dc in datacenters['items'] :
  dcid = dc['id']
  dc_name = dc['properties']['name']
  dc_location = dc['properties']['location']
  # .. do more for this dc .. #

The following statements return server and storage devices:

servers = pbclient.list_servers(dcid)
volumes = pbclient.list_volumes(dcid)

The depth parameter can be added to both statements in order to manage the level of detail (default: depth = 1). There is more than one approach to determine the NICs or storage devices of a server. It depends on the type of information you want to request which one you choose. It is important to keep an eye on the ratio of level of detail to the number of statements. The higher the level of detail (i.e., the higher the value of depth), the more extensive and resource-intensive the requests. It is recommended to keep the depth parameter low for less detailed requests. Writing only a few but more specific statements can be more effective in order to retrieve specific details.

We prefer the inventory overview shown above to also include information about NICs and attached storage devices. This can be achieved with the following statement:

servers = pbclient.list_servers(dcid, 3)
# depth 3 is enough to get into volume/nic level plus details

Now you can access most of the required properties at the level of the server entities, e. g.:

serverid = server['id']
# server_data contains server specific columns for later output
server_data = [server['type'], serverid, server['properties']['name'],
               server['metadata']['state']
              ]

As the licence type (Linux or Windows) is not a server property, it can only be accessed through the boot device. This could be a storage device, a CD-ROM or not exist at all:

# OS is determined by boot device (volume||cdrom), not a server property.
# Might even be unspecified.
bootOS = "NONE"
bootdev = server['properties']['bootVolume']
if bootdev is None :
  bootdev = server['properties']['bootCdrom']
  print("server %s has boot device %s" % (serverid, "CDROM"))
if bootdev is None :
  print("server %s has NO boot device" % (serverid))
else :
  bootOS = bootdev['properties']['licenceType']
server_data += [bootOS, server['properties']['cores'],
            server['properties']['ram']
           ]

The list of servers, which was created by applying depth = 3, also contains details about NICs and storage devices attached to the server and helps determine the number of NICs or storage volumes:

server_vols = server['entities']['volumes']['items']
n_volumes = len(server_vols) # count attached storage
server_nics = server['entities']['nics']['items']
n_nics = len(server_nics) # count attached NICs

The total size of storage devices attached to a server can only be determined by adding up the sizes of each attached storage device individually in a loop. In this loop, storage ID (key) and server ID (value) are mapped in a dictionary. When requesting information about a storage device, you can then return the ID of the server to which the storage is attached.

# this will build a hash to relate volumes to servers later
bound_vols = dict()
for server in servers['items'] :
# .. some other code .. #
  for vol in server_vols :
    total_disk += vol['properties']['size']
    licence_type = str(vol['properties']['licenceType'])
    bound_vols[vol['id']] = serverid

At this stage, you could return all storage devices that are attached to a server. However, you may also want to see storage devices that are not attached to a server, which is only possible by requesting all storage volumes:

volumes = pbclient.list_volumes(dcid, 2) # depth 2 gives max. details
for volume in volumes['items'] :
  volid = volume['id']
  # vol_data contains volume specific columns for later output
  vol_data = [volume['type'], volid, volume['properties']['name'],
              volume['metadata']['state'],
              volume['properties']['licenceType'], "", "", "", "",
              volume['properties']['size']
             ]
  connect = 'NONE'
  if volid in bound_vols :
    connect = bound_vols[volid]
  vol_data += [connect, volume['metadata']['createdDate'],
               volume['metadata']['lastModifiedDate']
              ]

See the example script pb_datacenter_inventory.py for more details.

The statement python pb_datacenter_inventory.py –d writes data on server and storage devices to the CSV file pb_datacenter_inventory.csv.

Requesting the Network Configuration of a Virtual Data Center

This section shows how to request detailed information about the network configuration of a Virtual Data Center. We want the following information to be written to a CSV file:

  • VDC name and ID and data center location
  • For all LANs of a VDC:
  • ID and name
  • Type of LAN (private or public)
  • State
  • Number of NICs
  • For all NICs of a LAN:
  • ID and MAC address
  • Type of IP address assignment (DHCP or static)
  • IP addresses assigned to a NIC
  • Name
  • Firewall rules
  • Type, ID and name of the server or load balancer with which the NIC is connected.

Here is an example of what the end result should look like:

Example CSV network config

In order to retrieve the data, firstly a list of all virtual data centers is created.

from profitbricks.client import ProfitBricksService

# login to ProfitBricks
pbclient = ProfitBricksService("user", "password")

# get all my data centers
datacenters = pbclient.list_datacenters()
for dc in datacenters['items'] :
  dcid = dc['id']
  dc_name = dc['properties']['name']
  dc_location = dc['properties']['location']
  # .. do more for this dc .. #

This statement returns the LANs of a data center:

lans = pbclient.list_lans(dcid)

We not only want a list of all NICs connected with a LAN, but also the server or load balancer to which they are connected.

In order to achieve this, the IDs and names of all load balancers are stored in a lookup hash. The depth parameter specifies the request so that it includes information at NIC level:

lbs = pbclient.list_loadbalancers(dcid, 2)
# build lookup hash for loadbalancer's ID->name
lbnames = dict(
            [(lb['id'],lb['properties']['name']) for lb in lbs['items']]
          )
lans = pbclient.list_lans(dcid, 3)
lan_inv = []
# lookup hash for server's ID->name
servernames = dict()
for lan in lans['items']:
  lan_data = ["LAN "+lan['id'], lan['properties']['name'],
              lan['properties']['public'], lan['metadata']['state']
             ]
  nics = lan['entities']['nics']['items']
  lan_data.append(len(nics))
  for nic in nics:
    # .. do something to connect to LB or Server .. #
    ips = [str(ip) for ip in nic_props['ips']]
    nic_data = [nic['id'], nic_props['mac'],nic_props['dhcp'], ips,
                nic_props['name'],
                nic_props['firewallActive'],
                servertype, serverid, servername
               ]
    # .. do more for this nic or lan .. #

The mapping of servers and NICs requires a workaround: The href property of a NIC needs to refer to href = "/datacenters/[dc-id]/servers/[server-id]/nics/[nic-id]"

From here you can determine server ID and server name. The ID, however, might not only refer to a server, but also to a load balancer.

In order to reduce the number of statements, servers that have already been requested are stored in a lookup hash (servernames):

# !!! this might also be a load balancer ID, although it's
'/servers/<id>/...' !!!
serverid = re.sub(r'^.*servers/([^/]+)/nics.*', r'\1', nic['href'])
if lbnames.has_key(serverid):
  servertype = "LB"
  servername = lbnames[serverid]
  print "server entry for %s is LOADBALANCER %s" % (serverid, servername)
else:
  servertype = "Server"
  if not servernames.has_key(serverid):
    server = pbclient.get_server(dcid, serverid, 0);
    servernames[serverid] = server['properties']['name']
    servername = servernames[serverid]
# end if/else(serverid)

For the full example, see pb_datacenter_inventory.py.

The statement python pb_datacenter_inventory.py –n writes data about network configuration to the CSV file pb_datacenter_networks.csv.

List of References