Community

Provisioning basics with Terraform

Hi, I´m looking fo best practices using terraform for provisioning infrastructure basics. When I rool-out a datacenter using terraform, I can provision (file, remore-exec) only nodes connected to a public network (right?). How should I provision nodes connected only to a private lans? Does it make sense to run a terraform-node in my datacenter to reach the "private" nodes? Any experiences on this? Many thanks, Uli

 
  • **bold**
  • _italics_
  • `code`
  • ```code block```
  • # Heading 1
  • ## Heading 2
  • > Quote
 

OK, a terraform backend (with remote executon) could be the solution, right?

Hello ulise,

I apologize for the delayed response, but I wanted to test and validate a few things before replying. This ended up taking a bit more time than expected.

There are a couple options for deploying and bootstrapping servers on an internal network with Terraform. As you suggested, you could run Terraform on an existing server that is part of the private network. Of course, this would require some pre-configuration - an existing Virtual Data Center, management server, etc.

You could also roll your own open source or commercial VPN solution. This would allow you to connect into the network from a remote client or perhaps define a site-to-site VPN between your location or another Virtual Data Center at ProfitBricks. You can find a number of VPN related articles by searching for "VPN" here on the DevOps Central site.

A potentially better solution, and the one I wanted to test and validate, uses the bastion provisioner connection included with Terraform. You can review the documentation here:

Connecting through a Bastion Host with SSH

A bastion server is a relatively secure and minimal server that sits between the public and private networks. The bastion provisioner connection can tunnel provisioning requests through the bastion server similar to how an SSH tunnel operates.

To further automate a bastion deployment, Terraform can actually build and configure the bastion server along with the private server infrastructure. Further internal server provisioning requests are then be forwarded through the bastion server. Here is a Terraform configuration I put together that does just that:

// Global variables
variable "location" {
  default = "us/las"
}

variable "image_alias" {
  default = "centos:latest"
}

variable "nameserver" {
  default = "208.94.37.18"
}

variable "private_key_path" {
  description = "Path to file containing private key"
  default     = "/path/to/ssh/id_rsa"
}

variable "ssh_public_keys" {
  description = "List of SSH Keys to be added to the VMs"
  default     = [
    "/path/to/ssh/id_rsa.pub"
  ]
}

// Virtual Data Center
resource "profitbricks_datacenter" "main" {
  name        = "Example Terraform"
  location    = "${var.location}"
  description = "Example datacenter with bastion"
}

// Public LAN
resource "profitbricks_lan" "public_lan" {
  datacenter_id = "${profitbricks_datacenter.main.id}"
  public        = true
  name          = "public"
}

// Public LAN
resource "profitbricks_lan" "private_lan" {
  depends_on    = [ "profitbricks_lan.public_lan" ]
  datacenter_id = "${profitbricks_datacenter.main.id}"
  public        = false
  name          = "private"
}

// IP block
resource "profitbricks_ipblock" "public_ip" {
  location = "${var.location}"
  size     = 1
}

// Build bastion server
resource "profitbricks_server" "bastion" {
  name              = "bastion"
  datacenter_id     = "${profitbricks_datacenter.main.id}"
  cores             = 1
  ram               = 1024
  cpu_family        = "AMD_OPTERON"
  availability_zone = "ZONE_1"

  volume {
    name              = "system"
    image_name        = "${var.image_alias}"
    size              = 5
    disk_type         = "SSD"
    availability_zone = "AUTO"
    ssh_key_path      = [
      "${var.ssh_public_keys}"
    ]
  }

  nic {
    name = "public"
    lan  = "${profitbricks_lan.public_lan.id}"
    ip   = "${profitbricks_ipblock.public_ip.ips.0}"
    dhcp = true
  }
}

// Connect bastion gateway NIC
resource "profitbricks_nic" "gateway_nic" {
  datacenter_id = "${profitbricks_datacenter.main.id}"
  server_id     = "${profitbricks_server.bastion.id}"
  lan           = "${profitbricks_lan.private_lan.id}"
  name          = "private"
  dhcp          = true
}

// Provision bastion IP forwarding
resource "null_resource" "bastion_provisioner" {
  depends_on = [ "profitbricks_nic.gateway_nic" ]

  provisioner "remote-exec" {
    inline = [
      "echo 'net.ipv4.ip_forward = 1' >> /etc/sysctl.conf",
      "sysctl -p",
      "firewall-cmd --permanent --direct --passthrough ipv4 -t nat -I POSTROUTING -o eth0 -j MASQUERADE",
      "firewall-cmd --permanent --direct --passthrough ipv4 -I FORWARD -i eth1 -j ACCEPT",
      "firewall-cmd --reload"
    ]

    connection {
      private_key = "${file("${var.private_key_path}")}"
      host        = "${profitbricks_server.bastion.primary_ip}"
      user        = "root"
    }
  }
}

// Build internal node
resource "profitbricks_server" "node1" {
  depends_on        = [ "profitbricks_nic.gateway_nic" ]
  name              = "node1"
  datacenter_id     = "${profitbricks_datacenter.main.id}"
  cores             = 1
  ram               = 1024
  availability_zone = "ZONE_1"
  cpu_family        = "AMD_OPTERON"

  volume {
    name              = "system"
    image_name        = "${var.image_alias}"
    size              = 5
    disk_type         = "SSD"
    ssh_key_path      = ["${var.ssh_public_keys}"]
    availability_zone = "AUTO"
  }

  nic {
    lan  = "${profitbricks_lan.private_lan.id}"
    dhcp = true
  }

  provisioner "remote-exec" {
    inline = [
      "route add default gw ${profitbricks_nic.gateway_nic.ips.0}",
      "echo 'nameserver ${var.nameserver}' > /etc/resolv.conf",
      "yum install -y mariadb"
    ]

    connection {
      private_key  = "${file("${var.private_key_path}")}"
      bastion_host = "${profitbricks_server.bastion.primary_ip}"
      bastion_user = "root"
      bastion_private_key = "${file("${var.private_key_path}")}"
    }
  }
}

It should be fairly straightforward, but here are a couple important notes:

  • I enable IP forwarding on the bastion server which the internal node1 will use as a default gateway. This will allow external connectivity for internal server(s) which is necessary for downloading packages.
  • The internal server route is not persistent. It will be lost if the internal server is rebooted. This can be adjusted to suit your needs.
  • ProfitBricks will soon be releasing a NAT feature for internal NICs. The Terraform provider and Go SDK already support the parameter, but it has not yet been enabled at ProfitBricks. When available, this may negate the need for the IP forwarding and default gateway configuration.
  • The IP forwarding configuration is specifically for CentOS 7. It will need to be modified to accommodate Ubuntu or other distributions.
  • The bastion provisioner block is located outside of the profitbricks_server resource under a separate null_resource. This is done purposely to workaround a build issue.
  • The specified nameserver is for the ProfitBricks us/las location; it may need to be adjusted if the location is changed.
  • The private LAN depending on the public LAN is intentional and is necessary to workaround a known issue with LANs provisioned through the ProfitBricks Cloud API.
  • The bastion server should be as secure as possible, so it is highly advised add the appropriate network firewall rules to the Terraform config. I only omitted these for brevity.
  • Sorry!!! Was out of the topic for some time :-( Sounds good. Will try. Many thanks. Uli