Exasol User Group Karlsruhe

Exasol on Microsoft Azure – automatic deployment in less than 30 minutes

On April 9th the Exasol User Group was guest at Blue Yonder in Karlsruhe. Exasol meetups are a event to share technical ideas and challenges in personal conversations with other users and experts from the Exasol community.

Beside the usual networking we had two talks, Uwe Hesse was giving a deep dive into automatic index rebuilds in Exasol and I've been talking about Exasol deployment on Microsoft Azure and how we invested into automation to setup new clusters in less than 30 min.

The Exasol database has its roots in an on premise world and we have been one of the first customers running the database clusters on Microsoft Azure. Over the time we worked together to automate more and more of the provisioning and deployment.

We now have parametrized Azure Resource Templates to provision a cluster with license server, multiple virtual machines, os and data disks and networking. So setting up a cluster is as easy as running an az command:

az group deployment create -g exasol --mode Complete --template-file exasol.json --parameters @exasol.parameters.json

All the configuration is now done in a exasol.parameters.json file, where we can specify the node and storage sizing:

{
  "nodeVmSize": {
  "value": "Standard_E64v3"
},
"numExasolNodes": {
  "value": 3
},
"nodeNdisks": {
  "value": 5
},
"nodeDiskSizeGB": {
  "value": 1024
},
"vnetResourceGroup": {
  "value": "database-network"
},
"vnetName": {
  "value": "database-vnet"
},
"subnetName": {
  "value": "exasol-cluster1"
},
"networkAddressPrefix": {
  "value": "10.3.141."
}
}

The initial provisioning creates one database per cluster. Most of the time we have more complex requirements. To configure logging, additional storage volumes and databases, exasol provides an XML RPC Api. We have wrapped the API into an ansible module so that we can setup databases also in an declarative way.

Over the time the duration to provision a new database has decreased from multiple hours (lots of manual steps) to a fully automated 20 min setup. This does not only pay off in freeing our engineers from tendinous work, but also helped us to set up a much better testing infrastructure where we now provision multiple new production like environments a day and have much better end-to-end test coverage.

We plan to open source the ansible module to our JDA Github Account. Until then, if you want to get a peek preview just ping me via email.