Table of Contents
- Introduction
- Requirements
- Installation
- Add a User Account
- Update Ubuntu
- Add the Crate Repository
- Install Crate
- Configure a Firewall
- Crate Admin Web Interface
- Import Sample Data
- Summary
Introduction
Crate.IO (Crate) is an elastic distributed SQL data store. It combines a SQL interface with the document-oriented approach of a modern NoSQL database. Crate is built from a number of different technologies including Java, Facebook's Presto, Elasticsearch, Apache Lucene, and Netty. Further details can be found in the Wikipedia Entry for CrateIO.
Requirements
- ProfitBricks account and access to the DCD (Data Center Designer)
Installation
First create a new data center or use one that is already provisioned. Then add a composite instance with at least one CPU core and some memory and hard disk space. In the Image drop-down menu, select the ProfitBricks Images menu and then choose Ubuntu-15.10-server-2016-03-01.
You will also want to add Internet Access to the data center, and connect "NIC 0" to it.
It should end up looking something like this:
Go ahead and provision the changes using the large button in the lower right corner of the DCD.
Once provisioning is complete, log in as root via ssh to the newly-provisioned server. You may utilize the "Remote Console" inside the DCD if you prefer. Since this is the first time we have connected to the server, we will be forced to change the root password.
Add a User Account
Our newly-provisioned server does not have any users yet, so we will add one:
root@ubuntu:~# adduser tutorial
Adding user `tutorial' ...
Adding new group `tutorial' (1000) ...
Adding new user `tutorial' (1000) with group `tutorial' ...
Creating home directory `/home/tutorial' ...
Copying files from `/etc/skel' ...
Enter new UNIX password:
Retype new UNIX password:
passwd: password updated successfully
Changing the user information for tutorial
Enter the new value, or press ENTER for the default
Full Name []: Tutorial User
Room Number []:
Work Phone []:
Home Phone []:
Other []:
Is the information correct? [Y/n] Y
We want to use this account to manage the system, so we will grant it sudo access:
root@ubuntu:~# adduser tutorial sudo
Adding user `tutorial' to group `sudo' ...
Adding user tutorial to group sudo
Done.
Log out and back in as the new user to confirm that it is working properly.
Update Ubuntu
It is a good idea to keep the system up to date, so run:
sudo apt-get update
followed by:
sudo apt-get dist-upgrade
Once the updates are installed, reboot the server so that the newly-installed kernel will be loaded.
Add the Crate Repository
The Crate Team has packages available for recent Ubuntu releases. This tutorial is focused on Ubuntu 15.10 server so we will proceed by configuring the appropriate repository on the server. Based on the information found at Crate.io Ubuntu Installation, we need to make sure some key packages are installed:
tutorial@ubuntu:~$ dpkg --get-selections | grep software
libsoftware-license-perl install
python3-software-properties install
software-properties-common install
Both software-properties-common and python3-software-properties are present, so we will move on and run sudo add-apt-repository ppa:crate/stable which should result in output similar to this:
tutorial@ubuntu:~$ sudo add-apt-repository ppa:crate/stable
[sudo] password for tutorial:
CRATE Packages for Ubuntu
More info: https://launchpad.net/~crate/+archive/ubuntu/stable
Press [ENTER] to continue or ctrl-c to cancel adding it
gpg: keyring `/tmp/tmpalmz4cmn/secring.gpg' created
gpg: keyring `/tmp/tmpalmz4cmn/pubring.gpg' created
gpg: requesting key 3D089FBC from hkp server keyserver.ubuntu.com
gpg: /tmp/tmpalmz4cmn/trustdb.gpg: trustdb created
gpg: key 3D089FBC: public key "Launchpad PPA for CRATE" imported
gpg: Total number processed: 1
gpg: imported: 1 (RSA: 1)
OK
Since that completed successfully, we will update the list of available packages:
sudo apt-get update
Install Crate
Next we will run sudo apt-get install crate:
tutorial@ubuntu:~$ sudo apt-get install crate
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
ca-certificates-java fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libfontconfig1 libjpeg-turbo8
libjpeg8 liblcms2-2 libnspr4 libnss3 libnss3-nssdb libpython-stdlib libxi6 libxrender1 libxtst6 openjdk-8-jre-headless python python-minimal python2.7
python2.7-minimal x11-common
Suggested packages:
default-jre equivs cups-common liblcms2-utils openjdk-8-jre-jamvm libnss-mdns fonts-dejavu-extra fonts-ipafont-gothic fonts-ipafont-mincho ttf-wqy-microhei
ttf-wqy-zenhei fonts-indic python-doc python-tk python2.7-doc binutils binfmt-support
Recommended packages:
python:any
The following NEW packages will be installed:
ca-certificates-java crate fontconfig-config fonts-dejavu-core java-common libavahi-client3 libavahi-common-data libavahi-common3 libcups2 libfontconfig1
libjpeg-turbo8 libjpeg8 liblcms2-2 libnspr4 libnss3 libnss3-nssdb libpython-stdlib libxi6 libxrender1 libxtst6 openjdk-8-jre-headless python python-minimal python2.7
python2.7-minimal x11-common
0 upgraded, 26 newly installed, 0 to remove and 0 not upgraded.
Need to get 75.5 MB of archives.
After this operation, 169 MB of additional disk space will be used.
Do you want to continue? [Y/n]
Answering Y will allow the installation to proceed and install a number of packages.
We can stop and start crate using the service command:
tutorial@ubuntu:~$ sudo service crate stop
tutorial@ubuntu:~$ sudo service crate start
And confirm that it is running by looking for it in the output of ps.
tutorial@ubuntu:~$ ps waux |grep crate
crate 4209 86.3 7.1 3030784 146068 ? Ssl 22:37 0:02 java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8 -Dcrate -Des.path.home=/usr/share/crate -Des.config=/usr/share/crate/config/crate.yml -cp /usr/share/crate/lib/crate-app-0.54.7.jar:/usr/share/crate/lib/*:/usr/share/crate/lib/sigar/* -Des.path.home=/usr/share/crate -Des.path.conf=/etc/crate -Des.config=/etc/crate/crate.yml -Des.path.logs=/var/log/crate io.crate.bootstrap.CrateF
tutorial 4228 0.0 0.1 11796 2060 pts/0 S+ 22:37 0:00 grep --color=auto crate
Configure a Firewall
Let's stop the server and get a firewall setup to provide some protection from unwanted visitors.
tutorial@ubuntu:~$ sudo service crate stop
Ubuntu provides the "Uncomplicated Firewall" tool, ufw, to manage iptables directly on the server. We also have the option to configure the ProfitBricks data center firewall using the DCD. For this tutorial we will use ufw. First we will check the status:
tutorial@ubuntu:~$ sudo ufw status verbose
Status: inactive
We will add two rules to restrict traffic to the server. One for ssh traffic on port 22, and one for crate traffic on port 4200. You will need to substitute the IP address that you are connecting from into these rules:
tutorial@ubuntu:~$ sudo ufw allow proto tcp from [your-ip-address-here] to any port 22
Rules updated
tutorial@ubuntu:~$ sudo ufw allow proto tcp from [your-ip-address-here] to any port 4200
Rules updated
Now we enable the firewall:
tutorial@ubuntu:~$ sudo ufw enable
Command may disrupt existing ssh connections. Proceed with operation (y|n)? y
Firewall is active and enabled on system startup
and verify the status:
tutorial@ubuntu:~$ sudo ufw status verbose
Status: active
Logging: on (low)
Default: deny (incoming), allow (outgoing), disabled (routed)
New profiles: skip
To Action From
-- ------ ----
22/tcp ALLOW IN [your-ip-address-here]
4200/tcp ALLOW IN [your-ip-address-here]
If you run into an issue accessing the server after the firewall is enabled, then use the "Remote Console" inside the DCD to disable it with the command sudo ufw disable and verify that the rules are correct.
Crate Admin Web Interface
Now that we have limited access to the server from our IP address, we can start up the database and take a look at the web interface. Open up a browser and go to https://SERVER_IP:4200/admin and you should see:
Import Sample Data
We did not step through the process of importing sample data from Twitter in the Get Started with Crate.IO on CentOS 7 tutorial, we will go ahead and do that now.
Selecting the i icon in the left-navigation section of the admin interface brings us to this screen:
Pressing the Import tweets for testing button takes us away to an authorization page:
Once we successfully Authorize app, data begins to stream in. Go ahead and press the Enough! Stop importing tweets button after some records have come in. You can return here later and import additional tweets.
Returning to the home screen of the admin app we now see that some sample data is available.
Summary
At this point, we have a single Crate node up and running on Ubuntu 15.10. If you are interested in learning how clustering works in Crate, keep an eye out for an upcoming tutorial covering that topic.
