Table of Contents
Introduction
Crate.IO (Crate) is an elastic distributed SQL data store. It combines a SQL interface with the document-oriented approach of a modern NoSQL database. Crate is built from a number of different technologies including Java, Facebook's Presto, Elasticsearch, Apache Lucene, and Netty. Further details can be found in the Wikipedia Entry for CrateIO.
Requirements
- Server with CentOS 7 installed
- Network access
- EPEL repository configured for installing the CLI utility
crash
Installation
The Crate Team maintains the official Yum repositories for RPM-based releases. This tutorial is focused on CentOS 7 so we will proceed by adding the repository to our server.
Enable the Crate Repository
All Crate packages are signed with GPG. To verify packages, the public key must be installed and then we can install the .rpm containing the Yum repository definition.
sudo rpm --import https://cdn.crate.io/downloads/yum/RPM-GPG-KEY-crate
Once the key is imported, we will install the repository.
sudo rpm -Uvh https://cdn.crate.io/downloads/yum/7/noarch/crate-release-7.0-1.noarch.rpm
Which should result in output like this:
[eknauer@centos ~]$ sudo rpm -Uvh https://cdn.crate.io/downloads/yum/7/noarch/crate-release-7.0-1.noarch.rpm
Retrieving https://cdn.crate.io/downloads/yum/7/noarch/crate-release-7.0-1.noarch.rpm
Preparing... ################################# [100%]
Updating / installing...
1:crate-release-7.0-1 ################################# [100%]
We can verify the repository is installed by running yum repolist
and looking for crate
in the output.
crate CRATE Packages for Enterprise Linux 7 28
Install Crate
We will proceed with the installation of Crate.
sudo yum install crate
Dependencies will be resolved and we are presented with a list of packages that will be installed or updated.
====================================================================================================================
Package Arch Version Repository Size
====================================================================================================================
Installing:
crate noarch 0.54.4-1 crate 43 M
Installing for dependencies:
java-1.8.0-openjdk-headless x86_64 1:1.8.0.71-2.b15.el7_2 updates 31 M
javapackages-tools noarch 3.4.1-11.el7 base 73 k
lksctp-tools x86_64 1.0.13-3.el7 base 87 k
python-javapackages noarch 3.4.1-11.el7 base 31 k
python-lxml x86_64 3.2.1-4.el7 base 758 k
tzdata-java noarch 2015g-1.el7 base 176 k
Transaction Summary
====================================================================================================================
Install 1 Package (+6 Dependent packages)
Total download size: 75 M
Installed size: 159 M
Is this ok [y/d/N]:
The package list may differ from what is shown above if you already have some of the packages installed. Confirm and proceed by typing y
Start the Service
When installation completes, we will see the following helpful information regarding the service:
CRATE has been successfully installed but not enabled or started.
To enable and start crate, run:
systemctl daemon-reload
systemctl enable crate.service
systemctl start crate.service
Let's start Crate by running:
sudo systemctl start crate.service
If you wish to have Crate run on startup, then enable the service:
sudo systemctl enable crate.service
Firewall Access
We need to enable access to Crate by opening up port 4200 through the local firewall.
[eknauer@centos]$ sudo firewall-cmd --zone=public --add-port=4200/tcp
success
If you want to make the change permanent, then add the --permanent
option. If you are running your server on a public IP address, and not on an internal network, then you may wish to scope access to a specific IP address.
We can double-check that port 4200 was added to the list:
[eknauer@centos]$ sudo firewall-cmd --zone=public --list-ports
443/tcp 80/tcp 4200/tcp
Verify
How do we know that Crate is running? We should be able to see a new java process:
[eknauer@centos]$ ps waux |grep crate
crate 11462 1.4 17.0 3209488 320552 ? Ssl 20:41 1:20 /bin/java -Xms256m -Xmx1g -Xss256k -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -Dfile.encoding=UTF-8 -Dcrate -Des.path.home=/opt/crate -Des.config=/opt/crate/config/crate.yml -cp /opt/crate/lib/crate-app-0.54.4.jar:/opt/crate/lib/*:/opt/crate/lib/sigar/* -Des.path.home=/opt/crate -Des.path.conf=/etc/crate -Des.config=/etc/crate/crate.yml -Des.path.logs=/var/log/crate io.crate.bootstrap.CrateF
We can also use curl
to see if the service is responding on port 4200:
[eknauer@centos]$ curl 127.0.0.1:4200
{
"ok" : true,
"status" : 200,
"name" : "Prime",
"cluster_name" : "crate",
"version" : {
"number" : "0.54.4",
"build_hash" : "a9613b13fccbf12a02493cc6b3aa3bf4afc6ccc4",
"build_timestamp" : "2015-10-22T11:23:24Z",
"build_snapshot" : false,
"es_version" : "1.7.3",
"lucene_version" : "4.10.4"
}
}
Web Interface
Lets take a look at the Crate web interface by accessing http://SERVER_IP:4200/admin
:
If you have a Twitter account, the web interface allows you to easily authenticate using OAuth against the Twitter API and import some data from the Twitter public stream. This handy feature makes it easy to start exploring Crate. If you choose to go through that process, the web interface will actually have some activity to graph.
CLI
There is also a Python-based command-line client available for Crate called crash
. Details on its installation and use can be found at GitHub Crate Crash. Crash can be installed quickly if pip
and python
are already available on the server. Note: Access to the EPEL repository also needs to be configured first.
sudo yum install python-pip
And then:
pip install crash
Upon successful installation, you can access crash
by invoking it from the command-line with a --hosts
parameter pointing to the Crate service:
crash --hosts 127.0.0.1:4200
This drops us into an interactive shell capable of executing typical SQL commands such as:
select * from tweets limit 5;
To exit crash
, enter \q
.
Summary
We have successfully installed Crate and have both web (GUI) and command-line interfaces (CLI) available to begin interacting with it. If you plan to create a production environment using Crate, please seek out additional guidance regarding resource requirements and security. Excellent documentation is available at the Official Crate Website, including links to a sample application. If you run into any issues following this tutorial, please post a comment here or open a discussion via the DevOps Community Portal.
Hi Eric,
nice guide. easy to use, short and simple - thanks for mentioning us and keep doing.
Couldn't see a way to use the web admin api securely over https :(