Cockroach Labs, the company behind CockroachDB, the global SQL database for building worldwide cloud services, yesterday announced Managed CockroachDB and the latest release of its mainstay CockroachDB 2.1.
Businesses are increasingly migrating their workloads to the cloud. However, traditional database technologies weren’t built for it and are therefore often unable to leverage everything the cloud has to offer: close to unlimited resources within a facility, across facilities, continents, and even over different cloud providers. When Cockroach Labs released its inaugural database in May 2017, its goal was to address these issues in a straightforward way. In the words of CEO Spencer Kimball in his opening blog post for the company, “The mission is simple, but as big and as broad as any we could happily commit to: Make Data Easy”.
Google and AWS each has their own globally-distributed database service: Google has Google Spanner and AWS, AWS Aurora. CockroachDB is the only smaller company currently able to compete with the tech giants. Two of its main advantages in contrast to Google and AWS is precisely that it doesn’t maintain the status quo, nor does it lock its customers into a specific vendor. You can deploy CockroachDB onto your own hardware, whether on-premises, in the cloud, or on top of a container/scheduler solution. As CockroachDB is cloud agnostic, it is capable of migrating its customers over from one cloud service provider to another at peak load without any downtime if business needs demand it.
Managed CockroachDB
The launch of Managed CockroachDB marks CockroachDB’s entry into the database-as-a-service (DBaaS) market for companies without expertise deploying globally-distributed database systems who want a managed option. Clients include the finance, gaming, health and retail industries. The target market includes businesses looking to grow into markets subject to data sovereignty regulations.
Demand for SQL databases with the capabilities necessary to serve customers located across geographies is growing. CockroachDB has typically been deployed on-premise by companies who lack the internal expertise to deploy and manage rapidly evolving new technologies.
“We’re launching Managed CockroachDB to meet the surging demand from customers who prefer to outsource the operational burden of deploying and managing a geo-distributed database,” says Cockroach Labs CEO and co-founder Spencer Kimball. “We’ve been seeing significant migration activity away from Oracle, AWS Aurora, and Cassandra, and we’re now able to get our customers to market faster with Managed CockroachDB.”
Features
Managed CockroachDB describes itself as “the first cloud-neutral, geo-distributed SQL DBaaS”. It is initially launching on Google Cloud Platform (GCP) and Amazon Web Services (AWS).
Features include:
- handling of hardware provisioning,
- setup and configuration of customers’ managed clusters for performance optimization
- automatic upgrades to the latest releases
- hourly incremental backups of data
- 24×7 monitoring
- enterprise grade security
- geo-partitioned clusters – to reduce latency by locating data as close as possible to the customer
- row-level data-domiciling constraints – to provide a guarantee of compliance with data localization requirements, such as GDPR
- automatic replication of data across three availability zones for single region deployments
CockroachDB 2.1
The latest version of CockroachDB 2.1 was released at the same time, which offers migration tools for MySQL and PostgreSQL, in addition to various performance and scale improvements. CockroachDB 2.1 has gone through testing to handle 5x more transactional volume than 2.0 and over 50x the transactional volume as that of Amazon Aurora, according to tests run by CockroachDB, who ran the comparison using the industry-standard TPC-C benchmark.
“I don’t fundamentally believe in Swiss Army knives or solutions that solve every problem, but CockroachDB gets surprisingly close to solving a large number of relevant challenges that we see in the industry,” says Simon Kissler, Associate Director of Emerging Technology at Accenture.
Some of its features include:
Data Partitioning & Scaling Out – CockroachDB is a transactional database engine built on top of a high-performance distributed key-value store. This means the data is truly partitioned across the nodes of the database cluster, which enables scaling out in order to gain more storage space.
Operations – Similarly to many other modern DevOps tools, CockroachDB ships as a single, self-contained binary file that acts as the following:
- Server
- Operational tool (cluster status, replication config)
- Simple SQL client
SQL – ANSI-SQL is fully supported. This includes foreign key constraints and transactions and join queries. There are also some custom extensions, such as interleaved tables. Indeed the API is wire-compatible with PostgreSQL, so any client that is able to connect to Postgres should also be able to talk to CockroachDB. This includes standard client libraries such as JDBC drivers and 3rd party tools like SquirrelSQL.
ACID Transactions – ACID transactions are supported in CockroachDB – to be used in the standard way, via SQL commands like BEGIN or COMMIT. As there is a wire compatibility with PostgreSQL, the majority of tools and client libraries will continue to use it as usual.
Containerised deployments – The CockroachDB design assumes the likelihood of deploying into a containerised / orchestrated environment, such as Kubernetes.
Operational tools – The admin-style tools packaged into CockroachDB comprise of a web interface and a command line tool. The web interface is used for monitoring – it displays metrics about the CockroachDB cluster such as query latencies, logs, data volumes, nodes, etc. It is available and exposed by default. It cannot be disabled. The command-line tool accesses CockroachDB through the data port and gives access to the creation of users, certificates and configuriation replication of various tables, in addition to giving information about the cluster (such as list of nodes, data volumes, ranges and replication).
A detailed architecture overview can be accessed here.
Customer Deployments
Since the release of CockroachDB 1.0 in May 2017, the company has witnessed one million CockroachDB deployments and has a customer base spread out over four continents.
Business Use Cases
Tierion: a blockchain company delivering a “global proof engine”
Tierion uses CockroachDB as the power behind its distributed blockchain ledger, which generates more than 100M proofs each day. CockroachDB stores Tierion’s entire internal blockchain and has a copy of its intermediary proofs – both in near real-time. In doing so, it significantly increases the number of transactions per second a typical blockchain network can handle.
In making the decision of who to work with, Tierion carried out a thorough evaluation of various databases including blockchains such as Ethereum and cloud services like Google Spanner, and decided that CockroachDB allowed them to have a consistent view of their ledger yet also operate on a global scale.
Glenn Rempe, VP of Engineering at Tierion explained the decision to use CockroachDB: “Tierion needed a database with very strong consistency, scalable throughput, that could be deployed anywhere. We evaluated many options and CockroachDB was the clear winner. It’s the beating heart of our architecture”.
Kindred Group: a next generation online gaming platform
Kindred Group uses CockroachDB to power its gaming platform. The company is one of the world’s top gaming companies. It is imperative that play can take place in real time, thus low end-to-end latencies delivered to a globally distributed user base are an essential. Similarly, automated regulatory compliance was a must in order that Kindred Group can be accessed worldwide.
Kindred became Cockroach Labs’ first design partner after significant evaluations of other technologies. The geo-partitioning feature available in CockroachDB was built in response to Kindred’s needs to keep data as close to users as possible (all over the world), providing a zero to low latency and making regulation compliance and data security as straightforward as possible.
“The CockroachDB 2.0 release sees the launch of the geo-partitioning feature we designed in partnership with Cockroach Labs. Once fully deployed it will enable us to domicile data more easily than we ever thought possible, significantly reducing latency and very importantly allowing us to keep our services compliant with the many different regulatory environments we operate in. CockroachDB is becoming the rock-solid foundation for our next generation of infrastructure,” said hMarcus Smedman, CTO of Kindred Group.
Baidu: a $90B Internet Company using CockroachDB to Automate Operations for Certain Apps
Baidu, a top ten global internet company, is using CockroachDB to automate operations for specific large-scale applications. The applications access 2TB of data with 50 million inserts a day. CockroachDB supplies Baidu’s DBA team with a distributed database that is capable of scaling horizontally as well as offering the SQL interface that app developers typically use. CockroachDB is able to store increasing amounts of data while simultaneously offering support of continuous inserts with highly concurrent and real-time access. No middleware is required so operators gain on time and reduced complexity. All the hardware that Baidu provisions to run CockroachDB has been used to serve live application traffic.
Baidu’s DBA team still contributes to CockroachDB, assisting in the build of new features to improve its usability.
Conclusion
CockroachDB is still a young product, which is continuing to evolve at great speed. Many of its tools are in the process of development. It is open source so it is being built on continually by developers. Its core team is also active and responsive. Crucially, there is no vendor lock-in.
CockroachDB offers full ANSI SQL support and is compatible with PostgreSQL. Its replica placement rules are highly sophisticated and it supports containerisation and schedulers. It is a cloud ready fault-tolerant system with horizontal scalability across the globe to where customers are. With the new release, it is available as a fully managed and cloud-neutral DBaaS.
As its own team admits, there are some challenges and use cases it is not the right choice for. The tooling is relatively immature. Its web UI can’t be turned off and doesn’t have access controls. Admin access uses the same port as data access. Configuration can be challenging for more advanced replica placements.
OpenCredo describes CockroachDB as “filling an important gap and offering the transactionality and richness of a SQL database while also providing the scale-out, always-on, multi-active nature of many NoSQL databases”.
Indeed one of its most attractive features is the fact that it is ultra-resilient, allowing it to stay up-and-running under the majority of circumstances. It achieves this via replication, so there is a backup ready if needed.
CEO Spencer Kimball has made it clear from the start that another of its most persuasive aspects is that there is no vendor lock-in; “You can migrate a Cockroach cluster from one cloud to another with no down time,” Kimball told TechCrunch.
The issuing of the managed services sees the open source SQL database attempting to make inroads into the mainstream beyond technical early adopters. It eliminates the need for heavy lifting in installing and running a database and its related infrastructure. This is the only way that it will grow into a true rival to Google Spanner and AWS Aurora. How much it will catch on remains to be seen. The company’s CEO is optimistic: “We’ve been seeing significant migration activity away from Oracle, AWS Aurora, and Cassandra, and we’re now able to get our customers to market faster with Managed CockroachDB,” Kimball said.
Until now, the company has largely been gaining a profit from its enterprise version, which offers backup, support and other premium services. The company will now be able to achieve a more direct revenue stream from customer subscriptions.