Accumulo
Apache Accumulo is a sorted, distributed key/value store that provides robust, scalable data storage and retrieval, designed for large-scale data management across clusters.
Publisher review
Apache Accumulo is a sorted, distributed key/value store that provides robust, scalable data storage and retrieval, designed for large-scale data management across clusters. It is built on top of Apache Hadoop's HDFS for storage and Apache ZooKeeper for consensus, making it a natural fit for organizations already invested in the Hadoop ecosystem. Accumulo is particularly suited for use cases requiring high security and fine-grained access control, such as government, finance, and healthcare, where cell-level security is a critical requirement. It is an open-source project under the Apache Software Foundation, offering a stable client API that follows LTM releases and semantic versioning, with extensive testing for each release.
Accumulo's key capabilities include its server-side programming mechanism called Iterators, which can modify key/value pairs at various points in the data management process, enabling custom data processing and filtering. The system supports cell-based access control, where every key/value pair has its own security label that limits query results based on user authorizations, providing granular security at the individual cell level. Accumulo is designed to scale horizontally across clusters using one or more HDFS instances, with nodes that can be added or removed dynamically as data volumes change. It offers strong consistency models, ensuring data integrity for transactional workloads.
In the market, Accumulo competes with other NoSQL databases like Apache HBase, Cassandra, MongoDB, and Azure Table Storage. Compared to HBase, Accumulo offers more advanced security features through cell-level access controls, while Cassandra provides higher write throughput and eventual consistency, making it better suited for time-series data. MongoDB offers a document model and flexible schema, which can be easier for developers but lacks the strong consistency and security features of Accumulo. Azure Table Storage is a managed service with simpler scaling but limited query capabilities. Accumulo's integration with the Hadoop ecosystem gives it an edge for organizations already using Hadoop, but it requires more operational expertise to deploy and manage.
The honest trade-offs with Accumulo include its operational complexity, as it requires a Hadoop cluster and ZooKeeper, increasing infrastructure and maintenance overhead. Its performance can be slower than Cassandra for write-heavy workloads due to its strong consistency and security checks. The learning curve is steep, especially for teams unfamiliar with Hadoop and Java-based systems. Additionally, the community and ecosystem are smaller compared to Cassandra or MongoDB, resulting in fewer third-party tools and resources. Accumulo is not a good fit for simple key-value lookups or real-time analytics where low latency is critical, as its design prioritizes security and batch processing over speed.
How it works
-
Sorted, distributed key/value store
Stores data as sorted key-value pairs across a cluster, enabling efficient range queries and ordered data access.
-
Cell-level security labels
Every key/value pair has a security label that restricts query results based on user authorizations, enabling fine-grained access control.
-
Server-side iterators
A programming mechanism that modifies key/value pairs during data processing, allowing custom filtering, aggregation, and transformations.
-
HDFS and ZooKeeper integration
Uses Apache Hadoop's HDFS for persistent storage and Apache ZooKeeper for consensus, ensuring reliability and scalability.
-
Horizontal scalability
Runs on clusters with one or more HDFS instances; nodes can be added or removed dynamically as data volume changes.
-
Strong consistency model
Provides strong consistency guarantees, ensuring that all reads return the most recent write, critical for transactional applications.
-
Stable client API
Follows LTM releases and semantic versioning, with extensive testing per release, ensuring backward compatibility and reliability.
Strengths and trade-offs
Strengths
- Accumulo provides cell-level security labels on every key/value pair, enabling fine-grained access control that is unmatched by most NoSQL databases.
- The server-side iterator framework allows custom data processing at scale, such as filtering, aggregation, and transformations, without moving data to the client.
- Accumulo integrates natively with Apache Hadoop's HDFS and ZooKeeper, leveraging existing Hadoop infrastructure for storage and consensus.
- It offers strong consistency models, ensuring data integrity for applications that require accurate, up-to-date reads after writes.
Trade-offs
- Accumulo requires a full Hadoop cluster and ZooKeeper to operate, increasing infrastructure complexity and operational overhead compared to standalone databases.
- Write performance can be slower than eventually consistent systems like Cassandra due to strong consistency checks and security label processing.
- The learning curve is steep for teams not familiar with Hadoop, Java, and distributed systems, requiring specialized skills for deployment and tuning.
- The community and ecosystem are smaller than those of Cassandra or MongoDB, resulting in fewer third-party tools, tutorials, and commercial support options.
Pricing context
Accumulo is open-source and free to use under the Apache License 2.0. There are no licensing fees, but users must cover infrastructure costs for Hadoop, ZooKeeper, and cluster hardware.
Getting started with Accumulo
-
Set up Hadoop and ZooKeeper
Install and configure Apache Hadoop HDFS and Apache ZooKeeper on your cluster. Ensure both services are running and accessible, as Accumulo depends on them for storage and consensus.
-
Download and install Accumulo
Download the latest stable Accumulo release from the Apache website. Extract the archive and configure the accumulo.properties file with your HDFS and ZooKeeper connection details.
-
Initialize Accumulo instance
Run the accumulo init command to create the necessary system tables and set the initial root user password. This step prepares Accumulo for first use.
-
Create a table and insert data
Use the Accumulo shell or client API to create a table. Insert key-value pairs with optional security labels, then verify the data by scanning the table.
-
Set up cell-level security
Define authorization labels for users and assign them to key-value pairs. Configure user permissions in the shell to restrict query results based on their authorizations.
Frequently Asked Questions
What is Apache Accumulo and what is it used for?
Apache Accumulo is a sorted, distributed key/value store built on Hadoop HDFS and ZooKeeper. It is designed for large-scale data management across clusters, particularly in government, finance, and healthcare, where cell-level security and fine-grained access control are critical requirements.
How does Accumulo's cell-level security work?
Accumulo assigns a security label to every key/value pair. When a user queries data, the system filters results based on the user's authorizations, so only cells with matching labels are returned. This provides granular access control at the individual cell level, unlike most NoSQL databases.
What are server-side iterators in Accumulo?
Server-side iterators are a programming mechanism that modifies key/value pairs during data processing. They enable custom filtering, aggregation, and transformations directly on the server, without moving data to the client. This allows efficient, scalable data processing at the cluster level.
How does Accumulo compare to HBase and Cassandra?
Accumulo offers more advanced security than HBase through cell-level access controls. Compared to Cassandra, Accumulo provides strong consistency but slower write throughput. Cassandra is better for write-heavy, time-series data, while Accumulo suits security-sensitive applications requiring data integrity.
What are the main trade-offs of using Accumulo?
Accumulo requires a Hadoop cluster and ZooKeeper, increasing operational complexity. Write performance is slower than eventually consistent systems due to strong consistency and security checks. The learning curve is steep, and the community is smaller, resulting in fewer third-party tools and resources.
Is Accumulo free to use and what are the costs?
Yes, Accumulo is open-source under the Apache License 2.0 and free to use with no licensing fees. However, users must cover infrastructure costs for Hadoop, ZooKeeper, and cluster hardware, which can be significant depending on scale and operational requirements.
Alternatives
How Accumulo compares
Direct head-to-head against 3 competitors. Picked by 7wData.
Accumulo
- Pricing
- Accumulo is open-source and free to use under the Apache License 2.0. There are no licensing fees, but users must cover infrastructure costs for Hadoop, ZooKeeper, and cluster hardware.
- Target
- Apache Accumulo is a sorted, distributed key/value store that provides robust, scalable data storage and retrieval, designed for large-scale data management across clusters.
- Strength
- Accumulo provides cell-level security labels on every key/value pair, enabling fine-grained access control that is unmatched by most NoSQL databases.
- Watch for
- Accumulo requires a full Hadoop cluster and ZooKeeper to operate, increasing infrastructure complexity and operational overhead compared to standalone databases.
Apache HBase
- Pricing
- Open source, free
- Target
- Real-time read/write on large datasets
- Deployment
- On-premises, cloud
- Strength
- Automatic failover between RegionServers
- Watch for
- Complex setup and tuning required
Amazon DynamoDB
- Pricing
- Pay per request or provisioned capacity
- Target
- Serverless key-value and document workloads
- Deployment
- Fully managed cloud
- Strength
- Single-digit millisecond latency at any scale
- Watch for
- Vendor lock-in and cost unpredictability at scale
ScyllaDB
- Pricing
- Open source core; enterprise from $450/node/month
- Target
- High-throughput, low-latency NoSQL workloads
- Deployment
- On-premises, cloud, hybrid
- Strength
- C++ implementation delivers consistent sub-millisecond latency
- Watch for
- Operational complexity and smaller ecosystem than Cassandra
User reviews
No user reviews yet. Be the first to write one.
Sources
Reporting on this tool draws on these publicly available sources.