Say Yes To The Data Mesh: Explore Multi-Model Data Collections And Stores At The Edge
Oct 19, 2022
Data is the “new oil” and organizations that are able to quickly analyze both real-time and historical data will have a competitive advantage. But ingesting, managing, storing, and correlating different types of data can become a costly, administrative nightmare.
In this blog, we will walk through details about Macrometa’s Global Data Mesh and its multi-model data collections and stores, search, and streams components. We will also go over the two other parts of the Macrometa Global Data Network (GDN) - Edge Compute that has features like complex event processing and Data Protection with features like geo-pinning.
Global Data Mesh: Store and serve any kind of data
With the Macrometa Global Data Mesh, you can use the data model that makes the most sense for your application and make it available locally, globally, or both. The result is a flexible, ultra-low-latency data layer purpose-built for global, real-time, and event-driven use cases.
One key advantage of the multi-model approach is that you don’t have to worry about time-consuming data integrations or storage maintenance. You simply use different databases at the same endpoint - so it is much easier to integrate different data streams into an application.
In a data mesh model, data can be stored, accessed, and processed where it originates and/or where it is needed. Organizations can use a data mesh to collect and process data where its users are and gather real-time insights.
Data is automatically replicated across the Macrometa GDN, and API requests are routed to the nearest available location. There are over 175 points of presence around the world. Changes at any location propagate across the network instantly, bringing all nodes to parity within milliseconds. Let’s run through the components of the Global Data Mesh.
Document Store - NoSQL document collections with flexible indexing and modeling
Macrometa DocumentDB makes it easy to read, write, replicate, query, and index JSON data on the Macrometa Global Data Network. Advanced features let you change data feeds for publishing or subscribing to document collections as streams.
Graphs - Dynamic, high-throughput graph store with ultra low-latency queries
Macrometa Graphs are a fault tolerant, fully managed graph database service that makes it easy to build and run graph powered apps and APIs on highly connected datasets. You can easily ingest millions of related events across tens of thousands of streams with billions of connected relationships in real-time and query the dynamic graph structure with milliseconds latency.
Streams - Distributed, real-time stream engine with pub/sub and message queues
Macrometa Streams integrate seamlessly with your existing workflows, giving you fast, scalable, and reliable access to your data from a wide range of popular frameworks. Build stateful low-latency micro-services and real-time data pipelines. You can create and use a stream on any collection or store.
Dynamo Store - Feature-complete and fully compatible with AWS DynamoDB
Macrometa Dynamo Store offers a low latency, globally replicated key-value and document database that is serverless, multi-cloud, multi-region, and multi-master. Macrometa Dynamo mode offers all the features of AWS DynamoDB with a 100% compatible API and wire protocol proxy (you can even use the AWS SDK with Macrometa Dynamo mode).
Key-Value Store - High-throughput KV collections
The Macrometa KV Store lets you build high-performance web and mobile apps and APIs quickly and easily without the need for managing anything. Use our simple API to quickly add state to your AWS Lambda functions, Cloudflare workers, or container apps.
Search - Flexible full-text search and similarity ranking across multiple data stores
Search has become such a critical part of application development and is key to improving customer experiences. Macrometa Search offers powerful full-text search and real-time ranked search with query parameters like spanning phrases, wild cards, images, proximity, and similarity. Full-text search is available in 20+ languages with dedicated analyzers and advanced features like stop word removal and stemming.
Bringing it together with Edge Compute and Data Protection
What separates Macrometa from other data storage solutions is that it also brings compute closer to people and where data is generated, regardless of where that is. Macrometa’s Edge Compute puts data into action and delivers real-time and event-driven apps and data in <10ms to 80% of the global population. Query Workers lets developers create simple REST APIs on top of the data mesh. Stream Workers enables Complex Event Processing (CEP) functions and Stream Processing workloads - in minutes instead of days or months.
Stream Workers play an important role with data collections and stores. With Stream Workers, data could come from different sources (GDN streams, GDN Database, HTTP endpoints, Kafka, and MQTT etc.) and can be published to different sinks or stores. Stream Workers also allows you to transform data from one data type to another, aggregate streams, and clean and filter data. This CEP engine automatically creates objects like streams, collections, and even tables and remote databases.
Macrometa’s Data Protection supports geo-pinning or geo-fencing to address data sovereignty and privacy use cases. Developers can set up localized data fabrics for regional and global data, and then adjust locations as needed. Macrometa offers three different types of authentication (user, token-based, and API keys) and is SOC 2 certified.
Try it out today
With many tutorials, Quickstarts, and easy-to-follow example documentation, you can get started with Macrometa’s Global Data Mesh in minutes. Request a 30 day trial, or schedule a demo with one of our experts.
