Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

Learn how developers deliver flags instantly to anywhere in the world

 is a developer-first feature management platform that focuses on the full lifecycle of feature development - from code to production, validation, testing, and iterating. They’ve partnered with Macrometa to build 

 - an ultra low-latency global feature flags solution to deploy software faster and reduce release complexity.

Feature flags are a powerful tool that act as a toggle to switch features on and off. They are used to control the release of new features and functionality - especially to activate different features/functionality and experiences for different users.

Feature flags give development teams the ability to enable or disable features/functionality remotely without making experimental or embargoed features universally visible to users.

Feature flags decouple deployment from release - so engineers can quickly do their best work while product managers can get features out to users in the best way possible.

What is the DevCycle Edge Flags solution?

fastest way to deliver feature flags using data stored

 at the edge- are a key differentiator of the DevCycle experience.

Data for feature targeting isn’t always where you need it at the time you need it. With Edge Flags, developers only have to send the information once and DevCycle delivers flags instantly to anywhere in the world.

Developers use Edge Flags to make near-instantaneous flagging decisions using simple User IDs with real-time user data. The data is globally replicated at the edge and this lets them pass data once and deliver the same experience everywhere - across all devices, anywhere in the world. So developers don't have to set up expensive data pipelines to sync data across multiple platforms, applications, or microservices.

With edge-localized data synchronization, developers ensure that feature flags are updated in real-time, providing a better experience for customers and making it easier to test and iterate on new features. Features are turned on or turned off in core services at the server level (or on the website and mobile app) and that decision is made and implemented immediately across the entire system.

Real-time feature flags provide an additional layer of control by allowing development teams to change the behavior of a feature on the fly, without having to rewrite or redeploy code. This can be useful for things like A/B testing, error handling, or instantly rolling back a feature that isn't performing as expected.

In terms of customer experience, Edge Flags can be used to serve up customized landing pages for different sets of customers. For example, if a retail store knows they have two sets of customers - one that prefers to walk into the store and another that prefers to shop online - they can set this preference as a flag that toggles different landing pages with custom walk-in or delivery options. These preferences can be baked into a simple property such as User ID that - combined with targeting rules - sets up personalized user experiences.

 from DevCycle Product Manager Victor Vucicevich.

 to support the near-zero latency that Edge Flags mandates. Macrometa’s flexible, ultra low-latency data layer purpose-built for global, real-time, and event-driven use cases helps them solve the problem of making data instantly available everywhere without passing it around anywhere. 

With Edge DB providing real-time user data, development teams can deliver flags in an instant - without complicated data pipelines or tradeoffs between storage and latency.

Curious what it takes to build a game-changing real-time feature flags solution? Watch Cobi Druxerman (Co-Founder & Head of Product at Taplytics and DevCycle) in conversation with Justin Johnson (Head of Product at Macrometa) as they discuss the practicalities of implementing Edge Flags while also elevating the developer experience.

Interested in digging deeper and seeing Edge Flags in action? Head to the Edge Flags on Macrometa 

How DevCycle Built The World's Fastest Feature-Flag-As-A-Service On Macrometa

Authors: Taric Mirza, Founder, and CEO @ Arrive, and Alan Evans, Principal Technologist @ Macrometa

, is a lifelong gamer who’s made it his business to redefine the Esports and multiplayer gaming experience. When he hit severe lag playing League of Legends, he tried everything - changing ISPs, home routers, just name it - but nothing fixed it. Eventually, he decided to come up with his own solution to defeat lag for LoL and other games. His current favorite games are League of Legends and Apex Legends.

Today, Taric is in conversation with Alan Evans, Principal Technologist at Macrometa where he explains how Arrive conquers lag by finding the fastest network route. He also talks about how Macrometa’s 

 allows Arrive to manage its global infrastructure very easily and with very 

. Can you tell us a little bit about your company - Arrive - and your motivation for building it?

 Arrive is a startup company that is developing new products for Esports players to improve their experience playing games. The motivation behind starting Arrive was I hit severe lag playing League of Legends, and nothing I tried (changing ISPs, home routers, you name it) fixed it. Eventually, I switched from working on multiplayer network programming in my own small games to instead using some of those same techniques to come up with solutions for lag for other games. 

Our core product is an application for League of Legends players to improve their internet. We are working on supporting more games. I’m really excited about Arrive, as lag is a problem that impacts virtually every esports player at one point or another, and solving this would benefit players individually, as well as support the gaming ecosystem.

 - I was just thinking, though, some folks reading this might be asking what is “Esports? and how it is different from “normal” multiplayer gaming, and maybe describe what “Lag” is.

 - Sure! The main difference is Esports is more competitive in the sense you get an official Rank, much like chess players who are competitive get an “ELO” score that shows them numerically where they stand in relation to other players. For Esports, while having casual fun is of course still the main reason people play, there is a real sense of “serious competition” (that in turn, makes it even more fun!). At the highest levels, players have dedicated gaming teams they are on that receive significant corporate funding. Much like professional sports teams, at this level, players can get traded to other teams. The prize pool for some tournaments can exceed $1 million, and some of the top players earn that much or more from their teams as well. 

 - Ok, so Esports is more competitive, got it. Plus, the gamers get sponsorship deals, I saw, so there is lots of advertising involved as well, I guess?

 - Esports is part of the competitive video game online arena where players play the game competitively. These are games where players play against other players in each game and get assigned a rank based on how well they perform and if they win. The most successful esports games have tournaments that are watched by millions of people worldwide.

 - love watching the professionals playing LoL; there is so much happening on-screen. You can really see why lag can be a real problem.

 - Yes, and while the official tournaments hosted in person are lag-free (as the computers are right next to each other and directly connected), many LoL tournaments are online, where players sometimes play hundreds of miles away from each other. Lag is a huge problem in these cases, and Arrive would like to play a role in working with some of these tournament operations to help fix these issues.

Why is Traffic Acceleration critical for Esports and Multiplayer Game traffic?

 - Ah, I would have loved to have been an Esports player. When I first got into online multiplayer gaming, it was on the Sony PlayStation, and a game called SOCOM Navy Seals. Super competitive, and if you saw people lagging across the screen, it was a real nightmare as you just didn't know what was going to happen. Bad lag can really ruin multiplayer gaming, no matter how competitive the game is.

 - Yes, Internet lag is one of the top problems that video game players, particularly esports players, run into. Multiplayer Game traffic Acceleration, which Arrive is building with help from Macrometa on the 

, is able to reduce or eliminate lag for most gamers.

 - Taric, do you think you could describe or maybe explain what the Arrive solution does?

 - The solution includes client software that the player installs on their gaming computer that routes the game traffic onto the Arrive network. It is simple to operate as the gamer – you basically just turn it on and off as needed, but behind the scenes, the solution is scouting and utilizing the best internet routes available on the Arrive network to enhance the gaming experience.

 - Ah, since you mentioned Macrometa, why are you using the Global Data Network (GDN)?

 - To power Arrive’s product that accelerates network traffic, Arrive manages a fleet of servers in a variety of data centers spread across the world. Macrometa’s GDN allows Arrive to manage this global infrastructure very easily and with very low latency.

 - Nice. Yes, our Global Data Network is available in almost 200 locations around the world, providing a P90 sub-50ms RTT to any connected device globally. 

How do you leverage Macrometa GDN to Conquer Lag and Accelerate Traffic?

 - Do you think you could go into more detail on how Macrometa is being used under the hood for Arrive?

 - Sure, we’ve found Macrometa to be a game-changing (pardon the pun) :) platform that has allowed us to develop the end-to-end Arrive solution much more efficiently than would have been possible without Macrometa. 

Macrometa is our primary - and only database - and PaaS. It handles 100% of our persistent (and mostly real-time) data needs which is pretty incredible for a single technology. We use Macrometa GDN for many areas but the basic approach is that we use Macrometa to contain both our primary event and transaction data, in terms of being a “single source of truth” for data, but also operational data such as aggregations, analysis, and reporting.

In addition, we use Macrometa for our User Management system, handling everything from registration, authentication, and logins to what features the users have enabled.

Macrometa also powers our server management and resource allocation technology. This is the system that figures out which of our servers or network paths have sufficient capacity to serve more players who want optimization. There is quite a bit of complexity in keeping track of all this, and Macrometa makes it easy.

 data too. This is just another aspect of how Macrometa shines – in the old days, you’d have to research and integrate with a separate caching system requiring its own command language/API, integration between the databases, and pricing. With Macrometa, we get an easy-to-use, cost-effective, high-performance Cache system built right in. 

And finally, we use Macrometa for our internal reporting systems, too. Things like aggregating data and generating reports with it are easy with Macrometa. If we want to know how many players used Arrive today, Macrometa by itself answers that question. So yeah - we use Macrometa across the board as the platform that enables us to focus on building the world’s gamer service for multiplayer gaming - it lets us focus on what we do best - making games awesome for gamers by eliminating lag, improving jitter and latency by leveraging the edge in ways that are impossible to do in the cloud.

 - Wow, impressive, that’s quite a bit! It’s interesting to see how much you are using 

. I’m particularly curious about what made you choose Macrometa to be your primary platform as opposed to one system as a part of many others.

 - The biggest gain of leveraging Macrometa handling everything as opposed to a patchwork of various systems is it drastically reduces complexity. That, in turn, leads to many significant benefits:

First of all, we don’t need to pay for 3rd party ETL or spend a considerable amount of time developing it ourselves. This is all done automatically by Macrometa for us. This saves us quite a bit in both financial and labor costs.

Secondly, a few systems make our application much easier to operate. Things like configuration, software releases, testing, and general day-to-day development are all more straightforward. Even onboarding new developers is much faster when they can learn just a single system instead of 5-20 that all have to fit together.

And last but not the least, we also benefit from reduced risk and tighter security. Generally speaking, the more complex an architecture, the greater risk there is for an outage or security incident. It is more challenging to secure and operate a whole patchwork of various databases, ETL, and reporting systems, all with various synchronization processes, authentication keys, etc., than a single system.

 - Excellent! So, could you talk about what challenges Macrometa has helped Arrive overcome?

 - One of our greatest challenges is building a system that can handle users globally, with high performance, and that can scale very well as we continue to grow.

One of the great things about Macrometa is you can come up to speed incredibly fast, which is critical for us as a startup company, as getting a great product out as soon as possible is essential.

At the same time, we wanted to find a technology that would handle users around the world and scale. Macrometa handles all this for us seamlessly. With other technologies, this generally takes very expensive projects consisting of top consultants to help get all the geographic distribution, redundancy, security, and scaling aspects. What previously might take 6-12 months of work, best case, you now get on day 1 with Macrometa right out of the gate!

 - Thanks Taric. So, where do you see this going? I mean, what do you think the next steps are for Arrive?



 - For Arrive, on the product development side, we want to support many more popular Esports games. We want to expand to support players in more cities and countries around the world. We plan to improve the algorithms we use for internet traffic routing to improve gamers’ experience further. On the distribution side of things, we will partner with enterprises like ISPs to help get our product out to more players.

 - Can Macrometa help in any of these areas?

 - Macrometa can help with all these areas, actually!

For game expansion, Macrometa will play a critical role in this, as many of the other games we want to support require considerable data processing. Macrometa is ideal for this. We will evaluate using Macrometa’s Streaming technologies for some aspects.

Same story with the routing algorithms. Our next iteration uses more involved logic, and Macrometa’s data processing capabilities will make this part much easier. The 

 might be particularly powerful in enabling some of the features we want to add.

Macrometa will help not just on the tech and data side but also on the business side in terms of helping develop and bring a product to ISPs using the Macrometa GDN PoPs inside the ISP networks.

 - Hey Taric, this has been great. Have really enjoyed our chat, as usual, and I'm sure our readers have as well. Have you any closing remarks or ideas you would like to share? and where can our readers find out more about Arrive?

I’d first want to say I’ve really enjoyed our discussions and would extend that to some others at Macrometa. When I first reached out to Macrometa with a list (some might say a book!) of questions on how the technology works, Justin Johnson was amazing at answering everything. I also had the pleasure of meeting Macrometa’s CEO, Chetan, in person in Dallas earlier this year. Not only was it a productive conversation, but as a human being, Chetan is amazing and inspiring.

To learn more about Arrive, please check out our website, 

, and I’m always happy to answer any questions (or play LoL or Apex)!

We’re excited about the future, not only fixing lag for video game players as we’re doing today but expanding into providing valuable tools for players in other aspects of their gaming experience and helping support the greater Esports ecosystem, and extending our network optimization technology into all sorts of other areas where real-time performance is key.

If any of our readers would like to learn more about Macrometa GDN, request a 30 day trial. 

How Arrive Conquers Lag And Accelerates Esports Traffic With Macrometa Global Data Network

This is a guest post from Abhay Bhargav, the Founder and Chief Research Officer of 

“Attackers think in Graphs, but Security thinks in Lists” is a famous saying amongst security practitioners like myself. And this is very true.

 - who popularized this saying - pointed out that defenders or security teams think in terms of asset lists while attackers think of the relationships between different assets. They breach a system using one (or more) of the assets and find multiple paths to the highest value asset by navigating the “graph” of interlinked vulnerable assets. 

On the modern web, attackers look to identify and exploit vulnerabilities. Once they compromise an application(s), they pivot into internal applications and connected cloud services through chained vulnerabilities and exploits. These attack sequences lead to the eventual outcome that the attackers desire-, which could be anything from stealing sensitive information to deleting mission-critical datasets and disrupting operations.

Attackers honestly think in graphs. Their ability to form relationships between connected assets gets them the ultimate prize.

Security teams need to pivot from list-thinking to graph-thinking to defend their systems successfully. This means they must visualize the entire network as a series of security dependencies and implement controls to reduce the attack surface.

That’s precisely what we set out to do at 

We’re a leading EdTech Platform for Application Security. We provide labs, cyber ranges, and lessons on security-focused areas like Application Security, Cloud Security, Kubernetes Security, and more. As a result, thousands of students, from job seekers to experienced professionals, use our platform daily to upskill themselves and improve their job prospects.

Being a security training platform, we can’t help but take security seriously. One significant aspect of “taking security seriously” is to capture and enrich logs and possible security events. The idea is to capture logs and events we ingest and further transform them into data we can query and investigate as the need arises.

Our platform is serverless from the get-go as we were “born in the cloud”. We use 

 for object storage. Our application and user base have grown exponentially over the past year that we’ve been in business. 

 for log streaming, alerts, etc. However, we quickly realized this could have been more user-friendly and scalable. Cloudwatch is an excellent logging service, but it also has some significant downsides for our use case: 

Its native query interface is challenging to work with, especially for several thousand functions with multiple API Gateway instances you need to query. We found this frustrating, to say the least.

Cloudwatch comes with Log Limits. And that is fine. But, we wanted to retain logs for an indefinite period. This proved to be expensive with Cloudwatch.

Dashboarding and Alerts are not the easiest with Cloudwatch. It becomes a significant bottleneck and management overhead to manage all of this.

Looking at the downsides of Cloudwatch, we decided to move to 

. As the Chief Technologist, I was already a fan of Macrometa because of its ability to ingest multiple data types, from document to key: value to graph, etc.

In addition, since Macrometa is serverless, it makes management overhead “not our problem,” - which we use as a basis to evaluate every solution we use. We have a small team, so “managing” a lot of resources is a NO-GO. In addition, I was interested in leveraging Macrometa’s 

 capabilities to query events based on relationships. 

For example - If I want to query all the related security logs on API Gateway, Lambda Function, and User Security Activity for a given service at a given time, creating a graph makes the most sense. With this, I can get a complete set of data points for all these complex relationships without spending much time writing and optimizing SQL queries.

So we set to implementing this functionality with Macrometa.

We started to stream logs from Cloudwatch to Macrometa, thereby ensuring that we didn't have to rely on Cloudwatch at all.

We stored all the logs in Macrometa as documents (structured logs). Then, using Lambdas, we added valuable transformations in the form of additional context to the structured logs with service information and graph relationships for the dataset.

This is super-easy with Macrometa because all we have to do is just create a 

 with specific parameters that you pass. This is an API that we invoke with the suitable parameters, and the data is ingested into Macrometa. 

And tada! We were able to get up and operate very quickly. We were able to capture events quickly with zero human intervention and most importantly, with the graphing capability, we are now able to model complex relationships and identify security anomalies and events very soon on our stack.

The result? The much-needed observability into our stack allows us to out-think attackers without ever having to “manage” things.

Like how AppSecEngineer implemented Macrometa to overcome its logging challenges with CloudWatch, you can try out Macrometa to expand its query workers and graph capabilities. Get started by requesting a 

Out-Thinking Attackers With Graphs: Why AppSecEngineer Chose Macrometa Over Cloudwatch

The time when web was very slow and broke easily

In 1998, the web’s largest websites were straining to meet the demand of a new class of power user customers who had upgraded from dial-up internet access and were now using DSL and ISDN to connect to the internet. No more content to surf the internet (what a dumb term - 

Jean Polly I will never forgive you for that

), internet users were downloading audio and video and all kinds of big objects in a frenzied apoplectic feeding on multimedia.

They were using web-based email like Hotmail (and my personal favorite Rocketmail) and chat messaging services to instant message or IM each other. Powering this explosive growth of the cloud was a unique new infrastructure invented by Tom Leighton and Danny Lewin at a startup called 

 in Boston. And the brilliance of the algorithms and technology that they invented ended up becoming one of the most important critical infrastructures that enabled the web to become a part of our daily lives. 

In spite of the fact that the data these websites served was thousands of miles away in primitive and precambrian clouds run by companies like Exodus Communications, the 

 enabled the web to scale to be used by billions of people and made all these websites run quickly and instantly. Had it not been for Tom and Danny, perhaps the web as we know it today would not exist. That’s how important the technology they invented and the company Akamai that commercialized it is. 

Today’s cloud is also slow and breaks easily

Fast forward a quarter century to today (November 9th, 2022), the cloud is having a similar struggle. The challenge to scale and deliver data from cloud-native apps and APIs to billions of people using apps on smartphones and tablets, interacting with content on smart TVs, smart refrigerators, smart home appliances, still prevails. And not just that, folks are commuting in connected cars on rideshare services, getting food and groceries delivered by robots and drones, and transporting themselves on electric scooters and bikes.

The cloud desperately needs it’s Akamai-like breakthrough moment when a new technical infrastructure and paradigm enables the cloud to scale to meet the needs of serving data and apps to billions of humans, using tens of billions of devices nearly instantly, anywhere in the world. That infrastructure and paradigm are the 

The Global Data Network or GDN is a new kind of distributed computing infrastructure that solves the hard problems of distributing data and computing geographically to 1000s of locations around the world. A 

 can connect data from anywhere, on any system, in any format, and turn it into APIs. This can further feed edge compute and in-region processing and serving of apps, from the CDN and wireless networks to enable a seamless, private, and secure computing in compliance with sovereign data regulatory frameworks via a suite of 

Macrometa’s GDN is the edge computing platform that is designed to help developers build new innovative digital services that take advantage of time-sensitive events, spread around global geography, yet with hyper proximity to where data is being produced or needs to be consumed - all with the power and simplicity of a serverless API or SDK.

How Macrometa and Akamai are expanding the edge with a supercloud

Today, we are incredibly excited and pleased to announce that the original OG of edge - 

 and the new upstart disruptor of Edge - 

, are both partnering to create an Edge Cloud Super Platform that amalgamates the deep capillary network of Akamai’s edge (4200 locations) and its industry-leading network backbone, with 

Macrometa’s Global Data Network Platform

 for developers to build globally-distributed apps and APIs. 

By combining the power of Akamai's edge, Linode’s cloud, and Macrometa’s GDN via a set of product and technology integrations, enterprise developers can now seamlessly build apps across cloud and edge with a new level of simplicity, and speed. Where the old cloud was slower and breaks easily, the supercloud can power real-time apps anywhere. And a supercloud deserves a super platform for building those apps.

Macrometa has been a customer and partner of Linode for close to four years now. We adopted Linode early because we were impressed with the performance of Linode’s infrastructure, the very reasonable transfer costs of their dedicated network, and their impossibly amazing developer support.

Our use of Linode has grown over the years to the point where we started collaborating on bespoke customizations that only we may need among all of Linode’s customers due to our unique needs as a Global Data Network. With Akamai’s acquisition of Linode earlier this year, we were delighted to hear about Akamai’s plans to expand Linode from its current less than dozen worldwide regions to potentially fifty regions worldwide and more importantly their deep integration with Akamai’s backbone network and edge platforms.

This now means that our customers can leverage Macrometa to build apps that will run better than cloud economics in many more regions around the world on our 

. This also means that developers can leverage Macrometa on Linode for deep compute power to run CPU and memory-intensive workloads that may not fit a javascript service worker model (like Akamai’s edge workers or Cloudflare workers). We can build what we like to call poly cloud apps - that straddle not just different cloud providers like AWS and Google but also span from cloud to the edge across Linode and Akamai.

To enable customers to adopt the edge for enterprise apps, Akamai has announced the 

, that provides $120K annual usage credits for year 1, and discounts on usage of 75%, 50%, 25% in succeeding years. Customers can take advantage of Macrometa via the RISE program to build and deploy apps on the edge. Macrometa themselves went through the RISE program a few years back and has first hand experience of the power of the program.

Please watch the following video where Macrometa CEO & Co-founder Chetan Venkatesh, sits down with Akamai’s director for business development 

 to discuss how Akamai is scaling Linode to meet enterprise needs and the reasons for creating and then expanding the RISE program.

The Integration of Akamai Edge Workers and Macrometa Global Data Network

We are making a powerful and seamless integration between Macrometa Global Data Network via console available as a preview (with a general release planned for Jan 2023 where we will also support a common API and SDK) of our direct integration between Akamai edge workers.

Macrometa enables developers to quickly and easily build APIs on data at rest via 

. These query workers and stream workers function as a service that runs inside our poly model database with data locality and low latency but more importantly are exposed externally as a RESTful 

With our new integration with Akamai edge workers, we can automatically compile the query worker or stream worker into a native Akamai Edge Worker and automatically deploy it on to Akamai’s edge. All that the developer needs to do is to provide their Akamai credentials and with the click of a button, the API gets generated as an Akamai Edge Worker and automatically deployed to 1000s of Akamai edge PoPs that query and serve data from adjacent Macrometa regions on Linode. You can even provision, de-provision, debug and log your edge workers directly from Macrometa (debug and logging will be in the general release and are not available in the preview).

Demonstrating the power of the edge to enterprise developers

To showcase the power of our integration, Macrometa and Akamai have jointly developed a full video streaming app (like a Netflix or a Hulu) that runs on the Akamai edge and Linode leveraging Macrometa, Linode and Akamai Edge Workers. We chose this particular example to show how a complex cloud based backend composed of many microservices could be moved to the edge to power a range of APIs and web services to power the front end experience of a video streaming app or Over The Top (OTT) video services. Netflix engineers have written extensively about their 

complex and arcane microservices architecture on AWS

Our sample application suggests that much of that complexity and cost can be reduced and performance improved by 25X to 50X per web request from client to backend and back by leveraging Macrometa, Linode and Akamai.

Macrometaflix OTT/Streaming Video Application

Closing thoughts: The Edge is your oyster

Edge Computing for the enterprise is finally here and now - with Macrometa and Akamai. The Global Data Network or GDN does to the Cloud what CDNs have done to scale the web. We haven't just built a new cloud - we've built a supercloud for next-gen apps.

Enterprise developers can now write applications powered by real-time data that run unfettered by the constraints of a centralized cloud deployment model, across a world wide network of data centers and edge PoPs, delivering powerful new digital experiences, instantly, securely and cost effectively at a scale that is 1000x the cloud.

The future of the cloud is edge, and the edge is already here on Macrometa and Akamai. Come build on the cutting edge with Macrometa and friends. Excited to try out Macrometa? Request a 

Expanding The Edge With Macrometa And Akamai

Macrometa is pleased to announce that a number of exciting new features are now available with the release of the 

, which enables developers to easily create full text search indexes, are now Generally Available (GA) for all Macrometa developers. The Macrometa integration with 

, a powerful compute engine for creating and deploying microservices at the edge, is now available in public preview. Available in public beta are 

, Apache Kafka wire compatibility, and Redis API compatibility.

 and streaming analytics engine, is now Generally Available (GA) to all Macrometa developers. With 

, Macrometa users can build sophisticated Complex Event Processing applications. Stream Workers includes high level primitives for pattern matching, flow control, joining multiple streams and data from operational data stores, and other CEP capabilities. To learn more about Macrometa Stream Workers and to see this in action, check out this blog and demo from the Macrometa Product team.

 makes powerful full text search as easy as writing a SQL query. Macrometa Search supports integrated search indexes on any type of data collection and across multiple collections in the 

. Macrometa search includes not only standard text search features like similarity scoring and a variety of tokenization options but powerful advanced capabilities like graph search and geospatial queries. To learn more about Macrometa Search, be sure to check out our 

Macrometa and Akamai are announcing Macrometa Query Workers and Stream Workers integration with Akamai EdgeWorkers. Akamai EdgeWorkers is a powerful compute engine for creating and deploying microservices at the edge. The combination of Akamai’s compute engine with the Macrometa GDN allows our customers to develop microservices with powerful data storage and streaming capabilities, all running milliseconds from your users. To learn more about Macrometa’s Akamai EdgeWorkers integration, check out our 

Postgres-compatible SQL is now available in public beta and joins our existing Macrometa query language. While the Macrometa language has always been largely SQL-compatible, having Postgres language support will allow our customers to port and in many cases “forklift” existing code bases directly into the Macrometa GDN without needing to rewrite their queries. To learn more about Macrometa SQL support, check out our 

 is a popular data streaming engine. Macrometa users have been able to read from and write to Kafka topics from our Stream Workers engine and we now support writing directly to Macrometa data stores with beta support for Kafka wire compatibility. Macrometa’s Global Data Mesh makes no distinction between data at rest and streaming data and all data stores are available to connect to as streams using the Macrometa API. Now, customers can connect to collections as if they were Kafka topics without having to learn a new API. Kafka consumers receive a change data capture event stream from the data store that includes both new data streamed to the collection and inserts, updates, and deletes made through data at rest CRUD operations.

Redis is a popular NoSQL database for use cases including caching, session management, real-time analytics, and more. Macrometa now includes beta support for Redis compatible collections in the Global Data Mesh, bringing your data within 50ms of all of your users, no matter where they are. Redis developers can get up to speed with Macrometa’s new Redis collection type very quickly. Porting or extending existing Redis applications to the Macrometa GDN is as easy as swapping out a library.

Excited to learn about the entire release? Follow the video below and 

Announcing Macrometa Latest Release - Global Data Network 17.11

We’re excited to announce the general availability of 

Macrometa’s complex event processing (CEP)

Complex event processing is a tricky topic, made simple by 

 giving an overview of CEP and the distinction between CEP and streaming analytics. 

CEP allows you to work with one or more data streams, optionally combined with data-at-rest, to infer “complex events”. For example, you might have streaming financial trade data, and by joining that with other data about macroeconomic conditions, you could infer a “market correction” event.

Stream Workers provides typical streaming analytics capabilities, including joining streams, aggregations, counts, and filtering as well as sophisticated high-level primitives for 

 such as pattern recognition, conditional logic, and flow control. The Stream Workers engine runs within the Global Data Mesh, allowing you to incorporate operational data-at-rest with streams of data from both Macrometa’s streaming collections and other event streams, such as from 

 topics. Stream Workers can write to data stores within the Global Data Mesh or stream results to other streaming frameworks like 

Stream Worker's applications can be queried directly using a REST API

. Since Stream Workers is tightly integrated with the 

, there is no need to transfer data from your operational data store to the CEP engine. And they are, ultimately, making large amounts of reference data available instantly for sophisticated predictive modeling at very 

This may sound very complicated and, until now, it has been, but with Macrometa Stream Workers, all developers need is basic SQL and Javascript skills, and the Stream Workers engine handles the rest. In addition, your application can run simultaneously at over 175 locations worldwide and synchronize automatically at the speed of light. As a result, companies and developers using the Macrometa Global Data Network have a real competitive advantage with access to a powerful distributed 

 engine without the developer and administration complexity you’d expect.

Stream Workers can write self-contained microservices and traditional streaming analytics applications. Using Stream Workers, you can create 

, create microservices, and serve the logic of an application backend. To demonstrate this, we’ll show our cryptocurrency trading demo below. Stream Workers can solve so many problems, but here are some simple use cases:

Transform data from one format to another, for example, from XML to JSON, and clean and filter your data:


) data from log files and scrape HTTP endpoints.

Obfuscate sensitive information by filtering it from messages, logs, or other data streams that may include Personally Identifiable Information (PII) or Personal Health Information (PHI).

Correlate data by joining multiple streams to create an aggregate stream.

Derive insights by identifying event patterns in data streams:


Summarize data with time windows and incremental aggregations

Trigger actions based on streaming data, either as a single service request or a complex enterprise integration flow.

Build and refine predictive models for machine learning

Make real-time offers to customers who are visiting an eCommerce site based on browsing or shopping history

Threat detection in cybersecurity applications

Stream Workers Demo: High-Frequency Crypto Trading App

 to trade Bitcoin locally in US, Germany, and Japan exchanges.

An exchange arbitrage is a trading strategy based on the differences between the price of the cryptocurrency at different exchanges. Generally, these opportunities open up if there is a price discrepancy, and the contrast accumulates over time to finally become a significant amount, generally 1–3%. However, these arbitrage opportunities typically only exist for seconds, so an arbitrage trader must constantly search for the best opportunities and implement them immediately.

Implementing arbitrage strategies in the crypto market takes work and involves several technical complexities. Besides having deposits in multiple major exchanges, on the technical side, it consists of the ability to monitor arbitrage opportunities simultaneously, making trades locally at each exchange with minimum delay and timely visibility and interactions between the trading agents distributed globally. The challenge requires big dollars and, until now, a team with the technical expertise to build a distributed infrastructure.

To build the demo, we used the following:

Macrometa Global Data Mesh collections, both streaming and data-at-rest, and 

We use a document store to store trade data and share all trade-related metadata and state between the decentralized, stateless bots and components.

We use geo-distributed streams for publishing real-time price data from each exchange. Each bot subscribes to the stream and decides on whether to buy or sell.

Lastly, use real-time updates from the datastore to notify the clients/bots/subscribers of changes to the database so they can modify their trading strategy.

The entire backend of this application is written in SQL and Javascript with only 174 lines of code! Watch the video for a complete app walkthrough, or check the code on 

Stream Workers logic is written in standard Postgres SQL and Javascript. Our web console has a full-featured REST API, code editor, and publisher. You can query any data store directly in your code without any configuration.

We provide native sources and sinks to connect to Kafka, Google Pub/Sub, MQTT, and S3.

Archive

How DevCycle Built The World's Fastest Feature-Flag-As-A-Service On Macrometa

How Arrive Conquers Lag And Accelerates Esports Traffic With Macrometa Global Data Network

Out-Thinking Attackers With Graphs: Why AppSecEngineer Chose Macrometa Over Cloudwatch

Expanding The Edge With Macrometa And Akamai

Announcing Macrometa Latest Release - Global Data Network 17.11

Announcing Stream Workers: Complex Event Processing And Streaming Analytics At The Edge

The Macrometa Platform Now Supports The SQL Language

Announcing Macrometa Search: Powerful Full-Text Search, As Easy As Writing A SQL Query

Macrometa’s New UI And Design Process - Unravelling An Aesthetic Pleasure