Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

Build Geo Distributed Apps with Macrometa

Here's a few examples of how Macrometa turns complex 

Build a multi region database that runs in 6 different data-centers across the world in about 40 seconds (the video is not trimmed in any way):

Create a data collection in the global database (like a table in an RDBMS), insert some data in it, and see it replicate with consistency guarantees in all the regions your fabric runs in:

Create a query to retrieve data from the collection, save and export the query as an API end point that you can call from anywhere in the world. The request will always be served from the closest datacenter to the requester

Call your restQL API end point from a web application such as a Single Page Application (SPA) or Progressive Web App (PWA): using the showContacts example from above:

Realtime data is easy with Macrometa - all changes to the database are automatically pushed to all clients via Streams (no polling ever):

And thats just the beginning. There are so many incredible ways to use Macrometa to create fast, powerful, data driven apps and APIs that run worldwide automatically such as graph processing, stream processing and serverless functions (coming soon using JavaScript WASM and V8 isolates). BTW, everything you see above can be done via our 

There's a brave new world of distributed apps and APIs waiting to be built by you for 

. It's time to say Goodbye Cloud, and Hello Edge with Macrometa.

 and get started in minutes. You can get all the 

 and we're a chat box away (to the bottom right of the macrometa web page)

Create Global Apps & APIs In 5 Mins With Macrometa's Seamless Architecture

Today, June 12th, 2019 marks a very special milestone for all of us at Macrometa. Starting today, anyone can 

 for a free developer account and build geo distributed apps and APIs that run across our network of 25 global cloud data centers.

Macrometa for building geo-distributed apps

While the idea behind Macrometa had been knocking around in our head for a little over 5 years, turning all those napkin sketches, sticky notes, midnight conversations and 

 took more than 2 years and the combined efforts of many talented and passionate engineers on our team in Palo Alto California, Paris France, Sofia Bulgaria and Pune India.

One of our most cherished ideas at Macrometa is that anyone should be able to build a globally geo distributed application or API without knowing any more than they already know about using databases with languages like JavaScript and Python. The era of needing large cabals of Phd's with years of expertise in building distributed systems to get correct, accurate distributed systems behavior with high performance needs to end.

We set out with a goal to democratize the platforms that until now only companies like Google, Facebook and Amazon have been able to build. With Macrometa - anyone can build apps and APIs that run distributed, cross region and multi cloud and they don't need to know anything about concurrency, consistency or mastering the arcane complexities of distributed systems.

PS - like our Octopus mascot? Her name is Calico and if you want stickers of her for your laptop, drop us a note at justin@macrometa.com

Build Geo-Distributed Apps - Hello Globally Distributed World!

Contributed by EDGE GRAVITY from their new blog series: 

. This collection of blogs will focus on new technologies and industry leaders in the 

 as way to educate a broad IT audience in the future capabilities at the edge.

Macrometa Global Edge Fabric is an Edge Native Platform as a Service (PaaS) for building distributed applications and APIs for the web, mobile devices, 

, gaming and more. It delivers instant, low 

 stateful data for Single Page Apps (SPAs), Progressive Web Apps (PWAs), and more.

Chetan Venkatesh, CEO at Macrometa on how edge will generate an entire new class of services based on proximity to customers, location awareness, and the ability to co-ordinate data and services that don’t work in a centralized cloud. The edge requires a paradigm shift from the current architecture of databases as edge demands the processing of massive amounts of data in real-time to present the customer with instant responses.

Challenges of edge computing for application developers:

Real-time data analysis at significant volume

To see more thought leadership videos from the Global Edge Forum event visit the EDGE GRAVITY YouTube playlist: 

Reprinted with permission. Original source at 

Building Applications For The Edge With A Datacentric Model

Life happens in the HERE and in the NOW. The HERE and the NOW are defined by where we are and what interactions, events are going on around us and with us. These events and interactions occur in the real world and not some far off data center a thousand miles from us. Similarly every interaction we do typically involves “state” or data about the interaction that combines with other state to create “context” which is essential for us to create and understand meaning from all the interactions we have . But we store this state (data) 1000s of miles away in far remote locations - cloud data centers.

This is like interacting with someone who either doesn’t remember from one sentence to the next sentence (aka stateless) or someone who has to travel 1000s of miles to respond to us each time (aka centralized state stores). If we put it this way, it certainly sounds silly, doesn’t it?

But this is what is presented as a normal way of doing things at the edge with stateless apps and centralized databases. There has to be a better way!

 and streams that provide data at the edge, can scale across tens and hundreds of locations worldwide, and yet act in concert to provide a single coherent multi master database or stream.

It means we need to design systems that work on the internet with its unpredictable network topology, maintain causality over networks that are lossy, and avoid centralized forms of consensus - while still arriving at some shared version of truth in real time. Can this be done? This is the the question that piqued our interest and started a new journey at Macrometa.

There are many good distributed database (sql, nosql,..) and streaming systems out there. So why can’t these systems enable us to have state locally at the edge?

Note: There are some NoSQL/NewSQL databases that can span 3 or 4 or 5 regions, but are not exactly a true geo-distributed or edge native database.

 is something that allows reads and writes locally in all locations in parallel and does not require a user to magically know which data should be placed in which location or require that user to redesign the schema every time they want to add/remove a location.

Coming back, so why can’t the current generation of databases and streaming systems handle multiple locations?

One problem with most of the current distributed databases and streaming systems is that they rely on consensus protocols like Zab (zookeeper), or Raft or Paxos as their foundation. These are certainly beautiful protocols; but these protocols are meant for running inside data centers where the networks are ultra-reliable and low latency.

These protocols (and thereby the systems built on these) are not a good fit for WAN environments where the network is lossy, jittery and high latency. What we need database and streaming systems that are coordination-free and offline-first.

Let’s say you go to a nice restaurant with your partner and order a wine and dessert. Let’s say you ordered wine from the bartender and your partner ordered dessert from the waiter. How would you feel if the restaurant serves you nice folks only one i.e., wine or dessert but not both?

It certainly does not make sense. But that is how most current generation NoSQL databases work with their last-writer-wins approach. In other words, when two edge locations receive writes in parallel (i.e., not causally related), then only one change gets picked up and other change is lost.

What makes more sense is for both of your orders converged and the restaurant serves everyone both wine and dessert. In other words, what we need are databases that can intelligently converge changes in real-time at each location without coordination, but all come to the same definite version of truth.

If interested you can look at CRDT research for more info on this. CRDTs are objects that can be updated without expensive synchronization/consensus, and they are guaranteed to converge eventually if all concurrent updates are commutative and if all updates are executed by each replica eventually.

Similarly, there are some use cases like the double-spend problem where strong consistency is a must. But that does not mean that other use cases are penalized to support this subset of use cases. What we need are databases that can support various levels of data consistencies so that developers can make their choice depending on the use case.

Imagine you went to a coffee shop with your colleague. Your colleague orders one `Cafe Latte` Tall. You order something like below :

1/4 teaspoon vanilla extract - mix in after cycle is complete

You can imagine how the coffee barista looks at you. You might get a similar look from your colleague as well. The reality is both you and your colleague ordered the same thing i.e., `Cafe Latte` but what is different is the abstraction level of the order.

In recent years, there is significant progress on CRDTs and CRDT related technologies. But much of the research is about data types like Counters, OR-Sets etc. That probably makes sense from an academic perspective. But what developers need is `Cafe Latte`. Not the ingredients.

What is needed are higher level abstractions that developers can work with and quickly be productive - like databases with various consistency semantics, query languages etc.

When you choose a database today, you’re not choosing one piece of technology. You’re choosing three i.e., storage technology, data model, and API/query language.

For example, let’s say you choose Postgres. What you are choosing actually is the Postgres storage engine, a relational data model, and the SQL query language. Similarly if you choose MongoDB you are choosing the MongoDB distributed storage engine, a document data model, and the MongoDB API.

A fundamental problem with many of the current generation databases including the above mentioned ones is that they are below big monolithic systems with features interwoven between all of the layers. Monolithic databases continue to be useful pieces of technology, but they are evolutionary dead ends. They are the products of a smaller, less-connected era, and their shortcomings risk becoming liabilities for your business as it evolves.

Document databases, column-oriented, row-oriented, JSON, key-value, etc. all make sense in the right context, and often different parts of an application call for different choices. This creates a tough decision: Use a whole new database (and licenses, etc.) to support a new data model, or try to shoehorn data into your existing database.

What we need is layered model that decouples its various layers - data storage, data organization, data manipulation and data model.

This allows a single database system to support multiple data models against a single, integrated backend. Document, graph, relational, and key-value are examples of data models that may be supported by a multi-model database.

Support multiple data models against a single, integrated backend,

Employ a unified query language to retrieve correlated multi-model data such as Graph-Json-Key/Value etc in a single platform and

Support multi-model ACID 2.0 transactions.

Data privacy is a big issue and rightfully so. GDPR is one big step in that direction and I would imagine more regions and countries will follow this path in future. Database and Streaming systems (being holders of data) fall right into middle of this.

Most databases and streaming systems currently do not address this. The databases that attempt to address this typically use techniques like Geo-Partitioning (break the dataset into multiple subsets or partitions) and keep each subset in a different region.

While on the surface it looks right, this approach creates few problems. For example, it requires customers to know which subset of the dataset will be used in which region and then partition the dataset so that the partition falls into that region.

Similarly if a business is doing well and their customer base expands to other regions, this approach would require deep schema changes to support proper placement of data.

What is better? A database or streaming system that can geofence the data natively with strong isolation and does not require schema changes when a business expands to more regions.

The above are some of the fundamental issues with current database and streaming systems. We cannot address these by doing more of the same. We need to build systems with architectures that address these problems natively. Is it possible?

Certainly yes! We have built Global Date Network that addresses the above and other issues natively. It is currently running in 25+ locations worldwide with scheduled expansion beyond 100 by end of the year.

Clearly, more databases and streaming systems will emerge to address these fundamental problems. The important point is that the technology is available today, and provided by 

Why Global Data Network? Or Why We Built A Truly Coordination-Free Distributed Database For Edge Computing

“Do we need a next generation of game consoles?”

Guest post by Gaming Industry veteran Justin Keeling who has held senior leadership roles at Fox Interactive, Disney, and many other amazing industry leaders.

GameSpot recently polled their readers and asked them the following question:

With 5G on horizon and end-to-end fiber optic pipelines, how close are we to a future where gamers can just stream everything?

75% of readers chose: “Not remotely close”.

The skepticism is understandable. Since 2009, cloud gaming platforms like OnLive and Gaikai have been hyped as the disrupters of the traditional gaming console. I played the first OnLive demo at GDC 2009: Crysis running on a mid-range laptop. Most gaming desktops couldn’t run the game at the time and OnLive felt like black magic.

Outside of the demo environment - and in the real world- it couldn’t live up to the hype. Neither OnLive or Gaikai still operate, although some of their core networking technologies made it into current platforms like PlayStation Now.

But even a decade later nothing yet offers the invisible controller-to-cortex response that most games require.

Over the next 18 months, the game streaming experience is set for a resurgence. Microsoft’s xCloud and Google’s Project Yeti have fired the first shots by leveraging their parent companies’ end-to-end ecosystems to build platforms designed around streaming. Amazon and PlayStation are also cooking their own solutions. And all of the top five game publishers are betting heavily on a game streaming future.

What’s different today, is that an industry-wide critical mass of telcos, CDNs, hardware makers, tool providers, content creators – and crucially– a new generation of deep tech researchers are working in coalition to finally make game streaming a reality.

Edge computing is one of the jigsaw pieces needed to make this all work. Simply put; the Edge is a post-cloud network architecture that is hyper-local, decentralized, and much more efficient than the current, monolithic cloud infrastructure that is behind the current digital ecosystem. And the implications for gaming go far beyond just streaming latency.

I’ve been advising Macrometa for a little over a year to understand the implications of their offering to the 

, and I’m excited that Macrometa is attacking one of the most significant parts of the problem at the intersection of 

. This holds the promise of radically transforming the experience of both gamers and game developers. Macrometa has recently emerged from stealth with its planet scaling serverless edge cloud – the Global Edge Fabric. I had an opportunity to speak with Joshua Petty, Macrometa’s Vice President of Products about their potential impact on gaming. He quickly agreed that edge computing offers incredible potential to easily create this new breed of streamed games.

“The team realized early on that there were many benefits to relocating game processing, logic, and analytics closer to the end user. Burying the infrastructure in traditional cloud architectures and databases is costly and severely limits the ability to create truly global, immersive environments for all users. What was missing was a powerful data and messaging platform at the edge that allowed developers to quickly create applications and games using a pre-built and flexible backend for managing dynamic data and content. So we decided to build it as a simple service for developers to consume. Moving everything closer to the end user was the obvious approach to resolve issues of performance and scale.”

“We also realized that improving the experience of developers required more than just providing a global multi-model database or data platform. We went to work at making the developer experience simple and seamless, combining a flexible and globally scaled edge database with serverless functions, containers, Streams that unify pub-sub messaging, and the ability to expose your data models and content as GraphQL or RESTful APIs automatically. The result is a simple backend delivered as a service supporting Unity and Unreal engines, along with java, Python, and other common web languages and runtimes.”

This type of approach could have a significant impact in the near term on some of the bigger challenges faced when attempting to build a ‘streaming everything’ game experience.

Millions of gamers play Fortnite, but the experience is segmented across different platforms and multiple devices. Recently, progress has been made by stitching together some elements of these ecosystems (thanks Microsoft, Nintendo, Epic).

However, Fortnite is a special case. It’s the world’s biggest game, the users have spoken, and “stitched” is the operative word here. PlayStation is still holding out. For most other games, forget it.

Users clearly want the Netflix & Spotify account experience. Cueing a track at home on an Echo, transitioning seamlessly to phone during a commute, and back onto a laptop at work.

The decentralized architecture at the core of Edge Computing, and platform providers like Macrometa that eliminate backend complexity for developers will help to enable this same seamless user experience for high bandwidth games.

2. Next-Generation Match-Making & User Management

There are plenty of back-end-services out there for multiplayer gaming. If you’re looking for the basics of match-making, leaderboards, and friends lists you’re already covered.

Larger game communities are magnitudes more demanding and require custom built tools – entire studios dedicated to live ops management and custom network architecture.

For example, Activision subsidiary Demonware helps build the multiplayer architecture for millions of concurrent players throwing mountains of real-time data at games like Call of Duty and Destiny 2. Under the current limitations of a centralized cloud/CDN internet architecture, it is a miracle these teams have made it work as well as it (usually) does.

But often it doesn’t work well, especially at scale. Destiny 2’s matching has been the source of community frustration for years. The game’s sluggish inventory systems are constantly lagging. Meanwhile, the PlayStation Network achievement ecosystem is bewilderingly slow to sync. Legacy cloud-based database architectures simply can’t keep up with increasing data and state management demands for much longer.

We need a better core solution for this. As we transition into the upcoming generation with more ambitious game designs, more users on the network, and more data flowing through that network, systems built on hyper-local Edge architecture can dramatically improve match-making accuracy and multiplayer user experiences across the board.

As much as we love Pokémon Go, the sheer volume of users and location data made for a laggy launch experience. It still gets sluggish around high data focal points.

Edge Computing’s architecture is tailor made for a new generation of challenging, high bandwidth AR games. Distributed platforms like Macrometa let developers synchronize player activity in much smaller groups based on location data, and then can automatically trigger functions that respond to local gameplay actions on a peer-to-peer basis.

The states of all in-game entities are also stored in real time at Edge locations much closer to the user than current centralized cloud servers, letting developers execute much more complex AR simulations, and with much less lag.

4. New frontiers in Real-time Data driven gameplay

 can sync local game world and player states in real-time, this also has huge implications for game design. If you’re designing experiences with a fixed, or intermittently updated version of what the game world is and what the player’s doing in it, you’re limited to a much less dynamic set of options

Edge Computing also manages data not just in real-time but in context– designers can use this contextual data to create custom experiences based on actual player progression. That opens up exciting new possibilities for truly persistent game play in online worlds, while actually lowering the processing and network overhead on a gamer’s console or device back home.

As we approach gaming’s next generation – our industry’s 10th– we collectively have an opportunity to harness the potential of Edge computing. The applications promise not only to solve age-old problems, but also unlock new experiences for developers increasingly frustrated by an infrastructure built for an older paradigm. 10 years on from the OnLive experiment, it’s time to embrace that future.

Macrometa is a global platform created from the ground-up on the Edge, powering the next generation of planetary scale games and apps. To arrange a demo at GDC or Google Next, contact: 

Beyond The Cloud: Edge Computing Opens New Frontiers For Online Gaming

OK, I admit. This is a controversial statement. After that admission, I can honestly say that as a 

, cloud computing doesn’t matter. It doesn’t make any difference where my apps live. Now, to the fun part…

I recently joined Macrometa for reasons that were fairly clear to me. My professional life has been largely spent waiting. Waiting for responses from customers, waiting for responses from colleagues, but worst of all - 

waiting for responses from applications and websites

What really matters to me? Performance. Content availability. Dynamic data delivered in real time. What do all three of these things have in common? From an application perspective, the only way to deliver them is with edge computing.

Edge computing is all about running applications closer to where I am accessing them, and eliminating latency and hops between my Mac and wherever the app happens to call home this week… AWS, Azure, anywhere. If you imagine the internet and by extension cloud computing as a hub and spoke architecture, AWS and similar services are the hub. I’m at the far end of the spoke, with points of presence (PoPs) in the middle. The PoP is the first piece of the app-enabled network that I access.

Until recently, the PoP had very limited value. It connects me to apps living hundreds (if not thousands) of miles away, and also had resident Content Delivery Networks (CDNs) running locally. The CDNs cache the static content and graphics I use from commonly accessed websites, but as soon as I make a weird content request (which I’m known to do) it is forwarded light years away to a sub-basement of a cloud datacenter and served up with the most fantastic latency possible. If all I did was browse Buzzfeed articles, this would be fine. But I’m actually trying to use web applications to 

Here is where edge computing comes in. The operative word is 

. Moving application logic, analytics, and content closer to me improves my experience. Moving cached content closer to me doesn’t help once I navigate away from Buzzfeed’s “30 Items for Valentine’s Day” article. Having said that, once I navigate to ‘open source games with Python APIs’ I’m on my own in a wilderness of cold storage and waiting for the page to load.

Macrometa is a company that really understands this problem. They’ve created an easily consumed PaaS for developers that blends a globally replicated database with all of the developer tools necessary to get a globally distributed application ready for distribution in minutes – not months. That application’s compute resources live 10 miles from my house or in my browser, not a thousand miles from me. The result is the best possible (and shamelessly selfish) user experience, while retaining enterprise management and distribution.

It may be too early to say that Macrometa is the “

”, but I can honestly say that it is definitely not the “

” that most of my web apps (including CRM) are currently delivering.

If you’re a developer, this is the new “

” platform. If you’re a consumer of apps, cross your fingers and hope that in the near future your apps will be delivered on this model. Cloud is no longer new, or optimal. It’s all about edge computing.

Cloud Computing Doesn't Matter In 2019

We are told that low latency and imagination are the only prerequisites for building tomorrow’s edge applications. Tragically, this is an incomplete and false hope. In order for robust, world-changing, enterprise applications to emerge, Macrometa believes we must first solve the very hard problem of bringing stateful data to the edge. Without stateful data, the edge will be doomed to forever being nothing more than a place to execute stateless code that routes requests, redirects traffic or performs simple local computations.

At Macrometa, we have created a cloud service for Web, Mobile, IoT and Augmented Gaming app builders to create sophisticated, real-time, globally distributed, edge applications. Much like how CDNs bring static content closer to end users making websites quick to load, the Macrometa Global Data Network (GDN) moves cloud based SaaS, web, mobile, IoT and database apps & APIs to the network edge. We enable anyone to build & run globally distributed, synchronized, fault tolerant applications powered by dynamic, query able data milliseconds away from users and devices across major cities and urban metropolitan regions worldwide.

During our beta program – customers built new e-commerce, mobile, IOT, gaming apps and interactive web experiences enhanced with real-time edge located dynamic data, enhanced their static CDN with our dynamic content engine delivering instant database results via GraphQL and REST, created distributed, high frequency analytics apps with edge based stream processing, solved complex GDPR problems by geo-fencing data with automatic consent compliance tracking. They have augmented (and even replaced) apps & API backends that were built on Google Cloud Spanner, AWS DynamoDB, Firebase, MongoDB, Kafka, & more.

We are excited to announce our exit from stealth with a server less edge cloud that spans 25 global metropolitan regions offering a “global database, global stream processing & function/container” as a service platform. Our edge cloud will expand to over 50 regions in 2019 and we plan to expand service to over 100 regions by mid 2020.

Turns out that that imagination is the only prerequisites for building tomorrow’s edge applications with Macrometa - Learn more at 

Macrometa Exits Stealth With A Planet Scaling Serverless EDGE Cloud - The GDN

Macrometa CEO and Co-Founder Chetan Venkatesh sits down with the brilliant & funny CEO of RackN systems Rob Hirschfeld and CMO Stephen Spector to talk about Edge Computing on the Latest shiny podcast.

We talked about how cloud architectures will evolve to create a true edge cloud capable of complimenting and perhaps even replacing public clouds some day.

Livin On The Edge @L8istSh9y Podcast

We are told that low latency and imagination are the only prerequisites for building tomorrow’s edge applications. Tragically, this is an incomplete and false hope.

In order for robust, world-changing edge native applications to emerge, we must first solve the very thorny problem of bringing stateful data to the edge. Without stateful data, the edge will be doomed to forever being nothing more than a place to execute stateless code that routes requests, redirects traffic or performs simple local calculations via serverless function. This would be the technological equivalent of Leonard Shelby in Christopher Nolan’s excellent movie Memento. Like Shelby, these edge applications would be incapable of remembering anything of significance, forced, instead, to constantly look up state somewhere else (e.g., the centralized cloud) for anything more than the most basic services.



 problem. It’s more than simply a distributed compute problem. It’s full power cannot be realized by simply spinning up stateless compute on the edge. Conventional enterprise-grade databases systems cannot deliver geo-distributed databases to the edge while also providing strong consistency guarantees. Conventional approaches fail at the large globally-distributed scale ng because our current database architectures are built around fundamental tenets of centralizing the coordination of state change and data.

To continue reading this article, please click here

Edge Computing Is A Distributed Data Problem

Macrometa is the hyper distributed cloud platform for enterprises building global applications at the Edge. 

PhotonIQ

Global Data Network

Learning Hub

Case Studies

Developer Hub

Demos

Support

About

Partners

Press

Careers

Contact Us

Terms of Service

Privacy Policy

Trust Center

Macrometa vs. Edgio

Are you an Edgio customer facing uncertainty? Discover how Macrometa ensures continuity and delivers high-performance solutions for your business.

Archive