Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

The time when web was very slow and broke easily

In 1998, the web’s largest websites were straining to meet the demand of a new class of power user customers who had upgraded from dial-up internet access and were now using DSL and ISDN to connect to the internet. No more content to surf the internet (what a dumb term - 

Jean Polly I will never forgive you for that

), internet users were downloading audio and video and all kinds of big objects in a frenzied apoplectic feeding on multimedia.

They were using web-based email like Hotmail (and my personal favorite Rocketmail) and chat messaging services to instant message or IM each other. Powering this explosive growth of the cloud was a unique new infrastructure invented by Tom Leighton and Danny Lewin at a startup called 

 in Boston. And the brilliance of the algorithms and technology that they invented ended up becoming one of the most important critical infrastructures that enabled the web to become a part of our daily lives. 

In spite of the fact that the data these websites served was thousands of miles away in primitive and precambrian clouds run by companies like Exodus Communications, the 

 enabled the web to scale to be used by billions of people and made all these websites run quickly and instantly. Had it not been for Tom and Danny, perhaps the web as we know it today would not exist. That’s how important the technology they invented and the company Akamai that commercialized it is. 

Today’s cloud is also slow and breaks easily

Fast forward a quarter century to today (November 9th, 2022), the cloud is having a similar struggle. The challenge to scale and deliver data from cloud-native apps and APIs to billions of people using apps on smartphones and tablets, interacting with content on smart TVs, smart refrigerators, smart home appliances, still prevails. And not just that, folks are commuting in connected cars on rideshare services, getting food and groceries delivered by robots and drones, and transporting themselves on electric scooters and bikes.

The cloud desperately needs it’s Akamai-like breakthrough moment when a new technical infrastructure and paradigm enables the cloud to scale to meet the needs of serving data and apps to billions of humans, using tens of billions of devices nearly instantly, anywhere in the world. That infrastructure and paradigm are the 

The Global Data Network or GDN is a new kind of distributed computing infrastructure that solves the hard problems of distributing data and computing geographically to 1000s of locations around the world. A 

 can connect data from anywhere, on any system, in any format, and turn it into APIs. This can further feed edge compute and in-region processing and serving of apps, from the CDN and wireless networks to enable a seamless, private, and secure computing in compliance with sovereign data regulatory frameworks via a suite of 

Macrometa’s GDN is the edge computing platform that is designed to help developers build new innovative digital services that take advantage of time-sensitive events, spread around global geography, yet with hyper proximity to where data is being produced or needs to be consumed - all with the power and simplicity of a serverless API or SDK.

How Macrometa and Akamai are expanding the edge with a supercloud

Today, we are incredibly excited and pleased to announce that the original OG of edge - 

 and the new upstart disruptor of Edge - 

, are both partnering to create an Edge Cloud Super Platform that amalgamates the deep capillary network of Akamai’s edge (4200 locations) and its industry-leading network backbone, with 

Macrometa’s Global Data Network Platform

 for developers to build globally-distributed apps and APIs. 

By combining the power of Akamai's edge, Linode’s cloud, and Macrometa’s GDN via a set of product and technology integrations, enterprise developers can now seamlessly build apps across cloud and edge with a new level of simplicity, and speed. Where the old cloud was slower and breaks easily, the supercloud can power real-time apps anywhere. And a supercloud deserves a super platform for building those apps.

Macrometa has been a customer and partner of Linode for close to four years now. We adopted Linode early because we were impressed with the performance of Linode’s infrastructure, the very reasonable transfer costs of their dedicated network, and their impossibly amazing developer support.

Our use of Linode has grown over the years to the point where we started collaborating on bespoke customizations that only we may need among all of Linode’s customers due to our unique needs as a Global Data Network. With Akamai’s acquisition of Linode earlier this year, we were delighted to hear about Akamai’s plans to expand Linode from its current less than dozen worldwide regions to potentially fifty regions worldwide and more importantly their deep integration with Akamai’s backbone network and edge platforms.

This now means that our customers can leverage Macrometa to build apps that will run better than cloud economics in many more regions around the world on our 

. This also means that developers can leverage Macrometa on Linode for deep compute power to run CPU and memory-intensive workloads that may not fit a javascript service worker model (like Akamai’s edge workers or Cloudflare workers). We can build what we like to call poly cloud apps - that straddle not just different cloud providers like AWS and Google but also span from cloud to the edge across Linode and Akamai.

To enable customers to adopt the edge for enterprise apps, Akamai has announced the 

, that provides $120K annual usage credits for year 1, and discounts on usage of 75%, 50%, 25% in succeeding years. Customers can take advantage of Macrometa via the RISE program to build and deploy apps on the edge. Macrometa themselves went through the RISE program a few years back and has first hand experience of the power of the program.

Please watch the following video where Macrometa CEO & Co-founder Chetan Venkatesh, sits down with Akamai’s director for business development 

 to discuss how Akamai is scaling Linode to meet enterprise needs and the reasons for creating and then expanding the RISE program.

The Integration of Akamai Edge Workers and Macrometa Global Data Network

We are making a powerful and seamless integration between Macrometa Global Data Network via console available as a preview (with a general release planned for Jan 2023 where we will also support a common API and SDK) of our direct integration between Akamai edge workers.

Macrometa enables developers to quickly and easily build APIs on data at rest via 

. These query workers and stream workers function as a service that runs inside our poly model database with data locality and low latency but more importantly are exposed externally as a RESTful 

With our new integration with Akamai edge workers, we can automatically compile the query worker or stream worker into a native Akamai Edge Worker and automatically deploy it on to Akamai’s edge. All that the developer needs to do is to provide their Akamai credentials and with the click of a button, the API gets generated as an Akamai Edge Worker and automatically deployed to 1000s of Akamai edge PoPs that query and serve data from adjacent Macrometa regions on Linode. You can even provision, de-provision, debug and log your edge workers directly from Macrometa (debug and logging will be in the general release and are not available in the preview).

Demonstrating the power of the edge to enterprise developers

To showcase the power of our integration, Macrometa and Akamai have jointly developed a full video streaming app (like a Netflix or a Hulu) that runs on the Akamai edge and Linode leveraging Macrometa, Linode and Akamai Edge Workers. We chose this particular example to show how a complex cloud based backend composed of many microservices could be moved to the edge to power a range of APIs and web services to power the front end experience of a video streaming app or Over The Top (OTT) video services. Netflix engineers have written extensively about their 

complex and arcane microservices architecture on AWS

Our sample application suggests that much of that complexity and cost can be reduced and performance improved by 25X to 50X per web request from client to backend and back by leveraging Macrometa, Linode and Akamai.

Macrometaflix OTT/Streaming Video Application

Closing thoughts: The Edge is your oyster

Edge Computing for the enterprise is finally here and now - with Macrometa and Akamai. The Global Data Network or GDN does to the Cloud what CDNs have done to scale the web. We haven't just built a new cloud - we've built a supercloud for next-gen apps.

Enterprise developers can now write applications powered by real-time data that run unfettered by the constraints of a centralized cloud deployment model, across a world wide network of data centers and edge PoPs, delivering powerful new digital experiences, instantly, securely and cost effectively at a scale that is 1000x the cloud.

The future of the cloud is edge, and the edge is already here on Macrometa and Akamai. Come build on the cutting edge with Macrometa and friends. Excited to try out Macrometa? Request a 

Expanding The Edge With Macrometa And Akamai

Macrometa is pleased to announce that a number of exciting new features are now available with the release of the 

, which enables developers to easily create full text search indexes, are now Generally Available (GA) for all Macrometa developers. The Macrometa integration with 

, a powerful compute engine for creating and deploying microservices at the edge, is now available in public preview. Available in public beta are 

, Apache Kafka wire compatibility, and Redis API compatibility.

 and streaming analytics engine, is now Generally Available (GA) to all Macrometa developers. With 

, Macrometa users can build sophisticated Complex Event Processing applications. Stream Workers includes high level primitives for pattern matching, flow control, joining multiple streams and data from operational data stores, and other CEP capabilities. To learn more about Macrometa Stream Workers and to see this in action, check out this blog and demo from the Macrometa Product team.

 makes powerful full text search as easy as writing a SQL query. Macrometa Search supports integrated search indexes on any type of data collection and across multiple collections in the 

. Macrometa search includes not only standard text search features like similarity scoring and a variety of tokenization options but powerful advanced capabilities like graph search and geospatial queries. To learn more about Macrometa Search, be sure to check out our 

Macrometa and Akamai are announcing Macrometa Query Workers and Stream Workers integration with Akamai EdgeWorkers. Akamai EdgeWorkers is a powerful compute engine for creating and deploying microservices at the edge. The combination of Akamai’s compute engine with the Macrometa GDN allows our customers to develop microservices with powerful data storage and streaming capabilities, all running milliseconds from your users. To learn more about Macrometa’s Akamai EdgeWorkers integration, check out our 

Postgres-compatible SQL is now available in public beta and joins our existing Macrometa query language. While the Macrometa language has always been largely SQL-compatible, having Postgres language support will allow our customers to port and in many cases “forklift” existing code bases directly into the Macrometa GDN without needing to rewrite their queries. To learn more about Macrometa SQL support, check out our 

 is a popular data streaming engine. Macrometa users have been able to read from and write to Kafka topics from our Stream Workers engine and we now support writing directly to Macrometa data stores with beta support for Kafka wire compatibility. Macrometa’s Global Data Mesh makes no distinction between data at rest and streaming data and all data stores are available to connect to as streams using the Macrometa API. Now, customers can connect to collections as if they were Kafka topics without having to learn a new API. Kafka consumers receive a change data capture event stream from the data store that includes both new data streamed to the collection and inserts, updates, and deletes made through data at rest CRUD operations.

Redis is a popular NoSQL database for use cases including caching, session management, real-time analytics, and more. Macrometa now includes beta support for Redis compatible collections in the Global Data Mesh, bringing your data within 50ms of all of your users, no matter where they are. Redis developers can get up to speed with Macrometa’s new Redis collection type very quickly. Porting or extending existing Redis applications to the Macrometa GDN is as easy as swapping out a library.

Excited to learn about the entire release? Follow the video below and 

Announcing Macrometa Latest Release - Global Data Network 17.11

We’re excited to announce the general availability of 

Macrometa’s complex event processing (CEP)

Complex event processing is a tricky topic, made simple by 

 giving an overview of CEP and the distinction between CEP and streaming analytics. 

CEP allows you to work with one or more data streams, optionally combined with data-at-rest, to infer “complex events”. For example, you might have streaming financial trade data, and by joining that with other data about macroeconomic conditions, you could infer a “market correction” event.

Stream Workers provides typical streaming analytics capabilities, including joining streams, aggregations, counts, and filtering as well as sophisticated high-level primitives for 

 such as pattern recognition, conditional logic, and flow control. The Stream Workers engine runs within the Global Data Mesh, allowing you to incorporate operational data-at-rest with streams of data from both Macrometa’s streaming collections and other event streams, such as from 

 topics. Stream Workers can write to data stores within the Global Data Mesh or stream results to other streaming frameworks like 

Stream Worker's applications can be queried directly using a REST API

. Since Stream Workers is tightly integrated with the 

, there is no need to transfer data from your operational data store to the CEP engine. And they are, ultimately, making large amounts of reference data available instantly for sophisticated predictive modeling at very 

This may sound very complicated and, until now, it has been, but with Macrometa Stream Workers, all developers need is basic SQL and Javascript skills, and the Stream Workers engine handles the rest. In addition, your application can run simultaneously at over 175 locations worldwide and synchronize automatically at the speed of light. As a result, companies and developers using the Macrometa Global Data Network have a real competitive advantage with access to a powerful distributed 

 engine without the developer and administration complexity you’d expect.

Stream Workers can write self-contained microservices and traditional streaming analytics applications. Using Stream Workers, you can create 

, create microservices, and serve the logic of an application backend. To demonstrate this, we’ll show our cryptocurrency trading demo below. Stream Workers can solve so many problems, but here are some simple use cases:

Transform data from one format to another, for example, from XML to JSON, and clean and filter your data:


) data from log files and scrape HTTP endpoints.

Obfuscate sensitive information by filtering it from messages, logs, or other data streams that may include Personally Identifiable Information (PII) or Personal Health Information (PHI).

Correlate data by joining multiple streams to create an aggregate stream.

Derive insights by identifying event patterns in data streams:


Summarize data with time windows and incremental aggregations

Trigger actions based on streaming data, either as a single service request or a complex enterprise integration flow.

Build and refine predictive models for machine learning

Make real-time offers to customers who are visiting an eCommerce site based on browsing or shopping history

Threat detection in cybersecurity applications

Stream Workers Demo: High-Frequency Crypto Trading App

 to trade Bitcoin locally in US, Germany, and Japan exchanges.

An exchange arbitrage is a trading strategy based on the differences between the price of the cryptocurrency at different exchanges. Generally, these opportunities open up if there is a price discrepancy, and the contrast accumulates over time to finally become a significant amount, generally 1–3%. However, these arbitrage opportunities typically only exist for seconds, so an arbitrage trader must constantly search for the best opportunities and implement them immediately.

Implementing arbitrage strategies in the crypto market takes work and involves several technical complexities. Besides having deposits in multiple major exchanges, on the technical side, it consists of the ability to monitor arbitrage opportunities simultaneously, making trades locally at each exchange with minimum delay and timely visibility and interactions between the trading agents distributed globally. The challenge requires big dollars and, until now, a team with the technical expertise to build a distributed infrastructure.

To build the demo, we used the following:

Macrometa Global Data Mesh collections, both streaming and data-at-rest, and 

We use a document store to store trade data and share all trade-related metadata and state between the decentralized, stateless bots and components.

We use geo-distributed streams for publishing real-time price data from each exchange. Each bot subscribes to the stream and decides on whether to buy or sell.

Lastly, use real-time updates from the datastore to notify the clients/bots/subscribers of changes to the database so they can modify their trading strategy.

The entire backend of this application is written in SQL and Javascript with only 174 lines of code! Watch the video for a complete app walkthrough, or check the code on 

Stream Workers logic is written in standard Postgres SQL and Javascript. Our web console has a full-featured REST API, code editor, and publisher. You can query any data store directly in your code without any configuration.

We provide native sources and sinks to connect to Kafka, Google Pub/Sub, MQTT, and S3.

After you’ve written your code and are ready to save the Stream Worker, click save, and pick the regions to deploy your Stream Worker.

When you’re ready to go live, click Publish. You don’t need to manage any software images or configuration; we get your app up and running in a few seconds. Macrometa’s intelligent traffic routing means every time a user or client sends data to the GDN, and it’s routed to the region closest to them. That means your applications will be FAST.

We provide several sample Stream Workers samples in the web console to get started quickly.

We think Stream Workers is ushering in the next era of developing applications at the edge, but don’t just take our word for it. Instead, read through our documentation to see all the available APIs, and 

 to try them out. We can’t wait to see what you build.

Implement Complex Event Processing in Minutes (Macrometa tutorial)

Using Macrometa Stream Workers In A Real-Time Log Analytics Dashboard

Announcing Stream Workers: Complex Event Processing And Streaming Analytics At The Edge

 now includes native support for SQL. This blog post discusses how we did it (and why it is important) and then digs deeper into how to write queries.

 (GDN) is a Platform as a Service for building and running applications at the edge. Run regionally, globally, or both - the GDN automatically synchronizes data between locations in milliseconds. Build new applications or extend existing applications to improve performance and comply with data locality requirements. We provide data and compute infrastructure from more than 175 locations globally, reaching two billion users and ten billion devices in 50ms.

 in our latest release, developers don’t need to learn a new query language to harness the power of the platform.

Historically, SQL has been used to manage relational databases on-premises, and more recently it powers many major legacy cloud databases, such as Snowflake and Amazon RDS. SQL has been with us since the 1970s and is the database query language powering most websites, most business logic, and business intelligence applications in businesses both large and small across the globe. SQL is by far the most popular language in the world for querying databases. When we compare SQL with any general-purpose language by usage, it sits around tenth place. (

 SQL sits in fourth place and is ranked above general-purpose language giants such as C# and Java.

The Macrometa platform is a distributed edge-native platform that runs across multiple data centers. Most real-life applications are stateful and our platform maintains state across up to 175 locations globally, allowing you to automatically synchronize session state and any other data you store in the Global Data Mesh, all at the speed of light.

In the latest release, we have added support for the Postgres SQL query language in the Macrometa Global Data Network. With this addition, SQL - which has been traditionally used with monolithic platforms - can be used with the distributed Global Data Network. There’s no learning curve because developers can continue using the tools and language they are comfortable working with.

The Macrometa Global Data Network provides very low latency and very high performance for your applications. You can get productive on Macrometa’s platform right away by using standard SQL. We also support the Macrometa native query language, DynamoDB APIs, and Redis APIs.

 and be sure to check out the rest of our 

The Macrometa Platform Now Supports The SQL Language

We’re excited to announce the general availability of Macrometa Search in the 

. In addition, Macrometa Search is now available to all Macrometa customers.

Full-text search is a powerful capability that opens up many use cases but is less widely adopted than it might be because traditional search engines such as Elasticsearch and Solr have required data movement, complex search engine management, and developer search expertise. Macrometa Search takes the complexity out of the search. Getting started with Macrometa Search is as easy as writing a SQL query, with no data movement is required because Search Views index your data in place in the Global Data Mesh; no separate search engine to administer because Macrometa Search is a part of the 

, and with many advanced search capabilities for developers who are search experts.

With Macrometa Search, developers can build powerful search-powered apps by creating Views on existing Key-Value, Document, and 

. There is no need to spin up a new search cluster and build a data pipeline – all you have to do is configure a Search View across collections in Macrometa and start running queries.

Search indexes and data are automatically synchronized across any number of chosen locations (over 175 regions globally). This brings the power of search close to your users for lightning-fast query responses. Search Views are automatically synchronized across geographic locations without any configuration or management required, so your user who is in New York today and Singapore tomorrow still receives the same low latency, customer-delighting experience from your application. The GDN’s global 

 is less than 50 ms from over two billion users and ten billion devices worldwide. So, your apps will be FAST.

What can you build with Search in Macrometa?

Search allows you to perform instantaneous and sophisticated full-text search queries for 

 in documents across multiple linked collections. Each search view represents an inverted index that contains the search configuration and all document attributes in the linked collections.

You can facet queries by document attributes, sort the results by relevance using popular scoring algorithms such as TF-IDF and BM25. Macrometa Search supports fuzzy search, correcting typos and spelling errors. Macrometa Search supports analyzers for text in multiple languages. Even if you don’t need the full power of Macrometa Search, you can call APIs to leverage Macrometa Search for stemming, tokenization, n-grams, and other search indexing functions “a la carte”. Macrometa Search provides powerful advanced features like geospatial search and graph search in addition to standard search engine functions such as pattern matching, range queries, token search, phrase and proximity search, and faceted search.

It’s now straightforward to add search functionality to a website. Still, you can also do things like add location awareness to an app (to find a restaurant close to you, for example), have an up-to-date inventory for your eCommerce store search, or power a real-time analytics dashboard with up-to-the-second data - the possibilities are endless.

So, how easy is it to add a search to an app? Let’s check out a demo to find out!

Search Demo: Extending the Address Book Quickstart app

In this demo, we will add search functionality to our 

, an address book. The guide walks you through creating a complete globally distributed backend with a CRUD API and native data streaming in less than 10 minutes.

We have already created a datastore, added some data, and created 

 We have a pre-built front end for you to use.

We will create a Search View, define which fields we want to index, and create an API with a Query Worker to power the search field on the app's front end. Check out this video.

The Macrometa Search developer experience

Building search functionality with Macrometa is surprisingly easy compared to traditional methods. You're ready to go if you have data already stored in Macrometa. You can also create a cache of an existing database to bring your centralized data to the edge and add search, vastly decreasing user latency and improving the user experience.

Once you have data to search, create a Search View and define what to index among your collections.

The index will populate nearly instantly depending on the size of the dataset. Once the Search View is ready, you can interact with it by 

 through any of Macrometa’s supported query languages as if it were a collection or through the 

. You will have immediate access to a powerful search engine with advanced features such as geospatial searches and graph searches right out of the box.

One of Macrometa’s guiding principles is to make every developer a hero by making simple what was complex. Full-text search is a powerful feature that opens up many use cases but is not as widely adopted as it might be because traditional search engines have required data movement, complex search engine management, and developer search expertise to get started.

Macrometa Search is a powerful product for search experts, and all of the tuning and power user functionality is available. Best part? You don’t need to know about any of that to get started and start realizing value from Macrometa Search immediately. We can’t wait to see what you build.

Beyond Text Search: Exploring Advanced Search Capabilities With Macrometa

Expand Your Search Capabilities With The Macrometa Global Data Network

Announcing Macrometa Search: Powerful Full-Text Search, As Easy As Writing A SQL Query

We’re excited to share some big changes happening here at Macrometa. Over the past 6 months, the Macrometa design team has been hard at work crafting a new visual aesthetic, brand, and design system that touches every aspect of the company, as well as completely rebuilt web application. This has been a labor of love, and is still very much ongoing. But today, I would like to share a look at what’s new, along with a peek behind the curtain at some of our tools and methods.

First and foremost, I’m delighted to introduce our new logo, one of a kind that defines us.

The octopus has long been a cherished creature by all of us at Macrometa. I mean, who doesn’t love the noble octopus? They’re such marvelous beings; incredibly intelligent, adaptive, and elegant.. an evolutionary masterpiece by all accounts. We also love the octopus because it’s a perfect metaphor for the Global Data Network.

For example, the brain of an octopus is fully distributed across their entire body, with each limb containing its own independent “mini-brain”, much like how the GDN is a fully-distributed active-active architecture with no single point of failure.

But I digress, we’re here to talk about logos and pixels! Read more about our love for the octopus here: 

With our new logomark, the beloved octopus finally has a home front and center in everything we do. This mark brings clarity, symmetry, and confidence into balance with the playful, fluid nature of the octopus. It’s unique and yet quite familiar, offering a bit of personality and energy wherever it goes.

The octopus can be a rather challenging creature to incorporate into a logo, simply because they’re so amorphous and pliable. It becomes an exercise in balancing several contrasting attributes: how detailed vs. abstract, how cartoonish vs. realistic, and how cute vs. serious? Too much or too little in any direction and the concept misses the mark entirely.

Our new brand typography is centered around the 

, a geometric san-serif family by type designer and typographer 

. I love this typeface. Averta combines the stab reliability of rounded geometric glyphs with an open, refreshing playfulness. It also looks great for large headlines while maintaining sharpness and legibility at smaller sizes.

Early days of the Macrometa Design System (and a new Web App too!)

Over the past several months, our engineering and design teams have been hard at work building a new and greatly improved web console. This app serves as a primary touchpoint for all users of our platform. Our web console is by every measure a mission-critical application and was previously written in 

If you’re familiar with BackboneJS, you will have some sense of what we were up against. If not, BackboneJS was an early JavaScript application framework based on the MVC architectural paradigm. When BackboneJS first launched in 2010, it brought a huge leap forward for authoring interactive web apps. But alas, the time has marched on. We considered a handful of ways to port our app away from BackboneJS incrementally, but in the end realized we needed to bite the bullet and rebuild the entire thing from the ground up. And so, that’s exactly what we did.

, and all the wonderful open-source conveniences one might expect under the hood of a modern frontend stack. This new app architecture has already vastly improved our development velocity, which will soon become evident with more frequent and featureful releases. Having worked on a lot of React apps in my time, I can’t tell you how unnerving it is to suddenly be sent back in time 10 years, losing access to all of the amazing tools and design patterns that have happened since. Let’s just say, it’s good to be back!

Of all the enhancements under the hood, I’m most excited about the addition of 

 as the foundation of our UI component library, and how this has allowed us to move fast while unifying the presentational layer of our built applications with our design tooling and resources. I’ve used Chakra UI on several occasions prior to joining Macrometa, I really can’t sing its praises loudly enough. I highly recommend taking it for a test drive, if you haven’t already.

Like many design teams, we work almost exclusively in Figma, and have produced a comprehensive collection of foundations, components, patterns, and guidelines to support and streamline our work. The mocks we produce in Figma use components, styles, and spacing rules that correspond precisely to our Chakra UI component library and theme configuration, leaving zero room for divergent or ambiguous details. The official 

 was a huge help in bootstrapping our initial component library and allows for easy customization as well.

This setup helps our relatively small design team move much faster and more effectively while helping our frontend engineer teams implement our work faster and more accurately.

We’re still very much in the early days of building the Macrometa design team and are always looking for help. If that sounds like fun to you, feel free to reach out and say hi!

Macrometa’s New UI And Design Process - Unravelling An Aesthetic Pleasure

Hi and welcome to Macrometa’s inaugural Developer Week. I’m Chetan, Macrometa co-founder, and CEO. This week is the beginning of something very special and something a long time in the making and that is the reason that I and my 100 plus colleagues in 14 countries are incredibly excited to share Macrometa with you.

Macrometa is a new kind of platform for developers for building an entirely new generation of advanced distributed microservices using our 

 (or GDN) - a hyper distributed edge cloud available across 175 edge locations around the world. The GDN lets developers like you create real time data powered services that run closer to their users and their devices in edge locations (or PoPs - short for point of presence) around the world, within 50 milliseconds of access to more than 90% of internet connected humans and their devices. 

As a developer platform, it’s important for us to build and create out in the open. Our company is built on the values of Honesty, Humility and Heartfulness. And our mission is to make every developer a superhero. We want to share knowledge, share ideas, but most importantly, we want to be available, transparent and open to the developers that use Macrometa today - and those that may in the future. Having a community and being heard are deeply important and fundamental to our mission, and we’re pursuing both this week with sincerity and honesty.

There are a lot of things to be excited about here at Macrometa, and we want to share all of those things with you. So whether you’re a curious observer, a power user, or just looking to learn, we’re happy you’re here and look forward to seeing and hearing more of you in the future.

Watch the announcement video below or keep scrolling.

My cofounder Durga and I started Macrometa in Nov, 2017 with a handful of talented and wide eyed technical dreamers to realize an idea that we both had quite accidentally been pursuing for the past 20 years - an idea of building a distributed architecture to efficiently solve the challenge of keeping data in lots of places in synch with each other. We both had worked on consistent replication and synchronization techniques for all kinds of interesting problems. And invented lots of unusual ways to solve this problem across the data stack - (I have 19 patents in data related inventions alone and Durga has invented way more than I have – but he brings the humble to our values and doesn't talk about them much).

Our young and small team envisioned a world where developers could build distributed cloud apps that were free from geographical constraints and free to run concurrently in lots of places in parallel - the challenge being keeping data (or stateful data) in synch between all these locations so that compute logic would always see consistent state. This problem has vexed computer scientists for decades who have largely accepted that the C-A-P theorem forces a tradeoff between having good consistency and sacrificing throughput (and performance) due to coordination protocols needed to guarantee consistency, or no consistency (LOL eventual consistency) and simplistic conflict resolution like last writer wins to achieve a level of throughput and practical performance required to run modern apps.

To cut a long story short - Durga and the team invented a new coordination free protocol that uses Conflict Free Replicated Data Type (CRDT) we call Coordination Free Merge Data Transforms as its underlying substrate (read more about it on 

). The next set of challenges we faced were in making this radical new protocol and architecture work at Edge Scale.

Where a typical cloud is really a handful of data centers - maybe 15 -20 regions in total and a customer using maybe a couple of regions (mostly for backup or high availability) - the edge is defined by extreme scales of needing apps and data to be available, replicated and synchronized and served from potentially 100s of regions for most use cases and even thousands of regions for some use cases. To do this, the team designed and built a global network (with a Content Delivery Network aka CDN like topology). We called it the Global Data Network or GDN, with a 

 to connect, ingest, store, serve, and process data. With this architecture, developers can turn data into APIs, 

 functions and Containers to perform stateful computing at hyper proximity to data producers and consumers. We also set out to solve emerging challenges with data regulation and legal frameworks for data sovereignty with our built-in, in-region 

. The end result: a developer platform that is uniquely built to solve the problems of real-time data and operational analytics in a way that the cloud just can't.

And in reading all of this I hope it's obvious that our team is deeply passionate about developers, and have a clear vision for where technology is headed, and how the edge in particular can become a place as important as the cloud for where you build and run applications.

 is, “The future is edge native: Building on the cutting edge with Macrometa and friends.” So what do we mean by that? And why should you care? Allow me to show you.

The edge has been for the past few years - the shiny new thing, where it’s only been possible to build and run what we like to call “toy and trinket” apps leaving developers frustrated because it lacks the sophistication of the cloud primarily because the ability to store and manipulate stateful data at the edge has been missing. Most edge platforms in the market have hacks for stateful data - providing multi-region reads but single-region writes, or poor consistency protocols that make them impractical for real world use (such as not being able to update or modify a key-value pair more than once in 2 seconds).

We’re not interested in building toys here at Macrometa. We’re not a Fisher-Price platform for kids. Neither is this a Barbie dream house or a Castle Greyskull to idle away an afternoon or weekend building a toy or trinket app on. We’re giving developers the seriously powerful, enterprise grade, mission critical platforms and developer tools needed to build real world apps and APIs that developers can rely on to work well and work at scale. All with extreme levels of performance.

We’re teaming up with a few incredible partners such as Akamai and DISH Wireless, as well as amazing customers such as DevCycle, Resulticks, AppSecEngineer and Arrive.gg this week (that’s the “and friends” part) and will tell their stories about why and what they have built on the edge with Macrometa, and what you can do with it, and what’s in the future. We have demos, thought pieces, videos, giveaways, and other exciting things planned all week long and we hope that you will come and join our celebration of learning, growing, ideating, and building impossible apps on the edge with Macrometa and Friends.

I know I already said we’re not for kids, but we’re also celebrating Macrometa’s 5th birthday this week 🎉

Macrometa has grown into more than a hundred people around the world in 14 countries since those early days of November 2017. And on behalf of all of them, and my amazing co-founder and CTO Durga Gokina, I want to welcome you again to 

. Thank you for your time and support - see you on the edge.

Welcome to Developer Week Day 1

First principles thinking is a powerful but under-used approach to building systems to solve complex problems. The basic idea of first principles thinking is to break down complex problems into essential elements that you know are true and then reassemble them from the ground up to solve the problem. In my experience, this approach enables one to unlock creativity and build innovative and robust systems.

Why do folks ignore this? Or maybe folks don’t know how to break problems into first principles and reassemble novel solutions to the problem. I had excellent results with applying first principles thinking to solve complex problems.

So thought I would share my approach in this post to show how one can build systems (and how we built Macrometa) by applying first principles in two different and complex domains — financial markets and 

Building a Trading System from First Principles

Let’s take investing/trading as it is a hot topic this year with inflation and recession worries. The problem is simple - I want to build a robust and profitable trading system that generates profits with less risk & not require too much of my time. Specifically, financial markets (equity, bonds, currencies, commodities, etc.) are complex and vast, so our task is to navigate this vast complexity and solve our problem.

Most retail investors have strategies to trade/invest in the markets. If we distill those strategies, often the first principles unwittingly are:

I’m sure we can do a little better and decompose financial market behavior into three first principles.

First principles for building a trading system

In any time frame (daily, weekly, monthly, etc.),

Each market can move only in three ways: up, down, or sideways

Some markets move opposite to other markets

Archive