Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

At some point in the last 12 months, something historic happened quietly and right under everyones' noses. Without much fanfare, the center of gravity for cloud computing moved from the centralized databases and compute in giant hyper-scale data-centers to the edge of the network where CDNs live. This tectonic shift changes the way you (dear developer) will build applications, APIs and all things programmable in the months, years and decades to come.

The shift started with AWS introducing a radical new way to build applications with functions as a service 2 years ago. Dubbed "lambdas", AWS' serverless functions let you create functions of code that are spun up on demand in response to events in the real world - or in other adjacent systems. Without the burden of any infrastructure to operate or manage, Lambdas let anyone create web scale backends that scale elastically to meet demand with a powerful cost model - you only pay for the resources your functions consume while they run. As great as lambdas are they still sat in a single AWS cloud region and that meant any calls to your lambdas had to travel across the internet, over hundreds of miles, thousands of milliseconds of time to get to the runtime to execute the function. That coupled with the fact that your lambdas might be cold (no running instances to route the request to) meant that they were slow for many real time use cases. AWS solved the first problem by introducing Lambda @ edge - a node.js lambda engine that serves JavaScript functions at 100s of globally distributed locations around the world. With Lambda @ edge your requests could now go to the closest AWS edge and be served by a JavaScript function in milliseconds. Problem solved? Almost. The cold start problems remained.

Serverless functions and workers at the edge - exciting possibilities

There's still one giant problem waiting to be solved for the next era of edge cloud computing to really take off. While we've moved compute to the edge, the data is still sitting hundreds of miles and thousands of milliseconds away in lonely, centralized databases in a single cloud region and not at the edge.

Now imagine if like Lambda @edge, CloudFlare Workers and Fastly Lucet/Terranium, you had a database that was globally geo-distributed and ran in 100s of locations worldwide and delivered accurate, highly consistent, mutable data to your functions in real-time?

CloudFlare workers are similar to Lambda @ edge in philosophy but entirely different in design and implementation. Eschewing node.js, CloudFlare Workers instead use the V8 Javascript engine (the same V8 that runs in your Chrome browser but extensively reengineered to run at the CDN edge) to execute JavaScript functions inside their lightning fast edge locations spread around the world. Using their GUI Worker developer tools or the command line based wrangler tool, you can write, package and deploy a worker to all their 200+ locations in seconds and have your own global, geo-distributed workers responding to requests at insanely low latencies for fractions of a penny per invocation.

Also, the awesome folks at Fastly have their own vision and implementation of serverless workers with Lucet and Terrarium. Lucet is incredibly exciting - it can fire up a worker from (or isolate from) the cold in under 15 microseconds (that's 

). I've chosen not to write about WASM in this post - but will share thoughts on WASM in a separate post.

Macrometa and AWS Lambda @ edge - a global stateful data layer for serverless functions in the cloud and at the edge

Macrometa is an ultra scale, geo-distributed serverless database platform. Put simply, its a new kind of database that doesn't run in just one cloud region or data center - it runs across hundreds just like the serverless runtimes provided by Lambda @ edge, CloudFlare Workers and Fastly Lucet/Terrarium. But like conventional databases it provides strong consistency guarantees, a query language and a modern interface (multi model - K/V, JSON DocDB, GraphDB, Stream, pub/sub and more in one interface) and serverless API for working with it. Unlike previous attempts to build these databases that tradeoff consistency for speed, Macrometa provides a rich set of consistency primitives to ensure that data is not just mutable at the edge but is also globally and shared with accuracy and low latencies.

By coupling the Macrometa serverless database with your functions on any of these edge computing runtimes, developers can now write apps, APIs and all kinds of incredible things that automatically run globally in seconds across 100s of locations serving data and compute instantly - something that previously only the giant web scale tech companies like Google, Facebook could do.

To give you a taste of the awesome power that the combination of these technologies provide - let's switch gears and build a stateful geo distributed AWS Lambda @ edge function that runs globally and queries data from the Macrometa database edge location closest to where the function is running.

Writing a global stateful Lambda function in 5 easy steps

In this example we will serve a globally geo-distributed address book that has a bunch (about 500) names, email addresses and zip codes that are stored in the Macrometa global database across 25 global locations worldwide.

https://4uiu9z43qj.execute-api.us-west-1.amazonaws.com/default/addressbookFunc

When your client (mobile app or web browser) calls the URL of the Lambda @ edge function, DNS will resolve the address using global latency based routing to send the request to the closest Lambda @ edge PoP (a regional/local datacenter where the function is stored and runs). For context there are about 180 PoPs that AWS provides which means that users are never more than 25ms away from the vast majority of them. When the https request is received, the Lambda @ edge function is invoked and it makes a call to the Macrometa database using a light weight graphQL end point (we call it RESTQL - you can learn more about it by reading about our query language 

 - Get a free Macrometa developer account at 

 - Create a global fabric and data collection in Macrometa

Once on the main console page, click on collections and add collections

Click on create collection and use the dialog to provide details

Skip the key - we will use an autogenerated one

Give the collection a name - lets call it addresses and click save

 - Put some data in your global collection


In the query editor copy paste the address book data:

Copy the query and paste in the Macrometa query window

Click on Execute to run the query and insert the data

You can click on the collection / Addresses and see that the data has been inserted

Cool feature - on the main console click the top link global fabric and click on one of the map pins - by clicking on the >> you can open a direct console connection to that data center and use the console to browse data and perform operations on that location. You can click collections on a console in a different datacenter and see that the data has replicated.

 - Write a query to enumerate it and expose it as a geo-distributed RESTful end point


execute it and it will enumerate all the docs in the addresses collection

Click "Save as" and give it a name in the dialog - call it showAddresses

Click on queries and you should see it at the top of the list - click the Play button to run/test it

Now comes the magical part. When you saved the query, Macrometa automatically turned the function into a globally distributed RESTful end point

Click on Support in the main console, in the API viewer scroll down to the RESTQL section

Find the API end point for "Execute restql by name" (the second one under RESTQL) and click on it and then click "Try it out"

Provide your tenant id (you got this when you signed up)

Provide your fabric name (_system if you didn't create a new one - the default is _system)

Now click execute and you should see the API execute and its response displayed below

There's a section which shows you the curl command - we're going to need this in the next section

You can copy past the curl command on your command line and run it if you have curl installed.

IMPORTANT - copy the url after POST between the quotes

it looks like this (copy the highlighted part from the API explorer) - "https://<host>macrometa.io/_tenant/<tenantname>/_fabric/fabricname/restql/execute/showAddresses"</tenantname></host>

Paste in your favorite text editor and change the tenant name and fabricname to your tenant name and the fabric you created before

Now copy the part after the bearer (this is your authentication token)

It looks like this (copy the part highlighted in yellow from the API explorer in the UI console under support)-
"Authorization: bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE1NjY2OTEyNTYsImlhdCI6MS41NjQwOTkyNTYwMzQyMTIyZSs2LCJpc3MiNOTAREALTKEN3VzZXJuYW1lIjoicm9vdCIsInRlbmFudCI6Il9tbSJ9.GC5wZG0Ng6AM0wEna1un7-8G0PY6Q7ROnzCnxxrtxMA="

Copy the part AFTER the bearer till the end of the string and paste in your favorite editor

 - Integrating the Database RESTful API with the Lambda functions -

Assuming you already have a AWS account, create a new Lambda function and make sure it is placed to run at US-EAST-1

Once you get to the code editor, copy and paste the following code from pastebin


Now carefully copy the url from your text editor and paste the URL you copied from the Macrometa Console where it says host and path (make sure you don't inadvertently add or remove anything while copy pasting).

Change the tenant name to your tenant name

Next copy the authentication token and paste it inside the quotes starting from bearer in the headers section where it says 'Authorization' :

Click save and test and you should see it run inside the console and retrieve data from the nearest Macrometa databased edge location

Assuming you already have a AWS account, create a new Lambda function and make sure its placed to run at US-EAST- virginia

 - if you've not missed a step or made any typing mistakes so far, your Lambda will now deploy and automatically pull data from the closest Macrometa Fast Data PoP.

Use the API end point provided by the AWS gateway and test to see if your Lambda fetches data from on the 25 global data Pops closest to it that Macrometa provides.

Note that this lambda is not geo distributed yet

 - that will need you to setup a cloudfront distribution which I will cover in a separate post. But the steps to do that are easy and you can find plenty of examples on google for that if you want to try it out now.

So there you have it - a completely serverless, stateful, global, geo-distributed API end point that will serve dynamically queried database responses in milliseconds all around the world

Macrometa & AWS Lambda - Geo Distributed Stateful Data For Serverless Functions - A Step By Step Guide

By Christopher Meiklejohn
Consulting Scientist, Macrometa

, we discussed the challenges of conflict resolution when supporting a highly-available distributed database that operates at the edge. Let’s recap the challenges.

Moving applications to the edge is primarily done for two reasons: latency and availability. First, by moving application code and associated application data to the edge, user requests can be processed entirely at the edge and responses returned to the user quicker; this serves to reduce the user-perceived latency. Second, in the event of a data center or connectivity failure, user requests can be rerouted to another geographically close Point-of-Presence preventing periods of application inaccessibility, allowing the user to continue using the application without an interruption of service. Even further, if the user’s geographically closest Point-of-Presence is unable to contact other data centers because of connection failure, users can operate on data local to that location without having to experience an interruption of service.

Enabling this type of user experience ultimately means that conflicts at the data store are inevitable. Without waiting for agreement from all of the data centers in the system before modifying an object in the system – the key to remaining available under network partitions that are a result of data center failures and connectivity loss – concurrent modifications to the same object in the data store may conflict. These conflicts need not occur simultaneously – but only within a time window where a relationship between the two modifications cannot be determined.

To provide a practical example, if Alice and Bob both modify a document in their address book containing the current address of a relative – even if the values written are the same or minor variations using abbreviations – the system will detect a conflict and will be unable to proceed. Generally speaking, in this case the system has to make one of two choices: either keep both values, and upon the next read performed by the user return the user multiple values, when they may have been expecting a single value; or, pick one of the updates as the “winning” updates, by using some sort of arbitration procedure to figure out which of the modifications “overwrites” the other. Some systems (e.g., Riak, Bayou, Dynamo) return both values to the user and allow the user to replace the multiple values with a single value – a form of application-specific conflict resolution; whereas other systems (e.g., Cassandra) choose a single value based on a user provided time stamp. In some of the formal literature on the topic the former is known as a Multi-Value Register, the latter a Last-Writer-Wins Register. However, since applications work with richer data types than just registers – they use dictionaries, sets, and counters – keeping both values or arbitrarily choosing one might not provide the correct application behavior for the user. To provide a pathological example, if a counter is modeled as a register, for every conflicting set of writes a single increment or decrement of the counter would be lost – this may be very bad for the application!

Coming up with interesting ways to converge conflicting operations across different data types has a whole field of study and body of work behind it called Conflict-Free Replicated Data Types (CRDTs). We touched on the challenges of designing CRDTs in our previous post. In this post, we discuss how the Real-Time Intelligent Convergence Engine – at the heart of Macrometa’s C8DB offering – handles conflict resolution.

Real-Time Intelligent Convergence Engine (RICE)

Given that Macrometa’s platform must face the challenges of convergence and conflict resolution head-on to remain available at the edge when network partitions inevitably occur, it was paramount to establish a data model that was both usable from the application point-of-view and contained conflict resolution logic that was transparent to the user.

In C8DB, application users store unstructured documents – similar to JSON objects, or documents in a document-oriented database such as CouchDB. Documents are values that are associated with keys, and applications can either access a document by key – using a point query – or use a SQL-inspired query language that allows for aggregations and selections across documents located in the system. Documents can be arbitrarily nested and therefore are recursive data structures containing keys that map to values (which, themselves can be dictionaries mapping further keys to values.) To perform modifications to a document, users use the query language to perform INSERT or UPDATE operations against the data store – these operations can modify either a single document or a collection of documents.

Internally, these operations are totally ordered at each data center. Therefore, the types of conflicts the application developer needs to reason about are conflicting operations that happen concurrently at different data centers. These conflicting operations occur when the system cannot reason about which update preceded the other, nor when the system can determine some precedence between the different types of operations. In our example before, if Alice and Bob were to update the street address for the same record concurrently at different data centers, we have to make a choice. In lieu of using different timestamps provided by the user to arbitrarily pick a write – which, has been known to result in serious consistency problems in Cassandra when clock skew occurs – RICE instead chooses one of the updates arbitrarily from the conflicting writes based on the data center’s unique identifier. While this may not capture user intent, it ensures convergence and prevents clock skew from rendering the system unwritable if a clock happens to skew forward in time. These updates are performed per field in the document and when update-update conflicts occur where the system cannot determine an answer that reflects the true user intent.

When more information is available, the system can do better. If we consider the case where we have a document containing multiple keys and concurrently Alice removes a field while Bob is updating that same field to a new value, we can order Bob’s update after Alice’s, under the assumption that Alice’s removal is performed given the current value of the field – therefore, Bob’s update should survive and not be removed based on Alice’s modification. Again, we can perform this arbitration on a per field basis, and try to preserve user intent. While this might seem reasonable, this also can lead to unintended consequences.

Consider the case of an address book entry as a document containing street address, state, and a boolean flag to identify whether or not the user lives in Washington state. Alice updates the document to modify the address from Washington to California and untoggles the boolean; simultaneously, Bob updates the address to a different address in Washington state. Alice is located in DC2, Bob in DC1. In this case, we run into trouble using any methods when we try to converge.

Using data center identifiers and per field convergence, if we send only the modified fields to the system for convergence, we result in a checked boolean and an address in California. This is clearly wrong.

If we model our boolean as a register, we will not converge correctly, unless Bob sends an update for the boolean as well.

If we model our boolean using semantics where booleans should always merge from a false value to a true value, and model Bob’s change as an update of both the state (to its current value) and new address, this merges correctly, but only because Alice’s updates originated in DC2. If Alice was located in DC1, it would not have.

If Bob is located in DC1, we merge correctly.

The difficulty here is that the value of the boolean depends on the value of the address: this is an invariant that exists within the document itself, and converging documents based only on the fields in the document is not sufficient to perform convergence while preserving the invariant. Therefore, we need another mechanism to ensure that we can preserve these invariants under conflict resolution.

In RICE, these groups of updates to a document can be performed atomically. Atomic groups of updates ensure that convergence is performed per document where all of the updates for a given document are ordered together when performing conflict resolution – thereby helping the developer to ensure invariants. However, these atomic updates are only good for ensuring one-way invariants: where a given value depends on another value in the document; they are insufficient for preserving two-way invariants where the values of two fields of a document are interdependent and conditional on one another. For that, we refer you to the next post in our series on transactions.

Macrometa's CRDT Edge Database - Implementing Coordination Free Convergence of Data for Edge Computing

If you think about making your own twitter-ish app, you will soon stumble upon a few super interesting challenges.

A twitter style app will allow anyone to sign up, tweet, and then to follow people and see their tweets.

The feature Signing Up requires some kind of database to store the users. If this is just for your closest friends or a few thousand tweeters, SQLite will be fine.

Of course, you also need some back-end to handle the sign-ups and talk with the database. You could use a LAMP stack, Ruby on Rails or roller-skates, or Node.js or whatever. It doesn't really matter. Unless you want your app to be scalable, then you may want to make the back-end a stateless micro-service that talks with a stateful database like MariaDB, Postgres or MongoDB - each of which can be scaled up if you put some effort into it.

Once signed up, your users must have a way to become followers. In other words; they need a way to discover other users. That means that our app must have some search functionality. So - when someone signs up they could search for their friends, or people with similar interest (or opposite interests for the evangelists). What about other local users? How cool would it be to be able to find and follow people in your own village or neighborhood? Or in the same coffee shop or nightclub?

Since all internet connected devices have some kind of location awareness (so that advertisers can feed us the most relevant ads), how about exploiting that to locate nearby people? Getting your location from your device is trivial. But a simple database like SQLite has no geo-location awareness. So we need a database that knows about geo-locations to implement that feature. A proper geo-location aware database should allow us to directly query for the n closest entities of any geo-location (latitude, longitude) in the world.

Another way to find followers is to see who else is following the people we follow. That can be implemented using a traditional SQL database - to a certain extent. If we want to walk several levels into the follower network we are really in the realm of a graph database. A graph database tracks relations between objects, and allow us to make simple and fast queries to traverse the graph to fetch whatever objects we desire.

But the most important feature of them all is to make sure all of our followers get our tweets immediately. Not like Google Chat or Skype, that may take days to push a message notification. When we have something on our tongue, we want people to get it immediately so that they can react to it (read: Like it!).

The traditional way to implement a thing like this, using web technology, is polling. The client polls every second to see if any of the people it follows has sent any new messages. That's fine on a residential Internet connection with a PC, but not so much on a phone, where you pay for the traffic and where traffic drains the battery - not to mention the back-end resources. If you make your twitter for your friends, you just need one micro-service instance with Node.js, 5 with Ruby or 42 with Java, to serve 20 friends. On the other hand, if your app goes viral you have a real challenge. One million users, each polling each second, will be rather expensive to handle as you will need thousands of micro-services. A better approach is to push the messages to the device. That can be accomplished with grpc, push notifications, or just plain old web-sockets.

In a micro-service architecture, one way to handle pushing messages in real time to the clients is a pub-sub service. Think of it as queues or persistent streams, where each user has a personal queue or a stream she subscribes to. When someone publishes a message, the back-end copies the message to the queue or stream for each of her subscribers. As soon as the message is pushed to a queue or stream, the pub-sub service checks if the subscriber is connected, and immediately sends the message to those who are. Those who are off-line will get the messages as soon as they connect to their queue or stream.

So for our twitterish app, we need a trivial database for the users, a geo-location database, a graph database, and some kind of pub-sub service for the notifications. That's 4 stateful technologies that our stateless micro-service has to deal with (and we have to read up on, comprehend, deploy and interact with from the micro-service).

If the app goes viral, we also have to scale it from tens to thousands to millions of users. That's hard, especially if our users are spread all around the world. If we deploy our gang of 4 in Central Europe, users from USA, India, China, Australia will get latency and connectivity issues. (Even Netflix and HBO are unable to serve a global user base in a timely manner. I often switch off my TV before either of them starts to play the movie I want to watch). If your app is slow, you lose users. It's that simple.

So what if we could just make up some imaginary fairy technology that our stateless micro-service could connect to that would have low latency, scale to any size, and deal with the gang of 4 stateful technologies we require - from anywhere in the world? Then we could deploy our micro-services in Australia, in Asia, in Europe in USA, in UK - anywhere large clusters of users subscribed - and provide all of them with wonderful, reliable, low latency access to the latest fake news and trolling on the net? (Let's be realistic - if your app goes viral, all of the bad people will be there, but so will all of the good people, and we are the majority!)

Well - good news. The "fairy" technology is here!

I have spent the last few months working on it, after joining a small team of dedicated, talented people in India, Europe and USA. When it went online with an early free preview a few weeks ago, I wanted to spend a weekend or two making an app that consumed all of our main features - the gang of 4 - to get an idea about how easy or difficult it would be to actually use our service as a software architect and developer. So I made a simple app in Qt/QML that I called 

 (I'm more a dog person then a bird person) using our document database (a basic key/value store where the values are Json documents) for users and messages (barks), the graph database (edges attached to Json documents) for relations (followers), the Geo-location index type (for the Json documents) for Geo-location, and streams over websockets to receive new messages in real time.

It took me 46 hours and 39 minutes to do this. About 1/3 of the time was spent on figuring out how to implement all the client functionality in QML (usually I use C++ for all the logic - but I wanted to learn something new - and if someone wants to use my code as an example, JavaScript is much simpler to understand for most people than C++), 1/3 was spent working around bugs, figuring out undocumented features and tuning queries (don't worry - our documentation team and developers got a steady stream of feedback - so all of that is/will be fixed), and 1/3 was spent on implementing the thing.

I ended up with a very simple stateless micro-service back-end in Node.js dealing with subscriptions to the service, login, distributing barks (what Twitter refers to as tweets) and a few other things.

I could have distributed the barks in the client, but if the client is shut off or loses connectivity in the process, some subscribers may not see the barks - which will be devastating for the barker, so it was better architecture to put it in the back-end .

For me this was an expedition into unknown technological territory. I have not used geo-locations or graph databases in the past, and I have never designed a stateful, distributed application that's supposed to scale to any size just by deploying more back-end resources.

I did not make it secure, as I would have done with a real app. But that was a trade off for simplicity - I wanted the client to connect directly to the database API for as many things as possible, and only use the micro-service where it is really needed. That of course means that the client's user credentials need access to the actual databases, and this blows away pretty much all potential security. For example, if the user can write to the locations database to update her own location, she can also (by modifying the app or call the API with curl) change anyone else's location.

The Macrometa Fast Data platform is designed to offload the burden of scaling stateful components (the databases and the streams) from the application architecture and 'just work'. The service allows you to enable local endpoints all around the world just by clicking on a map in a dashboard. Then you can use geo DNS to allow your users to connect directly to the nearest endpoint of your database, or deploy micro-services in the same or close-by data-centers to handle the local users. Your document collections and streams are instantly replicated to all the geographic regions you enable. For a software architect or developer - using the database or the streams from a mobile app, webapp or micro-service is just as simple as connecting to Postgres, MongoDB or Redis from your code. You don't have to worry about deploying or scaling those components. It's just a nearby REST API endpoint (or web-sockets for the streams) with low latency and extra, cool features like geo-location and graph queries. I was totally blown away by this technology when I first learned about it a few months back. For a software architect or app developer - it opens up a totally new dimension of opportunities. Just take my barker app as an example - in one week, I created an app that can scale to millions of users all over the word with just a few clicks in the Macrometa dashboard - without doing anything at all in my architecture or code to deal with that.

Thank you for following me this far. If you think this was a cool read, please share it in the social media of your choice - even twitter.

Now - to prove that I am in fact a developer, and not some marketing guy or business person, I'll quote my update in today's Scrum call.

Boss: Did you start on the Big, Boring, Exhausting task I assigned to you?

Sign up for a free Macrometa account today by clicking 

https://github.com/Macrometacorp/barker-demo

How To Make Your Own Twitter In 5 Days

 to provide low latency and a fast and real time user experience. These kinds of games are hard to build because you need to understand low level networking inside game engines (like unity or unreal), and deal with lots of complexity when it comes to sending and receiving state data as well as storing it on a server for persistence. Now what if I told you that you could do all of this in a few hours without needing to deal with anything except a few API functions callable via JavaScript. Afterward, you'd have an 

 game with real time-data streaming and data storage.

In this tutorial, I'm going to show you how to use Macrometa to create a multiplayer, side scrolling platform game that uses Macrometa's streams to send messages to each player about the game state, controls (up, down, left, right and jump), and the database for storing persistent game state (thing like who is on what level, how many points collected, how many treasures like diamonds and keys collected etc).

It's probably good to play the game for few minutes to better understand the functionalities. The game is a single page app served off a CDN. You can start and play this game directly using below link and you can invite your friends to join by sharing the link with them):

 originally published by pubnub. The pubnub example tutorial can be found 

If you want real-time interaction among players, pubnub or socket.io won't work because both of their pub/sub architectures are centralized. That means that sending any data from the game client to each other will have to transit all the way to their central servers before being received by the other game clients. That just adds latency. Additionally - neither pubnub or socket.io let you persist game state so if you want to collect player stats in real-time - well you're going to have to use a database of your own or a cloud database - but guess what - those are centralized too and have high levels of latency to access.

While there are a lot of tools out there for writing games, most of them create more problems when you start handling real-time multiplayer interactions & state over tens and hundred of locations globally.

This is where something like Macrometa comes in. It provides ready made APIs, enabling you and your teams to build, deploy and run cross-region or globally – in real-time.

As Macrometa's Chief Architect, I am obviously biased. :-) So don’t take my word as fact. The point of this post is for you to try things yourself to determine what's easy or not. So let’s build a global, real-time, online multiplayer game called CastleKnight together!

After that, you can decide for yourself if Macrometa saves you and your team lots of time to focus on developing your game.

This game is a collaborative puzzle game that encourages you to work with your friends to collect the keys in clever ways.

Following are some useful capabilities we want incorporate into this game :

 i.e., reliable real-time game state and events and inputs across all connected players.

 i.e., incorporate interactive social features like chat across all connected players utilizing the streams feature of Macrometa platform.

 i.e., real-time score updates and game statistics to dashboards

 i.e., various levels in the game and player stats in each of those levels.

Obviously one can build more functionalities in the game, but I think for this post the above is sufficient. You are welcome to add more functionality.

The rest of the post is all about how we build the Game GUI and leverage 

 in the game. As a game developer, this is the part you should actually be focusing your precious time and efforts on.

Obviously you'll need a Macrometa account for this tutorial, but don't worry, it's free to try. Go ahead and sign up 

After signup, you will receive a tenant account along with credentials. After login, you should see a dashboard something like the image below.

You can get the source code for the game 

 to change it. Go ahead and clone the repository to a directory on your local machine.

After cloning, you should see something like this in your directory:

Step3: Configure the game to use your account

Open js/Config.js file and edit the following parameters to reflect your account:

CastleKnight is a single page app. So to run the game locally, you have to launch your local web server. If you have Mac OS or Linux (or have Python installed), open up your Terminal Application in your game folder and type in:

. There are some great tutorials out there on how to setup XAMPP on your machine.

Once you have your web server up and running, go to http://localhost:8000/ on your machine. Then navigate to the index.html file in your web browser and click on the link. You should now see a blank screen.

If you want to deploy and run the game from S3 then do the following. Go outside the current working directory i.e., tutorial-castleknight-game in this case. If you are using the AWS cli then run following command to recursively copy all files and folders inside the tutorial-castleknight-game folder to the S3 bucket.

Now go to the Properties tab in the AWS console for this bucket and open Static website hosting option. Then select the option Use this bucket to host a website and provide index.html for both Index document and Error document text fields. Click on save and the game is now live!

To begin, open up your main.js file. Following code fragment reads your credentials from Config.js and opens the connection to Macrometa data platform. Also it creates a geo-replicated document collection called occupancy to record the player occupancy in each level.

The following code fragment in loadingState.js loads the assets into the scene. We created an object called window.LoadingState with all of the loading state information inside of it. In the init() function, we made the sprite objects in the game look smoother by using the Phaser API this.game.renderer.renderSession.roundPixels = true;

In the preload function, we load the JSON level information from the data folder. This information is used to generate the various levels of the game. Then every asset that we will use in the game needs to be preloaded into cache.

Lastly we run the create() function that starts the game and loads whatever the window.globalCurrentLevel is.

Open again your main.js file. The following code fragment creates necessary global variables for the game.

Open chatEngine.js. The following code fragment initializes the chat engine and creates a geo-replicated stream between players to publish and receive chat messages in realtime around the world. The last part of the code adds the chat panel to the game GUI at the bottom.

This post became a lot longer than I originally anticipated, so I'll skip the code explanation for display of player interactions and elements in the game. Savvy javascript developers should be able to understand by looking at the code directly.

I hope you enjoyed this tutorial and it helps with your gaming project. If you notice, most of the code is about the game GUI i.e., display of assets, player interactions, etc. This frees the game developer to focus on what matters most. We’re looking forward to launching more multiplayer gaming tutorials in the future so stay tuned.

how to build your very own crypto arbitrage bot

CastleKnight - Using Macrometa's Edge Database & Streaming Engine To Build Low Latency, Realtime Multiplayer Online Games *With Bonus In-Game Chat

The emergence and the fast growth of Mobile Apps, Web Apps, Single Page Apps and APIs within the past few years is a telltale sign of the growing importance and demand for speed and faster user experiences by users. And this is not simply a psychological need for speed in our ever accelerating and connected world. It is a requirement.

Faster apps lead to better user engagement.

Faster apps lead to better user retention.

Despite the ever-accelerating pace of life, our reaction times remain constant, regardless of the type of application (online or offline), or medium (laptop, desktop, or mobile device).

 0–100 ms Instant 100–300 ms Small perceptible delay 300–1000 ms Machine is working 1,000+ ms Likely mental context switch 10,000+ ms Task is abandoned

For an application to feel instant, a perceptible response to user input must be provided within 10s of milliseconds. After a 1/2 second or more, the user’s flow and engagement with the initiated task is broken, and after 3 seconds have passed, unless progress feedback is provided, the task is frequently abandoned.

There are a number of technical solutions for improving an application’s performance, such as the data transfer capabilities of servers or routers, including compression and streamlining application roundtrip responses. But there is one thing that cannot be addressed with technology, and that’s proximity.

Solving Latency with Geography, Not Technology

Given the above, the most practical solution is to move the data geographically closer to the customer or end points for a business. This is the concept behind 

 (CDN) and caching sites. Most get deployed on the edge to provide faster access to common, or popular data and they bypass the ‘speed of transport network’ issues by getting popular content to the edge, where the customers are.

But CDN's were created to solve the problems of the "old web" - a web that was static and relied heavily on server-side rendering. They don't provide much for SaaS providers. SaaS applications are powered by query driven, dynamic data - and CDNs don't do well with dynamic data.

Macrometa brings the CDN into the cloud-native era by providing a global multi-model database and geo-replicated streams at the CDN edge. It means your apps always run on the edge with nothing to cache, purge or expire. The edge is the dynamic origin for every request and your users are always served the freshest state from the closest edge PoP.

Simply put, we may not be able to make the packets travel faster, but we can reduce the distance your dynamic data has to travel. This empowers you to serve your customers globally at high speed by enabling your Apps & APIs to utilize dynamic data in close proximity to your customers.

Given the benefits associated with making state (i.e., dynamic data) available in close geographical proximity to users, why don’t we see developers doing this? It is because maintaining state (i.e., dynamic data) at the edge across several locations is a very hard problem especially when one needs various consistency guarantees on that data.

It takes years and a team of engineers with deep expertise in distributed systems & databases to build this type of fast data platform. If you have the time, money, energy and skills then you can certainly build a multi-modal fast data platform like Macrometa. The industry can certainly benefit from more data platforms like Macrometa’s fast data platform that are coordination-free and utilize both consensus and convergence mechanisms.

On other hand, if your primary focus is in building Apps and APIs for your business needs. then the Macrometa platform can definitely help you serve your customers globally, at high speeds - thus fairly quickly, by providing a geo-distributed fast data platform with various consistency guarantees and other functionalities.

Let’s create the backend for an Address Book app in the 

. An app that is available globally at high speed (i.e., local read-write latencies) to its users by leveraging Macrometa’s fast data platform.

 in the above paragraph because typically doing the same exercise without using Macrometa’s platform will take months to build it (assuming the team has necessary expertise). Don’t take my word though. Verify it for yourself.

You need an account with Macrometa’s Fast Data platform to build the backend for the app. If needed, you can get a free developer account 

Faster apps lead to better user engagement, better user retention and higher conversions. The most practical solution is to move the data and application geographically closer to the customer or end points for your business.

Macrometa’s fast data platform enables dynamic data to be used like CDN by providing a global multi-model realtime database and geo-replicated streams at the edge. It means the backend for your apps always run on the edge with nothing to cache, purge or expire. The edge is the dynamic origin for every request from your app and your users are always served the freshest state from the closest edge PoP.

I hope the above shows how quick and easy it is to create a geo-distributed backend engine using Macrometa for your app to serve your users faster globally.

There are quite few other capabilities available in Macrometa’s Fast Data platform. Give it a try with a 

Serving Your Users Globally, Faster

Build Geo Distributed Apps with Macrometa

Here's a few examples of how Macrometa turns complex 

Build a multi region database that runs in 6 different data-centers across the world in about 40 seconds (the video is not trimmed in any way):

Create a data collection in the global database (like a table in an RDBMS), insert some data in it, and see it replicate with consistency guarantees in all the regions your fabric runs in:

Create a query to retrieve data from the collection, save and export the query as an API end point that you can call from anywhere in the world. The request will always be served from the closest datacenter to the requester

Call your restQL API end point from a web application such as a Single Page Application (SPA) or Progressive Web App (PWA): using the showContacts example from above:

Realtime data is easy with Macrometa - all changes to the database are automatically pushed to all clients via Streams (no polling ever):

And thats just the beginning. There are so many incredible ways to use Macrometa to create fast, powerful, data driven apps and APIs that run worldwide automatically such as graph processing, stream processing and serverless functions (coming soon using JavaScript WASM and V8 isolates). BTW, everything you see above can be done via our 

There's a brave new world of distributed apps and APIs waiting to be built by you for 

. It's time to say Goodbye Cloud, and Hello Edge with Macrometa.

 and get started in minutes. You can get all the 

 and we're a chat box away (to the bottom right of the macrometa web page)

Create Global Apps & APIs In 5 Mins With Macrometa's Seamless Architecture

Today, June 12th, 2019 marks a very special milestone for all of us at Macrometa. Starting today, anyone can 

 for a free developer account and build geo distributed apps and APIs that run across our network of 25 global cloud data centers.

Macrometa for building geo-distributed apps

While the idea behind Macrometa had been knocking around in our head for a little over 5 years, turning all those napkin sketches, sticky notes, midnight conversations and 

 took more than 2 years and the combined efforts of many talented and passionate engineers on our team in Palo Alto California, Paris France, Sofia Bulgaria and Pune India.

One of our most cherished ideas at Macrometa is that anyone should be able to build a globally geo distributed application or API without knowing any more than they already know about using databases with languages like JavaScript and Python. The era of needing large cabals of Phd's with years of expertise in building distributed systems to get correct, accurate distributed systems behavior with high performance needs to end.

We set out with a goal to democratize the platforms that until now only companies like Google, Facebook and Amazon have been able to build. With Macrometa - anyone can build apps and APIs that run distributed, cross region and multi cloud and they don't need to know anything about concurrency, consistency or mastering the arcane complexities of distributed systems.

PS - like our Octopus mascot? Her name is Calico and if you want stickers of her for your laptop, drop us a note at justin@macrometa.com

Build Geo-Distributed Apps - Hello Globally Distributed World!

Contributed by EDGE GRAVITY from their new blog series: 

. This collection of blogs will focus on new technologies and industry leaders in the 

 as way to educate a broad IT audience in the future capabilities at the edge.

Macrometa Global Edge Fabric is an Edge Native Platform as a Service (PaaS) for building distributed applications and APIs for the web, mobile devices, 

Archive