Explore insights on real-time edge computing, AI, and cloud solutions. Learn how our technology optimizes performance across industries.

Blog

Data is like inventory - the longer it sits, the less valuable it can become. Backend systems don’t always sync in time to provide updated inventory changes. Data traveling to centralized clouds and data centers only get delayed further. Retailers are now tracking inventory across channels and touchpoints, including online, in-store, and in-app. The goal of this omnichannel retail experience, in theory, is to provide a convenient experience for customers everywhere in the retail journey and to increase sales opportunities for retailers. To create a frictionless omnichannel experience, retailers need a real-time view of all of that data.

Data at the edge sits right next to customers, stores, and distribution centers. Instead of customer data being sent to a centralized cloud, data can be processed wherever customers and retail staff are, anywhere in the world or online. When retailers adopt edge technologies, they gain the ability to see a real-time view of inventory and other data at any access point. Retailers can move inventory across channels to the stores, warehouses, and mini-fulfillment centers. Website and application performance improves for shoppers and staff with faster response times.

Real-time offers not only benefits for inventory tracking but across all levels of the business. Buyers can analyze sales patterns and trends to adjust purchasing decisions. Marketing can offer social and email promotions based on inventory counts. Real-time data allows websites to recommend personalized, curated deals and product recommendations with greater context and relevance.

Deliver a globally visible, locally curated inventory

Many retailers operate globally or across many regions. In 

, it’s not unusual to have over a million SKUs across the world. Each geography will have a select set of products from the global inventory. These product inventory subsets add operational complexity. Mismanaged inventory can result in unnecessary mark-downs or other losses.

 change feeds provide an exact count of assortments and locations. It also simplifies the complex logistics of just-in-time inventory and dynamic pricing for the most profitable use of space. Retailers can determine what merchandise to move and what products to order with 

Customers and staff can seamlessly access available inventory by region with geotagging for product subsets. Plus, 

 is local on the edge so it can meet any and all privacy requirements.

Expand fulfillment options online and in-store

Extending the buyer journey over apps, websites, and in-store is now the industry standard. When we discuss topics like the future of retail, we’re speaking plainly about the ability of retailers to streamline inventory logistics and adopt the real-time connections between shoppers and stores. Omnichannel is the new reality of retail, and the companies that do it well will be the ones that see the most success.

Keeping store and online inventory separate is no longer an option. Traditionally, retailers kept in-store inventory separate from direct-to-customer (D2C) warehouses for online orders. Now, the line between online and in-store is permanently blurred. Shoppers can browse in-store and buy online for at-home delivery, or they can buy online and pick up in-store (BOPIS). Customers can buy online and ship-to-store (BOSS). Some retailers can cap shipping costs with buy online, and return in-store (BORIS). Customers prefer the convenience and if they come to the store, they may buy other items. Retailers need to be prepared for all the ways that customers browse, purchase, receive, and return inventory.

In the past, most distribution was handled through a large, centralized distribution center. Shipping times from these distribution centers largely depended on the customer’s proximity to that particular warehouse, which resulted in inconsistent shopping experiences. Now, retailers are moving away from this centralized model (Amazon, for example) and creating hubs. Stores now act as their own mini-fulfillment centers for retailers to ship from stores (SFS). In the past, retailers would mark down slow-moving inventory and lose profit. Now a shopper in Seattle can place an order that is fulfilled from a store in Seattle. If that store doesn’t have the inventory, it will be fulfilled by the next closest location. All these changes happen in real-time, enabling faster inventory turns and better use of selling space.

Retailers offer expedited shipping to compete with e-tailers. Fulfillment networks near the customer can use standard shipping instead. This lowers shipping costs and has a positive environmental impact while delighting the customer with a fast turnaround.

Data is also following this model. When we talk about 

, it’s really just examining where we send and store and process data. Are you using a centralized fulfillment center (cloud computing) or are you using local stores and mini-fulfillment centers (edge computing)?

The Edge of Retail. Sign up for the webinar.

Keeping data close to the customers and staff delivers faster response times to improve customer journeys and increase sales.

Customers form an opinion about all company channels based on their website experience. Page load time determines organic search results and could impact future opportunities. Improved 

 can reduce cart abandonment and deliver the experience customers expect.

Immersive customer experiences need real-time interactions and millisecond automation to support analytics, 

, and omnichannel touches. When a customer comes into a store, they can use virtual mirrors to try on clothes or makeup. Sales associates can upsell or assist customers based on real-time inventory suggestions.

Customers can also receive instant promotions and be directed to self-serve technologies like smart vending machines for faster checkout. After the transaction, they can receive a coupon for future purchases, or be shown where they can find a matching accessory.

Time and space are “money” for retailers. Real-time data and analytics - powered by the edge - are powering the most important business decision. Join our 

 to learn how your business can boost profits and customer satisfaction with real-time inventory management.

Data is active everywhere retailers do business with the Macrometa 

. Retailers can build a real-time inventory view for in-store, in-warehouse, and fleet management. Data is replicated in milliseconds and reconciled with automatic conflict resolution. Local copies of inventory data are kept in any of the Macrometa GDN 175+ points of presence.

Macrometa lets you store, serve, and process your data within 10ms of 80% of the global population.

How Retailers Can Drive Sales And Better Customer Experiences With Real-Time Inventory

Shopping behaviors during the pandemic have accelerated the need for a unified customer experience across various channels. 

 expect to receive consistent service on every channel. In order to win business and customer loyalty, shoppers require a hyper-personalized 

The same survey indicates that 32% of customers expect personalized content and product recommendations based on their past interactions with the business. 

 gives retailers the power to monitor and respond to customer behavior in-store, on social media, or web browsing by implementing personalized ads and offers. But “real-time” requires processing data close to the customers, anywhere they happen to be. 

This level of coordination and distributed computing adds complexity to retail development teams that are already stretched thin. Businesses that are unable to capitalize on real-time insights are the ones that get left behind. Retailers need to leverage the power of edge computing.

 either have deployed workloads or plan to deploy workloads at the edge. The goal is to bring data, computing, and storage near to customers, stores, distribution centers, and staff. This move to the edge of the network supports real-time analysis and personalization efforts closer to where data is generated. 

But in-store edge servers require frequent maintenance and security upgrades, and the initial set-up can present logistics challenges. For a retailer with thousands (or even just hundreds) of locations across the country, it’s cost-prohibitive to deploy on-prem data processing and analytics 

. While cloud computing is an option, millions of terabytes of data will need to travel over the network, adding egress costs and delays.

 can handle applications and workloads at scale by computing data through network nodes and data stores that are close to where the data is generated, eliminating data transfer delays to the centralized cloud. Many organizations deploy a hybrid model of edge, cloud, and on-premises data centers to deliver low-latency and the fast response times that 

 (AI) applications and real-time supply chains require - across all data sets. 

With the acceleration of new technologies to draw customers to the store and online from AR/VR experiences to self-service technologies, the edge lets retailers develop and update new applications much faster. This allows retailers to offer new services to customers ahead of the competition while reducing resource costs. Additionally, 

shoppers and staff can experience faster response times

 on their website or mobile apps because of the close proximity of data on the edge.

Process data close to customers to deliver a unified experience

AI/ML is required to accurately predict demand and even in some cases to create personalized experiences with 

 and other technologies. Having data and advanced analytics at the edge lets retailers deliver mass personalization in real-time and compete on a level field with large e-tailers. Product leaderboards, real-time data analysis, and historical customer behavior data support 

 pricing capabilities to let retailers set the best price as real-world conditions change.

Incorporating legacy back-end systems can be a challenge so many retailers are taking advantage of headless e-commerce and using APIs to build a seamlessly interconnected ecosystem for 

. It is much easier to build APIs that allow retailers to connect to all of their systems without worrying about syncing issues or migration. With headless e-commerce, retailers can easily provide an identity-aware customer experience across any channel with access to commerce, rewards, and promotion functions.

Inventory is the backbone of any retail org. Sales, profits, and customer satisfaction are all directly linked to effective and efficient inventory management. Retailers require global visibility — as well as the ability to support different local and regional assortments. Inventory is always in motion: Deliveries, pickups, and returns, shoppers in-store, and third-party retailers. Delayed response times or inventory issues can lead to discrepancies, bad customer experiences, and cart abandonment. Fun fact: ecommerce stores 

Traditional data systems already struggle to provide a real-time view of inventory, making it near impossible for retailers to meet customers’ demand for fast responses and consistent experiences across every channel. As retailers support more interactive online and in-store experiences, data speed becomes even more critical with the gamification of apps, personalized paths, virtual mirrors, and smart shelves.

The real-time edge can support retail inventory management with low latency across multiple geographies at significantly less cost than the cloud - without the hassle of maintaining servers. The edge also improves omnichannel experiences with faster performance and time to market.

to learn more about real-time inventory management at the edge.

. Retailers can build a real-time inventory view for in-store, in-warehouse, and fleet management. Data is replicated in milliseconds and reconciled with automatic conflict resolution. Local copies of inventory data are kept in any of the Macrometa GDN 175+ points-of- presence.

Data is close to the point of consumption - around the world - near stores and customer locations. Retailers can meet 

The Future Of Retail Depends On Real-Time Inventory Tracking

We're thrilled to announce that Macrometa is SOC 2 Type 2 certified.

The SOC 2 auditing and certification process looked at Macrometa's Security and Availability to ensure that the Macrometa 

 (GDN) is operating in a secure, controlled, and audited environment. To achieve SOC 2 certification, we demonstrated to the third-party auditors that the Macrometa GDN has these two security common criteria in place:

Systems and data stored by the company are protected against unauthorized access and unauthorized disclosure.

Information and systems are available for operation and use.

Organizations trust us with their most important data, and that responsibility is one we take seriously. We recognize that trust is earned by allowing third parties to objectively test the controls and procedures we've put in place. The team at 

 conducted the audit of Macrometa’s ISMS and controls on the SSAE-18 Trust Service Controls of Security and Availability over the course of several months. The certification enables our customers to comply with Information Security programs but also requires vendors to demonstrate adherence to industry-standard security certifications.

, just 10 short months ago today in fact, I saw that our Global Data Network had the all the right ingredients for developers to create next-gen technology. In a fully-distributed world that processes information in milliseconds, we have to instill confidence in our data policies and security. This certification is a huge milestone for Macrometa and the culmination of months of hard work and dedication toward the security of our platform.

The executive report is available upon request under NDA by contacting 

Macrometa Is Now SOC 2 Type 2 Certified

Some situations call for more complex search patterns. In this blog, we will cover range queries, faceted search, and geospatial search. Thank you for joining us again for the last blog in our search patterns series. If you haven’t seen the 

 and exact value matching examples. And if you are interested in prefix matching, full-text token, or phrase and proximity search, you may want to view the 

. We have covered seven different search patterns in the series to help application developers realize the full benefits of Macrometa search.

If you already downloaded the dataset from the 

 blog, you can skip this step. The dataset can be downloaded from this 

. After downloading the JSON file, replace <DATA> in Listing 1 with the content of the JSON file in the following command. The dataset can be imported to your GDN federation issuing the CURL command on a terminal as shown in the Listing 1. Before executing the following CURL command you need to first create a 

 named Hotels in your GDN federation and then create a document collection called hotel_reviews within that fabric.

The range query can be executed considering a set of numeric items. For example, the following query finds all the reviews which had ratings 3, 4, or 5.

Listing 10: Range search query which involves comparing strings

The execution of the above query results in a list of items (2037 items in total)¹ as follows.

Faceted search allows for combining aggregation with search queries to retrieve how frequently values occur overall. We need to first define a view using the identity analyzer as follows below.

The following query can be used to look up how many times a review carries the title "Very good". Note that here the case of the term "Very good" is exactly matched. Hence, although the title “very good” in all lowercase appears five times across this hotel review data set, only three reviews are listed as the result for the below query.

Geospatial search is a less frequently found but very useful feature when it comes to implementing geography-related information processing in general data analytics applications. Traditionally a special class of information systems called Geographic Information Systems (GIS) have been used to deal with spatial data extensively. However, 

 such as Macrometa GDN have the capability of storing and searching on spatial data which has become very useful when developing applications where geospatial search is required. Macrometa GDN supports geospatial queries such as finding coordinates and shapes within a radius or an area.

In order to try out geospatial search capabilities of Macrometa GDN we have selected a dataset from the Seattle metropolis in the USA. The 

 corresponds to public schools in Seattle. This dataset was obtained from the 

Listing 17: Get all the schools which are located within 1km distance from the Saint George Church

This should result in two schools named Cleveland STEM and Maple. These two schools along with the Saint George Church can be visualized on a map as shown below.

Visualization of geospatial query results

The ability to specify sophisticated search queries is very important for any data management system. It is because many of the search scenarios we encounter today demand the ability for conducting search queries at different levels of complexity and involve various types of datasets.

 explained how the search capabilities of the GDN have been implemented and showed an example of how to execute exact value matching. The 

 provided a step-by-step guide on how to set up prefix matching, full-text token, and phrase and proximity search on the Macrometa GDN using several real-world data sets. This last blog provided details on range queries, faceted search, and geospatial search. 

The search pattern examples also listed the details of the expected results so that you can verify the results when you try these queries in the 

¹The GUI displays up to 1000 records. Complete results can be found via Macrometa GDN's REST API.

Beyond Text Search: Exploring Advanced Search Capabilities With Macrometa

The search is not over. As we continue with the second blog in our search patterns series we will cover prefix matching, full-text token search, and phrase and proximity search. If you missed our 

, you may want to take a look to become more familiar with the basic 

If you already downloaded the dataset from the first blog, you can skip the steps in the first two listings

. The dataset can be downloaded from this 

Prefix matching can also be done considering multiple prefix terms. If you need to find all the reviews made for hotel names starting from either "The " or "Hotel " that can be accomplished using the following query. The results should indicate there are 4524 reviews¹ that satisfy these criteria.

When searching strings it is highly useful to search for tokens in full-text which can occur in any order. Text Analyzers tokenize the full-text strings so that each token can get indexed separately. There are two ways to search for tokens that are called token search and phrase search. While the former is described in this section the latter is presented in the section below.

This approach searches for token which can appear in any order. The words that are searched for have to be contained in the source string. First, a text analyzer view has to be defined via invoking a CURL command as follows below.

Phrase search allows you to search for phrases and nearby words in full text. You may also specify how many arbitrary tokens may occur between the defined tokens for word proximity searches. We can use the same search view defined in the previous section here as well.

Let's search for hotel review comments which say "rooms are small" and select the hotel names and their review ratings.

The PHRASE() function allows for specifying tokens and the number of wild card tokens in alternating order. This can be effectively utilized for two words with one arbitrary word in between the two words. For example, one could search for review comments specifying the number of nights the reviewer has stayed in the hotel as follows.

Listing 9: Identify whether reviewer has mentioned about the number of nights the reviewer has stayed in the hotel

Execution of the above query should result in 859 results.

Now that we covered prefix matching, full-text token search, and phrase and proximity search, you can get started based on these examples. Be sure to tune in later this month as we go over range queries, faceted search, and geospatial search.

Advanced Text Search With Macrometa

We are living in a world where the way people do business gets transformed into digital form. This phenomenon called digital transformation has resulted in propelling enormous amounts of data into digital repositories at unprecedented rates. The data can be of various shapes, sizes, qualities, and values. Indexing and searching capabilities can contribute to an organization’s success by allowing them to quickly and accurately locate specific information from large data repositories.

Search is often an underutilized feature by application developers. Typically developers use the term "search" to refer to the scenario of searching for a string. But the advanced search capabilities within the 

 enables sophisticated search. One of the goals of this blog series is to help application developers to utilize the full set of benefits provided by GDN Search. 

The first blog in this three-part series about seven different search patterns frequently found in the Macrometa GDN, we will introduce Macrometa 

 capabilities and go over exact value matching. The second blog will cover details about prefix matching, full-text token search, and phrase and proximity search. Lastly, the series will conclude with range queries, faceted search, and geospatial search. Let’s begin by providing a detailed classification and then walk through the first example.

The Macrometa GDN is a multi-model, streaming 

 data processing capabilities. When it comes to search, GDN brings two main advantages over other data stores.

First, the GDN is equipped with a state-of-the-art indexing and 

 facility that enables users to perform sophisticated search operations on 

 pairs, documents, and graphs. Users are not required to reformat the data and upload them to a separate system. Instead, the persisted data on GDN can be directly used for search as well. The complexity of the search can vary — from simple search queries such as exact value matching and range queries — up to complex search operations such as faceted search, and geospatial search. Each of these different categories can be classified as a search pattern.

Secondly, with the Macrometa GDN the search indexes are updated globally whenever their underlying data gets updated. This allows the GDN to answer search queries within a very short period of time with high accuracy.

Exploring the dataset sample for search patterns

The example in this blog uses the London-based hotel reviews 

. The refined dataset has 10,000 reviews collected by crawling a leading travel portal. Each of the data items in the dataset follows a schema shown in Figure 1.

. After downloading the JSON file, replace <DATA> in Listing 1 with the content of the JSON file in the following command. The dataset can be imported to your GDN federation issuing the CURL command on a terminal as shown in the Listing 1. Before executing the following CURL command you need to first create a 

Listing 2: Importing a single data item into the GDN federation via invoking the REST API of a GDN node

In the above example in Listing 2 we have imported only a single review made for the hotel named The Savoy by specifying its JSON content. The values <HOST> and <BEARER_TOKEN> refers to the host name of the GDN node and the bearer token can be copied by referring to the REST API of the GDN node.

The overall architecture of GDN Search can be summarized as follows (See Figure 2). The search is based on the concept of views which can be regarded as virtual collections. A view consists of an inverted index which provides fast full-text searching over one or multiple linked collections. The view also stores the configuration information for implementing the search capabilities such as the attributes to index. The view can use multiple, if not all attributes of the documents in the linked collections. Views can be created and maintained via the Web user interface, HTTP REST API, or via the Javascript API of GDN. All the examples listed in this blog series are based on the former two techniques.

Views can be queried using C8QL via its SEARCH operation. Search expressions are composed of the fields to search, the search terms, comparison and logical operators, as well as other GDN Search functions.

Figure 2: Overall architecture of GDN Search

The input values are processed by constructs called Analyzers. These can normalize strings and tokenize text into words, which enables various possibilities to search for values later on.

The search output can be sorted either by their similarity ranking to return the best matches, using popular scoring algorithms (TF-IDF, Okapi BM25), or by using the user-defined relevance boosting and dynamic score calculation.

In the most basic version of a search pattern you could try to match the presence of an exact value. The exact value can be either strings, numbers, number ranges, or booleans. Here we can index and search strings using an identity analyzer. The view used for exact value matching can be defined as shown in Listing 3.

It should be noted that since the default Analyzer is the identity it is not necessary to set the Analyzer context with the ANALYZER() function here. Therefore, exactly the same results can be obtained by executing the query shown in Listing 5.

Exact value matching can be conducted considering several item values. There are three approaches to do this — either using the logical OR operator, using the IN operator, and using the bind parameters.

Now that you are familiar with the Macrometa GDN search architecture, you can get started on exact value matching based on this example. View the 

 in our search patterns series to learn about prefix matching, full-text token search, and phrase and proximity search.

Expand Your Search Capabilities With The Macrometa Global Data Network  

While companies today want to collaborate throughout their organization and work closely with business partners, they might need to limit access to documents in a collection to meet 

, privacy, and quality control requirements. 

 is a fundamental component to help protect confidential information and minimize risk to your business or organization. In this blog, we go over the details of the User Access Control feature and how to set up who can view or modify data at the document-level and collection-level.

There are multiple ways admins can apply User Access Control attributes to their 

. These attributes can ‌limit access to documents at run time in 

Govern access by both users and API keys.

Document-level and collection-level access control with attributes

Let's use an example to illustrate how you can use attributes to implement document-level access control and collection-level access control. We have a fulfillment company that is made up of partners and employees. Partners have items they want the fulfillment company to fill, and the employees fulfill the orders of their partner's items.

For these examples, we have three collections:

Items: The items that the partners want the fulfillment company to ship.

Orders: The orders that the fulfillment company has to fill.

Employees: List of all the fulfillment employees.

And the fulfillment company wants to provide some queries that their partners and employees can use:

Partners can only see their items, but can't update them.

Partners can see their orders and update orders when filled.

Store employees can see all items and orders, but can’t update them.

Every partner has a unique API key. This key is used to authenticate them and restrict access to their information. Employees use their credentials for authentication and access control.

Partner #1: ACME Company -- API Key: acme_key.897b6e...

Partner #2: General Hardware Store - API Key: generalhw.28bc97…

Each API Key is being assigned a partner attribute:

In this section, we are creating four Query Workers to implement the queries mentioned above.

Partners can only see their items, but can't update them

User and API attributes are powerful tools to help with access control decisions on queries. Whether you are trying to limit documents or collections – or limit the viewing or modification of data – user and API attributes can address your requirements. After viewing these examples, you can quickly get started and apply attributes within your own GDN — based on your access control requirements. If you are new to Macrometa and would like to try out User Access Control or other features, 

 and see everything you can do in one platform.

Four Ways To Establish User Access Control With Macrometa

BAM! Simplifying Complex Event Processing

, we took a look at the high-level architecture of our log analytics dashboard example application that we developed with our partner 

. In this article, we look at the Stream Workers that are used for 

Macrometa provides developers with a simple way of performing 

 using Stream Workers. Macrometa Stream Workers gives you “superpowers” that allow you to transform data from one data type to another, enrich data by combining it with other databases or data services, aggregate streams, clean and filter data, and more.

 are written in Macrometa Stream QL. This domain-specific language combines the simple query language of SQL with declarative stream, source, and sink definitions. It also supports authoring arbitrary helper functions in Javascript.

 of the Stream Workers code in the Fastly log analytics demo application. We go over code patterns used in these Stream Worker examples, and you can view our documentation for more source code details.

In our Fastly log processing application, we use three stream workers. One sends HTTP requests to the Fastly service every three seconds to generate logs to analyze and the other two are responsible for reading and aggregating those Fastly web logs. For more information about the architecture of this log processing application, see the 

This Stream Worker uses trigger to make three HTTP requests to the Fastly service every three seconds.

Annotation to name the stream worker application:

 that runs the stream worker every three seconds. The stream worker will make three HTTP requests to the Fastly service to populate the Macrometa

FastlyGetLogGenerator, FastlyPostLogGenerator, FastlyPutLogGenerator, and FastlyInitCollectionGenerator streams.

The following stanza creates the stream definition for FastlyGetLogGenerator. There is a 

 for the stream, created using a sink annotation indicating that this is an HTTP sink along with a map annotation that defines the sink output format, in this case JSON. A sink, in the context of stream processing, is used to pass transformed data downstream for storage or additional processing.

Additional streams are defined for other HTTP methods for which we want to generate Fastly logs.

The Stream QL statement below retrieves a URL to contact from the FastLogGeneratorTrigger collection and inserts it into the FastlyGetLogGenerator stream (explained above). This will cause the FastlyGetLogGenerator stream to connect to the provided URL using the HTTP GET method.

Stream Worker fastly-http-request-worker (

This Stream Worker reads fastly logs from `logs` collection and performs reordering for out-of-order logs. Also, based on log timestamp, it determines the 10 second window and adds that window timestamp as an additional field.

Stream Worker fastly-http-request-stats-1m-worker (

This Stream Worker reads logs from `fastly-intermediate-stream` and performs aggregation within a 10-second window. It performs aggregation for response_status, url, response_body_size, latency.

 writes additional aggregates and summary data to Macrometa collections. Having read this article, the rest of the code should be straightforward to understand.

The Fastly log analytics application only scratches the surface of what can be accomplished with Macrometa Stream Workers. A rich selection of functions are available in Stream QL to facilitate Complex Event Processing.

. If you are new to Macrometa and would like to build your own superhero adventure, request a 

, and see everything you can do in one platform.

In our next blog in the series, we will be covering the use of Macrometa Query Workers in the log analytics application.

Using Macrometa Stream Workers In A Real-Time Log Analytics Dashboard

The code for this trading bot is open source. You can access it here: 

https://github.com/Macrometacorp/tutorial-cryptotrading

If you're going to try this project or customize it for your own strategy, you're going to need a Macrometa account. Take a minute to sign up for a free account by clicking on the button below.

I've been fortunate to have had a professional life that has exposed me to many experiences. I've not just tried my hand at software architecture and engineering, but also have been a full time professional quantitative trader for many years. The majority of my trading was with Index and Currency futures on Chicago Mercantile Exchange (CME), as well as trading US equities.

The general excitement surrounding cryptocurrencies piqued my interest. There are ~267 cryptocurrency trading exchanges spread around the world with probably around 8-10 exchanges with enough liquidity and reputation that one might consider working with them. Given the number of exchanges and volatility of cryptocurrencies, I wanted to see if it is feasible to build a trading bot that can trade on various exchanges doing something like exchange arbitrage.

An exchange arbitrage is basically a trading strategy based on the differences between the price of the cryptocurrency at different exchanges. Generally these opportunities open up if there is a price discrepancy and the discrepancy accumulates over time to finally become a significant amount like 1–3% at the cryptocurrency market. These arbitrage opportunities could exist for merely seconds, so an arbitrage trader would have to search for the best opportunities constantly and then implement them whenever possible.

Implementing arbitrage strategies in the crypto market is not that easy though, and involves a number of technical complexities. Besides having deposits in multiple major exchanges, on the technical side it involves the ability to monitor arbitrage opportunities simultaneously, making trades locally at each exchange with minimum delay and timely visibility & interactions between the trading agents distributed globally. The challenge requires big dollars and a team with the necessary technical expertise to build this type of distributed infrastructure.

Real-time crypto price data with Macrometa

 that offers a decentralized, distributed database, 

 engine and compute platform that runs across 175 global edge regions. Apps, APIs and web services written on Macrometa are automatically deployed globally and handle requests from the closest edge location relative to the request with local read-write latencies (i.e. very low latencies).

Macrometa offers a range of ready-made capabilities that enable an application like a crypto trading bot to be built quickly and easily. We will use the following features of Macrometa in this tutorial:


, decentralized and geo distributed developer platform that provides a database, a stream engine and stream processor and a function/container runtime.

 for storing trade data and sharing all trade related metadata and state between the decentralized, stateless bots and components

We will use geo distributed streams for publishing real-time price data from each exchange. Each bot subscribes to the stream and makes a decision on whether to buy or sell

Finally we will use real-time updates from the global database to automatically notify the clients/bots/subscribers of changes to the database so they can modify their trading strategy

So with the features listed above, I set out to see if Macrometa can be used to build a crypto trading bot to trade locally in multiple exchanges while providing global visibility & communication.

Archive