Advanced Text Search With Macrometa

curl --location --request POST 'https://api-<HOST>/_fabric/Hotels/_api/import/hotel_reviews' \
--header 'accept: application/json' \
--header 'Authorization: <BEARER_TOKEN>' \
--header 'Content-Type: text/plain' \
--data-raw '{
  "data": <DATA>,
  "details": false,
  "primaryKey": "",
  "replace": false
}'

Listing 1: How to copy the content from the JSON file into the CURL command

curl --location --request POST 'https://api-<HOST>/_fabric/Hotels/_api/import/hotel_reviews' \
--header 'accept: application/json' \
--header 'Authorization: <BEARER_TOKEN>' \
--header 'Content-Type: text/plain' \
--data-raw '{
  "data": [{
    "Property Name": "The Savoy",
    "Review Rating": 5,
    "Review Title": "a legend",
    "Review Text": "We stayed in May during a short family vacation. Location is perfect to explore all the London sights. Service and facilities are impeccable. The hotel staff was very nicely taking care of our kids. We'll be back for sure!",
    "Location Of The Reviewer": "Oslo, Norway",
    "Date Of Review": "6\/28\/2018"
}],
  "details": false,
  "primaryKey": "",
  "replace": false
}'

Listing 2: Importing the sample dataset to GDN federation via invoking the REST API of a GDN node

In the above example in Listing 2 we have imported only a single review made for the hotel named The Savoy by specifying its JSON content. The values <HOST> and <BEARER_TOKEN> refers to the host name of the GDN node and the bearer token can be copied by referring to the REST API of the GDN node.

Implementing prefix matching

Many search scenarios can be found where you are interested in knowing all the strings which start with a particular prefix. Finding the longest matching prefix from a collection of keywords is a long-studied problem with multiple applications such as dictionary searches, computational geometry, Internet packet routing, and DNA Sequencing. Searching for strings or tokens which start with one or more substrings is accomplished via the prefix search facility of the Macrometa GDN.

Prefix matching can be performed at different levels of complexity. If you need to apply exact prefix matching then indexing strings with the identity analyzer is adequate. Let's begin by finding all the hotel names which start with the term "The". The corresponding search query can be written as shown in Listing 3.

FOR review IN sample1_view1
  SEARCH ANALYZER(STARTS_WITH(review.Property_Name, "The "), "identity")
  RETURN review.Property_Name

Listing 3: Find all the hotel names which starts with the term "The "

This should result in a list of 3963 records¹ as shown below.

FOR review IN sample1_view1
  SEARCH ANALYZER(STARTS_WITH(review.Property_Name, "The ") OR STARTS_WITH(review.Property_Name, "Hotel "), "identity")
  RETURN review.Property_Name

Listing 4: Find all the reviews having the hotel names which start with either the term "The " or "Hotel "

The following example in Listing 5 shows how prefix matching is conducted on multiple attributes. In this scenario, we are interested in finding hotel names that start with "The " and the review titles that start with "Awesome ".

FOR review IN sample1_view1
  SEARCH ANALYZER(STARTS_WITH(review.Property_Name, "The ") AND STARTS_WITH(review.`Review Title`, "Awesome "), "identity")
  RETURN {
  Property_Name : review.Property_Name,
  `Review Title` : review.`Review Title`
  }

Listing 5: Find all the reviews having the hotel name start with "The " and review text start with "Awesome "

This should result in the following three reviews.

curl --location --request POST 'https://api-<HOST>/_fabric/Hotels/_api/search/view' \
--header 'Content-Type: application/json' \
--header 'Accept: application/json' \
--header 'Authorization: <BEARER_TOKEN>' \
--data-raw '{
    "name": "sample1_view8",
  "links": {
    "hotel_reviews": {
                    "analyzers": [],
      "fields": {
        "Review Text": {
          "analyzers": [
            "text_en"
          ]
        }
      }
    }
  },
  "type": "search"
}
'

Listing 6: Defining a view for token search using the text Analyzer

Once the view is ready we can specify a token search query which searches for the occurrence of at least one of the praising words — Awesome or Excellent or Lovely — in a review text and select its review score as the result as shown in Listing 7.

FOR review IN sample1_view8
  SEARCH ANALYZER(review.`Review Text` IN TOKENS("Awesome Excellent Lovely", "text_en"), "text_en")
  RETURN review.`Review Rating`

Listing 7: Find how relevant the keywords of praising to the review rating of a hotel review

When executed this should list 3803 review ratings¹ as the results as shown below.

FOR review IN sample1_view8
  SEARCH ANALYZER(PHRASE(review.`Review Text`, "rooms are small"), "text_en")
  RETURN {
  Property_Name: review.Property_Name,
  `Review Rating`: review.`Review Rating`
  }

Listing 8: Find the review comments which say "rooms are small"

This should result in 75 review comments as shown below.

FOR review IN sample1_view8
  SEARCH ANALYZER(PHRASE(review.`Review Text`, "for", 1, "nights"), "text_en")
  RETURN {
  Property_Name: review.Property_Name,
  `Review Rating`: review.`Review Rating`
  }