This content originally appeared on Level Up Coding - Medium and was authored by Lennant
In this article, I will present a way for you to do full text search using no extra service. You will avoid extra costs while enabling your users to do real time partial matching searches with some minor limitations. Live demo at the end.
Often times, you will want to have your users be able to search your data. The recommended method from firebase is to use an external service like Algolia. The problem with this solution is that its extra cost and maintenance that you have to deal with.
Pros
- No external search service needed.
- No synching of data required.
- Can paginate searches using firebase standards.
- Works with any data format you throw at it.
- Support for partial searches mid string.
- Can be combined with other queries to build advanced filtering.
- Super fast
Cons/Limitations
Let's go ahead and get the limitations out of the way so that you can see if this solution fits your needs.
- All queries are returned in the default order. You can’t “sort by price” for example. Potential anti pattern can be used to return search results in a desired order only if you care about ordering.
- User must type at least 3 characters for search to register.
- The total searchable string must not exceed a fairly reasonable limit (suggest 500, but use your best judgement). Meaning you won’t be able to have a full essay and search inside of it. Its meant more for titles, descriptions, etc…
- Search term must be continuous.
Example, “ere no” would match “Is there no one else”, however searching for “there one” would not return hits with this implementation since the words are split. - Egress cost is slightly more depending on how large a string you allow searching on.
Add Record Implementation
Here is an implementation of creating a search enhanced record using trigrams.
The trigram command is the magic here. Running this command yields the following :
triGram('Hello World')
['hel', 'ell', 'llo', 'lo ', 'o w', ' wo', 'wor', 'orl', 'rld']Every record you create stores the trigram result as an object. To do a search query, you run the trigram against your search string, then ensure that all trigrams from your search string are present in the array above. For example, for the search string of worl the following trigram is generated :
triGram('worl') => ['wor', 'orl']since wor and orlare found in the above trigram map, we assume that this string search string matches. This matching behavior can also lead to false positives however.
For example, if your data contained Work Orlando the worl would also match when that’s not what you’re looking for. Typically users will just refine their search so this doesn’t bother me at all.
How To Do Searches
Now that you have records marked up with the search meta, you probably want to run your first query. I won’t go over pagination in this article since it would be too long. I’ll follow that up in a future article on doing generic pagination of data. For a basic implementation you can play around with, visit this website :
https://demos.vizure.net/app/demos/search
The idea here is very simple though. We create a trigram of the current search term, add it as where clauses in our firestore query, and get the results.
FAQ
- Why not use arrays to store your trigrams instead of objects for trigrams?
array-containswill only look for one trigram in the array. There is no way currently to query firestore for records that contains many values specified in an array.
array-contains also has limitations that make it not ideal for this approach. - Why can’t I search against indexed data?
Once you create an index (like sort by date for example), if you want to have other where clauses on that index, you have to create a new index for each where clause. So essentially every search string the user enters now requests you to generate an index for each trigram. This is not a valid workflow. - Whats with the search length limitations?
Indexing in firebase has limits believe it or not. You also don’t want to make your records too large so that downloading chews up too much bandwidth. - What's with the anti pattern for ordered data?
To support retrieving data in date order, we have to structure object keys so that they are returned in newest first order. This is actually against the best practices of firestore key names.
The side effect of this is you’re most likely limiting your read/write speed on the document. So you may start to experience slower search results for example, but I can’t quantify by how much. For my purposes, its probably ok.
Conclusion
I’ve presented here a very easy solution that you can add to almost any project with very little overhead. It gives your users the ability to search any of your databases, without having the worries of tacking on a search service on top of your firestore database.
Firestore Full Text Search At No Extra Cost was originally published in Level Up Coding on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Level Up Coding - Medium and was authored by Lennant
Lennant | Sciencx (2022-03-13T01:54:42+00:00) Firestore Full Text Search At No Extra Cost. Retrieved from https://www.scien.cx/2022/03/13/firestore-full-text-search-at-no-extra-cost/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.