Algolia & BuddyPress | Launching Super-Charged Social Networks

Introductions

It’s impossible to launch into a technical case study about Algolia and BuddyPress without first talking about some of our favorite people. When our friends over at Mission Lab came to us to partner with them on an innovative new adventure, we couldn’t resist. I’ve known Tanner, the founder of Mission Lab, for a number of years now – dating back to before our BeachPress days! We’ve always had a genuine admiration for each other, the work we do, how we run our businesses and the way we lead our teams. With all of that mutual admiration, somehow, we never had the opportunity to partner together on a project! That changed this year.

Sales Hacker is a cutting-edge community of sales professionals. The Sales Hacker community has found a way to combine brilliance, sales savvy, and a genuine care for each other. It all seems totally counter-cultural to the competitive world of sales most of us imagine. They approached Mission Lab late last year to imagine what a completely new site could look like. The goal was to envision a site built around the vibrant community they already had – one of their greatest assets. The problem was that the previous site did nothing to showcase or engage the very people who make Sales Hacker. Creating sites that build community is where Mission Lab shines. They have years of experience with BuddyPress and building complex web applications using BuddyPress. Having the opportunity to bring our Algolia expertise to pair with their community platform development knowledge was an absolute pleasure.

The cherry on top of the entire project was having the opportunity to work with Shay Bocks. The only thing better than creating one of the fastest social networks on the planet was creating one of the most beautiful social networks. That is all Shay’s incredible talent on display.

Knowing the power of Algolia – the search indexing, faceting and filtering, advanced A/B testing and so much more – we set out to build something completely unique. We wanted Algolia to not only power the search experience, but the entire social network. Crazy? Probably. Here’s how we did it.

If you’ve never heard of Algolia before – imagine a tool that makes your website search just as relevant as Google…but faster. Beyond that, imagine that you could build and customize interfaces for searching your website that would rival the like of AirBnb and Amazon. Finally, imagine that you could use it for free (for up to 10,000 searches/month). You can imagine why we’re such huge fans.

Unfortunately, Algolia stopped supporting their WordPress plugin a number of years ago. But just as quick as their plugin died, a derivative version was born, like a phoenix rising from the ashes! Our good friends over at WebDevStudios took on the valiant task of maintaining and adding on to what Algolia let go, and they’ve done a brilliant job so far. Using this plugin as a base for much of the search functionality, we were then able to hook into the Algolia indexing for further customizations. As a technical note, we did find some of the libraries to be fairly outdated. But from the looks of it, there is good work being done to bring some of those up to date.

Once we had this plugin installed and configured – we were off to the races, getting BuddyPress to be powered by Algolia.

One caveat that seems very much worth mentioning: no two social networks are the same. One thing we quickly learned on this project is that every implementation detail is a decision. You may find that your mileage varies if you try and implement this code for your own projects, if you only because your project requirements are different. For the scope of this post, we’ll focus primarily on what it takes to index activities and groups in BuddyPress. There is a lot more to BuddyPress than activities and groups, of course. But for the scope of this post, those are the most important entities to understand.

Imagine you’re on Twitter, reading your favorite rage-tweeters and their rage-tweets. The two objects you’re interfacing with are a User (rage-tweeter) and an Activity (rage-tweet). Or perhaps you’re on Facebook and the kid who sat behind you in geometry just invited you to their fifteenth Pampered Chef party. You post in the “party” that you are utterly disinterested and proceed to leave the group. Before you rage-tweet about that lovely adventure – just know that you’re now familiar with Users (geometry guy/rage tweeter/you), Groups (Pampered Chef party) and Activities (rage-tweets/passive aggressive but fully warranted notices of disinterest in yet-another-MLM scheme).

Thankfully, the Algolia plugin does a lovely job indexing Users, with very little finagling required. The inherent complexity really lies with the indexing of the activities and groups. But before we can get into the technical details of how we accomplished that, we need to dig to a few key Algolia concepts.

A key difference between your standard WordPress database and Algolia is that Algolia is an index. That means it handles data completely differently (notably, it is not a relational database). This paradigm shift means understanding some important concepts will help you as you might be building out your own solutions.

First, indexes or indices. You will see these referred to throughout code samples. Sometimes, you might see a “primary index”. Indexes are simply a collection of stored entities. You can think of them as being like a row in your WordPress posts table. But unlike that row, this entity might have all of your relevant metadata and taxonomy data as part of that very entity. You can start to imagine where we can see some significant performance gains.

Next, watchers. Watchers are a part of the Algolia plugin code that we have to extend. The index tells us what is getting stored, and the watcher tells us when to store it. It even tells us when to update or delete it completely.

Finally, replicas. Again, coming from a database paradigm, replicas can feel tricky. If I want to sort by date, in ascending order, that’s super simple in WordPress. In Algolia, however, it’s not so simple. Algolia is an index, it optimizes for relevance and performance. So something like a simple date sort actually requires an entire replica index. A replica index in Algolia is exactly what it sounds like – a reproduction of the main index. The only difference is that Algolia configures the index to sort by a different metric. The beauty of Algolia is that all replicas are automatically kept in sync to the main index.

How can this play out practically? Have you ever tried to search or sort by post meta in WooCommerce? Maybe your server timed out when sorting by an ACF field with 2,000,000 rows? Those same searches (or filters or sorts) against Algolia would be measured in milliseconds.

Finally, we get to the the good stuff. The technical implementation was no small task. To cover all the finer points here would be impossible. Here’s what we’ll cover in this post:

Algolia Bootstrap – The initial integration point with Algolia.

Group Index – Our custom index for BuddyPress Groups

Group Watcher – Our custom watcher for the BuddyPress Groups Index

Activity Index – Our custom index for BuddyPress Activities

Activity Watcher – Our custom watcher for the BuddyPress Activities Index

Our bootstrap file is where we initially hook into the Algolia plugin. Here, we create our watchers, indexes and replicas. Additionally, we can modify any other entity attributes we might need to alter before they are sent to Algolia. Let’s dive into some code, then we can walk through some of the finer points.

How you’d execute this file is entirely up to you – lots of ways to do that. If you made it this far, you no doubt have your favorite way you’ll already be doing that. Singleton instance, calling a constructor, a class registry – pick your favorite and execute the code. Let’s go through the important code.

You’ll likely make it past the init bootstrap and notice something important. We’re hooking into filters from the Algolia plugin for our previously defined terms: indices, watchers, and replicas. Our add_indices callback is fairly simple – if BuddyPress components for either groups or activities is active, we add an object instance of those index classes to the array that we are filtering.

The add_watchers routine is slightly more complex. We have to iterate through all the indexes, detect which ones to target with the contains_only method, and add an instance of that watcher to the array.

Finally, our add_shared_attributes and add_replicas callbacks do some heavy lifting. In very few lines of code, we tell Algolia that we want to store a favorites attribute on our indices and that we want to create a replica index to sort by that attribute with (in both ascending and descending orders). We do this utilizing the Algolia_Index_Replica class, which is part of the Algolia plugin.

At this point, we’re fully bootstrapped in and customizing Algolia. The next important step is to create our first custom index. We’ll start with Groups.

There’s a lot to unpack here 🤯. The initial thing you may notice is that we’re extending the Algolia_Index class, found in the Algolia plugin. We set out to learn the best ways to extend it by first looking at internal examples of indexes for core object types. The core class covers much more than this abbreviated example, but for our purposes here today, we can understand the class in the following sections:

The initial key components of the class center around identifying the class. We do this by setting the contains_only property. This is used in a number of areas in the base class and throughout the core plugin to check what index is in use. You also see us using the base class method, contains_only(), in our bootstrap. Next, we set our admin name to a human-readable name in get_admin_name() and set our get_id() method as well.

When it comes to actually indexing the groups, there are a few vital methods we need to implement.

should_index() – This method is checked prior to any synchronization of records between WordPress and Algolia. This includes the initial indexing of records, but also any updates to them as well.

supports() – Similar to should_index(), supports() allows you to confirm whether or not this particular item supports indexation. This is checked prior to record deletion and used within watchers.

get_records() – This method transforms the output of an individual entity in WordPress to a suitable entity record for Algolia. Important to note, the output of this function should be an array containing the entity. That entity must contain a unique objectID parameter.

delete_item() – Called from the watcher, this function synchronizes the deletion of an entity in WordPress with the deletion in Algolia.

get_items() – Note the important parameters passed for the function, $page and $batch_size. When indexing, the Algolia plugin will intelligently handle batching large amounts of data, so you don’t have to worry about timeouts. Be sure to pass these along to any functions you use internally to retrieve data.

get_re_index_items_count() – Returns total number of entities locally. For the Users index, it’s the total number of site users. For posts, it’s all your site posts.

get_settings() – Set your base settings for this Algolia index. These settings are incredibly powerful, and worth spending a day researching. At the very least, take a look at how the plugin handles the settings for some of the core entities.

get_synonyms() – You won’t see this used in the core plugin, but as an abstract method, it is required in your class. It can actually be a powerful tool for defining alternate usages for terms found in your searchable attributes. An array of arrays is the expected output, with unique object IDs within each array. See the documentation for more information.

Now that you know how to create the index – you could technically stop here. If you somehow have content that never changes – never gets updated, deleted, or has new entries created – you could run the index once and be done! But if your application lives in the real world, that’s probably not the case for you. You’ll need a Watcher class that watches for specific events and modifies the Algolia index accordingly.

Unlike the index class, we’re not extending an abstract class. Rather, we are implementing a simple interface. The only method this interface expects us to implement is the watch() method. Additionally, we need to pass the related index through to the constructor, making it available on the index property throughout the class.

Within the watch() method, you can see that we’re watching the group actions, as well as the group meta actions, for any updates. After checking the validity and type of data in each method – we’re essentially calling the sync() method of the index whenever we’re managing updates, an calling the delete_item() method of the index whenever we are deleting an item.

Now that understand foundational concepts in Algolia and BuddyPress, and given basic examples of implementing Watchers and Indexes, we can get to the grand finale! In any social network – it’s great to have users, and helpful to have groups – bit if you don’t have your activities, you don’t have a social network! After all, what is Twitter without tweets? Facebook without conspiracy theory posts? LinkedIn without job anniversary congratulations?

If you’ve worked with BuddyPress for any amount of time, you know that activities can get complicated! Not only are there nearly a dozen types of activities, all of which may need special treatment when indexing, you may be dealing with an untold number of custom activity types as well! Needless to say, this example has been severely abbreviated in order to clarify what might apply more broadly, rather than the specific implementation details for this client project.

Wrapping up our implementation, we’ve created a custom Activity watcher. When considering how to integrate Algolia and BuddyPress, we found it helpful to first look at the cache incrementor hooks for each type. This gave us a great starting point for Algolia indexing and finding the right hooks to watch.

You’ll notice our delete_item() method looks a little bit different in activities than it did in groups. You may also find that you need to tweak how your methods are implemented based on the variables available in each hook that you implement the watcher on.

It’s all the great things about LinkedIn without all the bad things; especially for sales people.

You made it through 3,500 words! As a reward for the three of you that made it this far, check out this video of Algolia in production on the Sales Hacker site. You’ll see the result of thousands of lines of custom code, integrating custom autocomplete templates, using the powerful InstantSearch libraries, custom watchers, indexes and more.

Notice how everything – the searching, filtering, sorting, loading more results – is all instantaneous? That’s the power of Algolia ♥️.

None of this would have been possible without the combined power of technology like Algolia, open source projects like BuddyPress and the Algolia plugin from WebDevStudios, and the expert partnership of the good folks at Mission Lab and Sales Hacker. We’re unbelievably thankful for all of the moving pieces and people that created such an incredible experience.

Want to work with us to create your next super-charged user experience? Get in touch today.