Blockchain.com Explorer - Worlds most popular Bitcoin Explorer
Christian Tucker
Building the Blockchain.com
This project was probably one of my favorites. I was tasked with leading the development efforts to completely rebuild the popular Bitcoin explorer for Blockchain.comNextJS, ReactJS, NodeJS, and styled-components while tasked with utilizing data from several abandoned APIs I was also forced to familiarize myself with many services using languages such as JavaScript, Java, and Kotlin. Referencing some APIs that did not have documentation also meant inspecting mobile clients that were written in Swift (iOS) and Kotlin (Android) as well.
Combating the drawbacks of Server Side Rendering at scale
While building this project one of the requirements was the utilization of Server Side Rendering (SSR). Considering this meant utilizing one of my favorite frameworks, NextJS, I was more than happy to lead the charge in this regard. There were a few areas that we were not prepared for, but it was a lesson that we will employ for future projects moving forward.
NextJS does not properly handle scaling of parallel requests to resources that are scheduled for SSR..
This was a huge drawback that we noticed at release, which caused our service to utilize a much higher load than initially expected. The cause? NextJS. I couldn't believe it when we determined that the reason for our problems was that the framework was not correctly deferring processing of similar requests to the rendering pipeline. This meant that if we had 1,500 requests for the same resource that was not available in the cache, it would spawn 1,500 different rendering events for the same resource. This was unacceptable, but a solution was simple to engineer. In short, whenever a request to an identical resource is made, we don't send that request to NextJS for processing, instead we simply put it into a wait list. Once NextJS completed the SSR action and responded to the initial request, we simply forwarded that request to all clients in the wait list. This improved scalability, response, and speed of the Explorer by magnitudes of 1xxx% percent.
Working with inefficiently structured data..
Many engineers will immediately understand what I mean here, but for the clients that aren't aware, the structure of data and where we as engineers can obtain that data is extremely important. It can be the difference between a quick and snappy website, or several loading screens and waiting for actions to complete. In an ideal world, all data that you need to display certain components should be available from a single source. If you have to pull data from several different locations you start to build request overhead, especially if some of this data requires the earlier data to already be present as now you're creating a queue of requests, which linearly increases the delay experienced by end-users. This was one of the many challenges we had during creating this application, so we utilized something that we call a "proxy service". This service was in-charge of caching data from numerous sources and pre-building responses for the website. This meant that we were able to utilize internal bandwidth and network channels to communicate with each service and only provide the end-user (client) what was needed to render the page. This showed a large increase in performance and lowered our external bandwidth usage substantially, which resulted in not only a better user-experience, but saving the business money as well.
