Revenue Management Technology Leader Chargebee Reduces Data Request Resolution Time by 90% with Atlan
Founded in 2011, and since growing to support over 4,500 customers, Chargebee is a market-leading technology solution for recurring revenue management. “We power the entire recurring revenue life cycle, from subscription billing to invoicing, to cash revenue recognition, receivables, retention, and a lot more,” shared Lavanya Gopinath, Chargebee’s Senior Director of Culture & Systems.
Enabling revenue management on a global scale demands careful attention, a sophisticated architecture, and oceans of data. Chargebee supports more than 100 currencies across 53 countries, integrates with 55 revenue technologies, and maintains more than 30 payment gateways.
Underpinning this operation and architecture is Chargebee’s data team. “We take care of all internal reporting and data needs across the organization,” shared Lloyd Lamington, Business Solutions Manager. “All of us work together to drive the data culture at Chargebee.”
As data teams, we’ve painted ourselves into a corner. On one hand, no data team wants to be a help desk or dashboard factory, resolving Jira requests for data pulls or cranking out ghosted dashboards. On the other hand, as much as we might resent it, this is some of the most important work we do. Optimistically, we’re victims of our boring successes; cynically, our egos are bigger than our abilities.
Lloyd Lamington, Business Solutions Manager
Falling into the Data-as-a-service Trap
In early 2021, Chargebee’s growth accelerated significantly, with a commensurate increase in requests for data. Chargebee’s Data Engineering team was responsible for processing these requests, both from internal colleagues and customers. “This was a huge challenge,” shared Lloyd. “Internal data requests were pushed to the back of the queues as customers were always a priority. This meant that we were not meeting SLAs and there was unhappiness among our stakeholders and lots of escalation, which led to unhappiness within the team as well.”
To meet this increasing volume of requests, the Chargebee team first turned to hiring new colleagues, but found that the transactional nature of their Data Engineering function made hiring difficult. “It was a challenge for us to hire for these roles. People did not want to service just data requests all day long, as they did not find it as interesting as other roles in data engineering,” Lloyd shared.
Chargebee then turned to automation and standardization, creating dashboards and workflows to respond to repetitive requests. While helpful, these requests were often too bespoke to service with a single view of data. “What people wanted was to look at the underlying data at a very granular level, and they always wanted options to export to Excel, which again is another major pain that all of us who have been in the BI industry or in the data industry can relate to,” Lloyd explained.
While Chargebee’s data team kept pressing for a solution, data request volumes continued to grow. Data Engineering was receiving 350 requests per quarter, 80 of which were repetitive requests. 70% of requests were for raw data. Struggling to meet their SLAs, and with growing escalations to subject matter experts, Chargebee had to find a new way to meet their colleagues’ and customers’ expectations.
We were falling into this trap of data as a service where we were always on a reactive mode rather than a proactive mode. A huge chunk of our time went into servicing all these data requests and getting requirements and building products instead of proactively going about creating data products that people could consume.
Lloyd Lamington, Business Solutions Manager
Chargebee began an evaluation of the Data & Analytics software market, beginning with customer data platforms like Segment, and exploring the capabilities of existing tools like BigQuery. Focusing on self-service as a potential solution, the team discovered the Active Metadata Management and third-gen data catalog market, and began evaluating Atlan.
“We were happy with the features that Atlan had to offer us. So while self-service was not just the only problem it was solving, it also helped us set up the data catalog and the metric glossary as well,” Lloyd shared.
In time, Atlan would prove to be the missing piece for Chargebee; a layer of truth and collaboration atop their growing data estate, and a way for Data Engineering to finally break their backlog of requests. “Where we started out a couple of years ago, a lot of spreadsheets, some of them converted into dashboards, endless requests for raw data,” Lavanya shared, “We’ve come a long way. And Atlan is integral to this data experience that we’ve created.”
Getting Started the Right Way
After choosing to purchase Atlan, the Chargebee team got to work researching the nature of data requests to ensure they would yield value from the platform as soon as possible. “We analyzed tickets data from two previous quarters to understand who our most frequent requesters are, what type of data requests are coming into the system,” Lloyd shared. “Our initial sets of users were people from the business intelligence team, the analytics team, and the data engineering team.”
Understanding this baseline was crucial for prioritizing where the team needed to start, and lent a metric against which they could measure their success. And with the knowledge that Chargebee’s Business Intelligence, Analytics, and Data Engineering teams would get the most value from Atlan, they got to work familiarizing themselves with the platform and cataloging data sets, creating a minimum viable product for data consumers.
“Once we were comfortable, we onboarded a set of users, that is, we selected users from ops teams from across the organization, and we called them Atlan Champions,” Lloyd shared. Atlan Champions received thorough enablement, like walkthroughs, context on how to find data, and instructions on how to use Atlan. These users would grow to be evangelists for Atlan at Chargebee, not only using the platform to service their own requests, but to invite their colleagues to self-service, too.
As their initial set of users were becoming savvy on Atlan, the data team set their sights on the next cohort of users. “We identified more people across the organization who were tech-savvy and SQL-savvy, people who frequently worked with data, and people who had good hands-on experience on SQL,” Lloyd shared.
With a broad range of users across functions and skill levels beginning to get value from Atlan, Chargebee’s data team had an informed set of colleagues that could provide direction and prioritization as they grew Atlan’s footprint.
“We prepared a questionnaire and we conducted user interviews with all these stakeholders to understand how they use data, what type of data they need, what are the data needs of their team,” Lloyd shared. “Based on this, we tailored a plan to prioritize the onboarding of data sets to Atlan so that it can be consumed immediately.
And to ensure they were on the right track as the solution was scoped, the team scheduled an offsite to analyze their progress. “We wanted to test out how far we moved from the whole data as a service mindset, toward actually building data products,” Lavanya shared. “We said, yes, we want to be building reusable, scalable products. We want to iterate and improve, we want to be trusted by our customers, we want to add value to them, we want to be able to have our customers self-service, we want to enable better data discovery.”
The path forward was clear. Chargebee had the right users, the right problem statements, and the right technology, and were ready to build a single source of truth that was reusable, easily accessible, well-documented, and valuable to a broad set of stakeholders.
Eliminating the Data Request Backlog
The first priority for Chargebee’s data team was to reduce the volume of requests, especially basic questions related to the location of data.
“Where do I land, where do I go, is one of those standard questions people would ask you,” Lavanya shared. “We would give them three different things. You’d say ‘Go to this for Tableau, and go here for something else, and here’s your spreadsheet.’ That was always complicated.”
While these questions may have been basic, the tribal knowledge required to answer them was substantial, and the data and analytics architecture underpinning their operations was complex. “We do our analysis using data from a large number of sources,” Lloyd shared.
Over 20 data sources are consumed at Chargebee, including Salesforce, Hubspot, Gainsight, SAP, and Splunk, which are transformed and loaded through Fivetran into BigQuery by their data engineering team. Downstream, visualization and analytics teams consume this data in Tableau and Google Data Studio for reporting and analysis.
Navigating this data estate, system by system, was an impossible task for most of Chargebee’s data consumers. “We have so much data in our data warehouse,” Lloyd shared. “If you wanted to open access to users, I don’t think they would be able to find what data resides in which table, and they wouldn’t be able to do this on their own. This was where Atlan was a huge help to us.”
The team began by identifying key tables and columns consumed by their users, and consuming them in Atlan. Then, using Atlan’s data cataloging features, they created brief descriptions of each table and a single-line description for all columns within these tables, tagged data owners, and added their metric definitions.
Beyond the value these definitions and owners would represent to data consumers, Chargbee’s data team had long desired to better define their assets, and had finally been able to do so using Atlan. “As a growing startup, one of the challenges which we had was not having proper documentation for all the tables that were available in our warehouse,” Lloyd shared. “At one point, we had a random effort to pull in names of different tables and to write one line descriptions, but this effort did not scale, and the cataloging feature helped us complete long-pending documentation.”
Expanding the scope beyond their data warehouse, Tableau was also connected with Atlan, enabling data consumers to search for dashboards on Atlan, then land in the right resource in Tableau directly.
90% Reduction in Data Request Resolution Time
With this solution, users could now search for relevant metrics, learning directly in Atlan how they are defined and calculated with a sample calculation for the metric, a view of the tables used to calculate it, relevant queries, and the dashboards that display the metric. For the first time, data consumers would understand, at a glance, the nature, relevancy, and consistency of Chargebee’s enterprise data. “It’s a one-stop-shop for anyone who wants to explore data on their own,” explained Lloyd.
And with Tableau integrated, data consumers could now yield more value from existing reports, with Atlan serving as not just a data discovery tool, but a dashboard discovery tool, as well. “Our developers spent huge amounts of time and effort creating so many dashboards, but it was disappointing to see a lot of these dashboards go unused,” Lloyd shared. “After Atlan came into the picture, every single search resulted in at least one dashboard that could be explored for a particular metric.”
We now have this answer where we just point them to Atlan, and they just go there and search for what they want. That organically helped us build out a lot of the literacy around metrics. That’s been super helpful.
Lavanya Gopinath, Senior Director of Culture and Systems
Where a high volume of requests were once processed manually via a Slack channel or a common email distribution, data requests are now serviced with a link to the resource or a saved query on Atlan, driving further adoption and building helpful habits.
The impact of this shift in process and culture has been substantial. With adoption exceeding Chargebee’s expectations, their data team have offloaded 50% more data requests than expected to self-service users. And further, requests are serviced far more quickly than before. While data requests once took 24 to 48 hours, now, when stakeholders self-service on Atlan, time to resolution has dropped by 90%, saving as many as 6 hours per month once spent trying to search for and understand data. “The amount of time it saved for our stakeholders was huge,” Lloyd explained.
And where long wait times for critical data once persisted, the Chargebee team has received zero escalation requests since the Atlan rollout.
There were more people who were able to help stakeholders get data directly from Atlan. ‘Here’s the Atlan link’ is now the standard way of responding to data requests that we receive.
Lloyd Lamington, Business Solutions Manager
Thanks to the hard work of their data team, and the adoption of Atlan, a cultural change is occurring at Chargebee. “Some of the lines that stakeholders have actually told us are ‘All the data I need is there in a saved query.’ or ‘Thank you for bringing in Atlan, I am more data-driven.’ People have become more tech-savvy and SQL-savvy,” Lloyd shared.
Chargebee’s Advice for Data Leaders
Having escaped the data-as-a-service trap, Chargebee’s team has advice to share with their fellow Data & Analytics leaders. “One of the things that helped us greatly was we were able to measure what we wanted to improve, and what problems we wanted to solve using Atlan,” Lloyd shared.
Then, by defining a list of champions that were aligned with their organization’s domains, Chargebee ensured they could find value early, and that they were solving for clearly defined business goals.
Lastly, Chargebee’s data team were humble about their expectations of behavioral change, and recognized that for a large number of stakeholders to stop requesting data from Slack and email, and to move to self-service, would take time and trust.
Summing up her team’s accomplishments, Lavanya concluded, “Irrespective of where we are starting off in the data journey, be very clear about what your next step is. I think that’s all you need to know. If you need to be clear about the metadata in order to progress further, just make sure you’re super clear about the next step and then you can build from there. That’s been our learning. Because without that, if we go to a tool, the tool cannot help us unless we have clarified what our next step structurally needs to be.”
Header photo: Mario Gogh on Unsplash
