Content Management Technology Leader Reducing 6-hour-per-week Data Discovery Effort with Atlan

Valued at $3 billion, Contentful is a Series F technology company that has revolutionized content management systems, with a headless architecture that uses APIs to power great digital experiences for their 5,000+ customers.

As Contentful continues to grow at incredible pace across continents, industries, and use cases, data has played a crucial role in enabling smart decision-making. And standing behind that data to ensure it’s trustworthy, available, and activated is Robert Clifford-Jones, a 12-year veteran of the Data & Analytics space, and Contentful’s Head of Data Intelligence and Governance.

“Contentful is a scaling company, and we are taking market share and want to continue to do that. The next part is around how we transform from what we are today, which is a single product, into a multiple-product platform that is supported by a marketplace,” Robert explained. “Then, what we’re doing now is making sure that we’re maturing our processes and can scale even faster.”

Improving Context and Discoverability with Active Metadata Management

Contentful’s 20-person data team is spread across Data Science, Analytical Engineering, Data Analysis, and Data Engineering. Being a fairly new organization, their technology is unburdened by legacy environments, centered around Redshift as their data warehouse, fed by Stitch and Segment, and using dbt to build data models that are pushed to Redash and Looker for end-user access. And to ensure Contentful’s data consumers have the right data to make informed decisions, Robert and his team began searching for an Active Metadata Management solution.

“When I first started at Contentful, I could see the data team was a really valuable resource for Contentful. We had a very busy Slack channel, and a lot of the questions we were spending a lot of time on were ‘What data do we have?’ or ‘Can we use this data for this purpose?’,” Robert shared. “That led me to think that people clearly need some contextual information, and they need to know what’s available.”

Beyond the need for data context and discoverability, Robert’s team sought a solution that could advance their data governance processes such as access policies, and improve trust in data as Contentful was scaling. Launching an RFP process, Contentful thoroughly evaluated Atlan and Alation, eventually choosing Atlan.

“What really stood out to me is as we were stepping through the process, we had a lot of great connections that were built with the team. The documentation was second-to-none, and the training is provided without having to ask for it. When we were using Atlan in the proof of value stage, and we were comparing it to the competition, we just felt the UI was really user-friendly and it was going to be easy to get adoption from our business users,” Robert shared.

Eliminating a 6-hour Data Discovery Tax

Robert and his team, three months into their use of Atlan, have elected for a deliberate, step-by-step rollout, ensuring that users are onboarded appropriately, and fully understand how to use Atlan before rolling out new use cases.

Beginning by enabling Search for end users, Robert and his team integrated Atlan with Contentful’s most important applications, ensuring their highest priority data was visible and searchable, was enriched with descriptions, and was documented. Easing end users into using Atlan, Robert and his team began by responding to Slack requests about data with links to data assets within Atlan.

“We used to answer these questions and post a link, and now we’ve said ‘Were not answering these questions anymore. Go to Atlan, that’s where your source of truth is, and that’s where we want you to start self-serving’,” Robert explained.

With their critical data assets accessible, enriched, and available via self-service, Robert and his team’s next step was to enable Atlan Insights, a metadata-based query builder, in response to their previous querying tool, Redash, reaching end-of-life.

“Our hand was forced with Redash coming to the end of its life, so we had to very quickly create a journey for onboarding people into the tool to see that contextual information and to build their queries and do ad-hoc analysis as they wanted to.” Robert shared.

A critical tool for Contentful, Redash’s end-of-life meant that without a replacement, queries and ad-hoc analysis would be difficult-to-impossible for a large swath of data consumers. And while Atlan Insights was a promising solution, migrating meant re-creating queries in Atlan that already existed in Redash. With this problem statement in mind, Robert approached his Atlan Customer Success Manager, who worked with the team to build an integration that automatically migrated these queries to Atlan in mere weeks, saving extraordinary manual effort.

“We had to migrate all our queries over from Redash, and we didn’t want to have to do that manually, but we were going to have to,” Robert explained. “We mentioned this to the customer service team and within a couple of weeks, we were told ‘We’re building a connector. If you give us seven days, we’ll have something sorted for you so you don’t have to do all that manual effort.’ It’s saved us so much time.”

Yielding even more time savings for Contentful’s data team is automated lineage, significantly easing impact analysis as data engineers consider the downstream effects of changes and enhancements, and accelerating root cause analysis when breakages occur.

“It’s just building trust with our stakeholders because things aren’t failing. Through Atlan, we’re able to communicate these changes a lot better,” Robert shared.

Finally, Robert and his team are moving to enhance their catalog to drive a common language at Contentful, appointing subject matter experts to enrich glossaries, then own and maintain them going forward.

“Everyone has their own interpretation of what something means so we’re now building that common language within Contentful. It will be stored in Atlan and we’ll drive people to it.”

In just three months, Contentful’s data team has enhanced their ability to answer questions about their data, migrated to a new querying tool, and is reducing breakages, allowing them to focus on more important work. But for data consumers, Atlan represents a leap forward in productivity, offering self-service for data discovery that once cost each of them six hours per week.

Before, we’d be spending a lot of time fielding these questions. We did a user survey, and it was taking maybe six hours of people’s time per week to find the right information, to understand the information, and then to apply that to what it was they were building. With Atlan, we’ve managed to reduce that significantly just by having the data available and having it documented. So we’ve got value for our end users, because they have the information in their hands. But for us, it’s allowing us to focus on what’s really important to Contentful.”

Robert Clifford-Jones, Head of Data Intelligence and Governance

The Dreaded 3:00 a.m. Phone Call

The strongest illustration of what the data team’s hard work has yielded for Contentful arrived in familiar form. A 3 a.m. phone call to Robert informing him that critical models driving financial reporting had failed.

“Our models were running, they run batch processing through the night and everything failed. It was the end of the month, so we couldn’t do all our closing reports and check all of our financial reporting on Monday, as was expected,” Robert shared. “I got a phone call at three o’clock in the morning going ‘What’s going on? Where’s this problem? Can you fix it?’”

Before adopting Atlan, breakages meant manually going system by system, spending hours pinpointing where the failure was, and more hours, still, resolving the issue. But with automated lineage, Robert and his team were able to resolve the issue within hours, allowing the crucial financial reports to run, as expected.

The Atlan lineage module allowed us to go in and see where the failure was very quickly. So it meant we could resolve the issue before reporting kicked off on Monday, whereas before, they would have had to wait for that fix.”

Robert Clifford-Jones, Head of Data Intelligence and Governance

Full Speed Ahead on Data Governance

The next frontier for Robert and the Contentful team focuses on data governance. The team is hard at work tagging personally identifiable information in Atlan in order to apply new masking and access management policies to further enhance the security of sensitive financial data. And with upcoming enhancements like Atlan AI, Robert and his team are optimistic that Atlan can help them focus on what matters.

“The way I’m thinking about this is how do I free up my team to do the things that are important to them and the things they’re passionate about. A lot of the time is spent doing the things that are slightly mundane that could be done better by AI,” Robert explained. “That’s something we definitely want to do with Atlan going forward. Given the new product releases coming up, we could use Atlan more in that space to do the documentation that takes up a lot of our time, and free our people up to do the exciting stuff.”

Having driven Contentful’s data team to new levels of productivity, and unlocking a new level of understanding and access for data consumers, Robert remains confident that Atlan will be with he and his team every step of the way as they continue to grow.

I’ve adopted many a technology and I’ve had good support in the past. I’ve never had as good support from an organization as I’ve had from Atlan. I ask a question, and it’s answered almost immediately. I don’t expect that, but that’s what happens. If we raise a ticket or we spot something that’s wrong, the team are on it pretty much within the hour. The support has been second-to-none.”

Robert Clifford-Jones, Head of Data Intelligence and Governance

Photo by Kelly Sikkema on Unsplash