Driving Self-service and Improving DataOps with Atlan
The Active Metadata Pioneers series features Atlan customers who have recently completed a thorough evaluation of the Active Metadata Management market. Paying forward what you’ve learned to the next data leader is the true spirit of the Atlan community! So they’re here to share their hard-earned perspective on an evolving market, what makes up their modern data stack, innovative use cases for metadata, and more!
In the first interview of this series, we meet Heidi Jones, data analysis and program management extraordinaire, who explains the history of Docker’s data team, how they evaluated the market, and how they’ll use Atlan to help their colleagues drive one of the world’s best developer experiences.
This interview has been edited for brevity and clarity.
Would you mind describing Docker and your data team?
Docker is a platform designed to help developers build, share, and run modern applications. We handle the tedious setup, so developers can focus on the code.
Data professionals at Docker support a variety of different departments. So we have a core data team with engineers and analysts, and then we also have data engineers and analysts that support the major functions of Docker, such as Marketing, Sales, the different products at Docker, Finance, et cetera.
Several experienced data engineers and analysts who have joined Docker, have only started in the last nine months or so. So we’ve had quite a bit of growth on the data team, and are now at that stage where we’re trying to invest in good processes. That way, our data team can ensure that everyone at Docker has the data that they need to do their jobs, and can ultimately help developers do theirs.
And how about you? Could you tell us a bit about yourself, your background, and what drew you to Data & Analytics?
I think the main reason I’ve been drawn to data and analytics is because I just like being able to answer people’s questions.
I came into data analysis through a non-traditional route. I’ve been at Docker for about six months now, but I’ve been in the data space for about a decade. It started with Excel and providing insights via spreadsheets, up to PowerBI using Snowflake, that type of thing.
So I was always a data analyst, but then also a project manager. And so what I do at Docker combines both of those. The knowledge of data and the workflows required to get good data and provide good insights, and also the project management and operations side of it. It all allows data professionals to focus on what they do best, which is modeling data and providing insights without being blocked by anything that has to do with workflow.
What does your stack look like? Why did you need an active metadata solution?
We ingest data from a variety of sources in several different ways, depending on the source. And then our data warehouse level is Snowflake. Our modeling layer is dbt, that’s where we do modeling and transformation. And then our main BI tool is Looker, that’s where we do visualization and analysis.
We were just a one-person team not too long ago. So all of that data work was on one person’s plate, including documentation and understanding data sources. That is quite a bit for one person.
A lot of that burden has been spread out across several people on the team by now. But we’re trying to move away from, “Oh, let me go ask my favorite data person,” toward, “I can go check this tool and I know there’s a certified data asset.”
And so, because of our stack, we were drawn to Atlan because of things like the Looker Chrome extension plugin, the dbt integration, that type of thing. Because right off the bat we were able to say, “Okay, any descriptions we put in our dbt layer will automatically be exposed in Atlan.”
So non-engineering users who want to know what the data means can go straight to Atlan and see what is being done in the modeling layer.
Did anything stand out to you about Atlan during your evaluation process?
Atlan is a very cool tool that has a good suite of features that we were looking for, but the differentiator honestly came down to the people at Atlan.
You demonstrated very competent understanding of the problems in the data space and also very mature customer support. We could tell that your support was not just something you were promising for us, but something that you were already actively doing with other customers.
We knew that it would be a real partnership and that the customer support org was prepared to support the needs of an organization like ours. And that maturity stood out to us when we made our decision.
But then again, also the features like Playbooks, the integrations that I’ve already mentioned with dbt, with Looker, and just the constant innovation as well that we were able to observe even during the evaluation processes, which I believe took us about two months.
There were several innovations and releases that happened during that time period and we could see the cadence the Atlan was on to continuously improve. All of those were selling points to us.
What do you intend on creating with Atlan? Do you have an idea of what use cases you’ll build, and the value you’ll drive?
Our biggest value that we’re trying to drive with Atlan is to make sure that professionals at Docker can get the information they need about the data that they need to do their jobs.
We want to move towards self-serve analytics and allow both data professionals, and those who just want to be able to use data more freely in their work, to be able to do so without having to get into all of the SQL and technical details of the data.
They know they can trust the data set, they know they can trust the data that they’re looking at, and they can go ahead and make their decisions. Ultimately, it should help us support our mission of delighting developers, and developing tools that they enjoy using.
We’ll be supporting that with Atlan, and also supporting our data engineering and analytics teams. They need to have more supported and standardized workflows, so that they can focus on modeling, really digging in and doing what they do best with data.
Did we miss anything?
That’s a good question. I think how we discovered Atlan was interesting. I’ve been following Prukalpa, actually, for a couple of years just as a data professional, just kind of watching Atlan.
And so when I joined Docker, they were already looking at data catalog tools, but hadn’t been looking at Atlan yet. And I said, “Well, how about Atlan? Should we look at Atlan as well?”
So one of the first things I did at Docker was to start up that conversation, and the reason why I did that is because I had appreciated reading what she said in those spaces. About the reasons we need data catalog tools, and beyond just a catalog, how it could be part of data operations. And that piece of it really had spoken to me over time.
And we saw some impressive tools. It’s a burgeoning space. There’s some great tools out there. But I’m glad that we also looked at Atlan because ultimately it had a good combination of what we needed at Docker.
Photo by Annie Spratt on Unsplash
