As we at Atlan started to use big data (massive data from hundreds of sources), we quickly found that we needed to move everything to the cloud. With data sizes in TBs, we couldn’t keep a local copy of our data for analysis, so we needed to find a way to directly interact with data
We use GitHub issues to keep track of all issues. Please do not report bugs or issues in this blog’s comments. Instead, post them on GitHub as an issue. Before submitting a comment…
In our last post on Apache Airflow, we mentioned how it has taken the data engineering ecosystem by storm. We also talked about how we’ve been using it to move…
Even if you haven’t worked on Kubernetes, chances are you’ve at least heard or read about it. It is already one of the most popular open source projects ever, and…
Landsat is without a doubt one of the best sources of free satellite data today. Managed by NASA and the United States Geological Survey, the Landsat satellites have been capturing…
What is the first thing that comes to your mind upon hearing the word ‘Airflow’? Data engineering, right? For good reason, I suppose. You are likely to find Airflow mentioned…
“A little knowledge that acts is worth infinitely more than much knowledge that is idle.” –Kahlil Gibran Scalability, at the end of the day, has a lot to do with…
The number of villages in India is anywhere between 600,000 and one million, according to various government databases. The number and the definition of villages vary across databases, making it…