Once Upon a Time in Data Land

Once upon a time, more and more organizations (enterprises, governments, NGOs) had more and more data from more and more sources (internal and external).

They knew/felt that it must be valuable if only they could work out how to extract value – find the relevant data, integrate it reliably, turn it into insight … and automate all of that!

However, many were struggling …

1. Some were overwhelmed by all the different data.
   • What do we even have?
   • What can you do with data?
2. Some struggled to figure out the link between the data and concrete business needs challenges and goals.
   • Do we even have relevant data?
   • Can we use it cost effectively?
3. Some started to extract value by doing manual data extraction and analysis (emailing excel files, asking colleagues for data, diving into intranet), but it is slow, error prone and not scalable
   • Why am I only getting performance numbers two months late?
   • Why it taking Joe 3 days a month just to wrangle the excel?
4. Some started building semi-automated pipelines and hiring data scientists and engineers. This is an improvement but is expensive and often leaves them dependent on ad-hoc solutions, key staff, and buggy results.
   • Where does that number in the dashboard come from?
   • What happens if X leaves, they are the only one who knows how that pipeline works?
5. Some decided to buy expensive solutions from proprietary vendors whether for BI, ETL, or data governance but the results are often disappointing as data is diverse and messy and one size fits all is a poor fit plus they are now locked in to one more vendor
   • What happens if our needs change next year?
   • What happens if a great new tool becomes available and we want to use it too?

And then one day … they discovered Datopian who had thought about these problems for years and had a whole open source architecture and framework ready to go plus lots of expert data engineers to help implement and adapt it …

And they worked with Datopian and generated millions of dollars in value from unlocking the potential of their data … and they were happy ever after 😄