An Open Mind can Move Mountains
She was about to start working for a large bank the next month. “Big Data Analyst” she told me. I was very impressed. Big Data, wow! I was always impressed of the way she used to do her job. Starting with those days when we were used to work together. She stayed always cool and friendly even when work was very tough. She had quitted her job already but we still whatsapped. Considering what she was doing before, Big Data was a huge step forward.
October was heading to its end. We decided to meet for lunch once again. We agreed on a restaurant close to the river. You could sit outside at that place. It was a warm day when we met. One of those days at the autumn’s beginning where the sun still keeps alive hustle and bustle out there.
She was already waiting when I arrived. She sat at a table outside with a nice view over the river. There was a big fat camera on the table, close to that a book about big data and analytics.
“Hey!” I said, “What’s up? Sorry for being late!” I continued. “Hey!” she replied, “Don’t worry, all good!” she smiled. “What’s that for?” I pointed to the camera. ”Oh, Photography is one of my hobbies! It’s a good light today!” she explained, sounding like she’s bit proud of her camera.
Big Data with No Idea
After all that small talk stuff people bring up, when they don’t know each other very well but they feel like they have to go through, I asked her about Big Data. She started to tell, the very humble way she always told about herself. “Oh, I have no clear idea, I’ve just started reading this book here.”
And then a remarkable overview of big data technology came through. I learned a lot about Cloudera distributions, Lambda architectures, Elastic Search, NoSQL database
s and the amazing number of different open source initiatives for different purposes. That was what she has always been. A very skilled and open-minded computer scientist with no fear of change. She was going to start that job with a good general skillset in computer science and mathematics and no actual idea of big data. But with an open mindset and the will to learn. To learn fast.
This happened two and a half years ago.
Today she is her manager’s right-hand woman having a deep understanding of tools, technologies, what actually practically can be done and which pains one can expect in big data projects.
Delivery Thinking is made of These
Last Friday we met once again. This time she told me about automatic data loading, type and structure recognition via machine learning and the usage of database statistics for data analytics.
Interesting stuff. I have never thought about it. But it’s true. Data access statistic harvested by RDBMS like oracle might be a good starting point for data analytics, indeed! Data frequently accessed are more likely to be important than the less accessed data.
But she told me about the obstacles of getting access to data together with their metadata and statistics within a certain operational transaction processing database due to organizational barriers between different departments and lack of understanding as well.
Apparently, you need to convince a lot of people and consider things like data privacy and data protection, when you want to get access to data for analysis purposes. Even though it is clear that there will be no way to track back customers or any other persons, you must go through a lot of bureaucracy to get the data. Obviously, there is a long way to go and yet she hasn’t lost her faith, optimism and open mind. Delivery thinking is made of these.
