Big Data and Analytics Misconceptions- Valutrics

IDC released a forecast in the third quarter of 2016 that showed that the big data and analytics market hitting double digit year-on-year growth rates. If this is true, then many of those scary myths still floating around almost certainly must be wrong. Right?

The thing about the best and longest lasting myths, legends and lore is that there is always a nugget of truth that keeps the mistruth going. This is commonly the case with complex technologies that are often over hyped and ultimately become slower than expected to be adopted. Big data is one of those technologies, but it’s not the only one. Other recent examples where negative myths have been formed around technology include software defined WANs (SD-WAN), IT security and even cloud computing. Yet, if the technology is ultimately the right fit for enterprise organizations, myths eventually are overcome and the truth is exposed.

Keep all your data; you might need it one day

One of the biggest myths to come about during the early days of the big data movement is that enterprise organizations should hold onto every scrap of data that could ever be collected. For those who went down that path, they were met with the task of figuring out where to store data at the lowest cost. Many sought out cloud-based data archiving technologies such as Amazon Glacier or Google Coldline Storage. While these technologies are indeed excellent low-cost solutions for data archiving for disaster recovery, it’s not the right place for data analytics. Ultimately, it was discovered that the true benefits of big data come in the real-time analysis and reporting of recently procured information.

Big data analytics are far too expensive

When you start discussing the topic of big data to those who aren’t fully informed, you often come away with the sense that many IT leaders feel that they can’t afford the cost of getting started. This likely came about because big data first became popular with the largest enterprise organizations. Story after story about big data being leveraged in companies such as Facebook, Microsoft and Wal-Mart led many to believe that this was a technology only attainable by the largest of organizations. While this may have in fact been true very early on, cloud-based big data and analytics solutions now allow companies to start small and scale their big data ambitions on an as-needed basis with relatively low start-up costs.

We’re already too far behind the big data curve

If you’re in the camp that thinks your window of opportunity to use big data and analysis reporting to gain a competitive advantage has long since closed, I’ve got some news for you. Big data is going to be part of enterprise IT for a long time. If you think that you’re going to pass on big data and wait for the next big technology wave to ride, you’re going to be waiting for a long time. The truth is, the next innovations in data analysis are going to be nothing more than evolutions of a big data foundation that many already have in place. That being said, there’s still plenty of time to catch up given the advancements in tools and platforms that have cropped up over the last few years. This is especially true in the fields of artificial intelligence and automation that can dramatically reduce the time and effort spent capitalizing on information gleaned from big data analysis.

You need to know what questions to ask prior to starting a big data project

While it’s always a good idea to start any big data project with a general idea of what you’re trying to achieve, it’s not necessary to know what exact questions you are looking to get answered. Often, analytics can help to answer questions or solve problems you never even considered. If you’re too focused on reporting on very specific answers, you may lose critical insight that may end up being more useful that you thought. Even though insights can be bizarre, they also remain very useful.

IT staff can’t easily move into data analytics roles

One of the biggest concerns surrounding big data projects in 2017 is the shortage of data scientists often reported in the media. This news has led many to believe that data analysts are only good if they have a background rooted in data science. In other words, you can’t take some IT pro that’s coding or managing virtual machines and turn them into an analytics expert. While it is true that the learning curve for data analysis is steep, it stands to reason that anyone with a background in enterprise IT (and perhaps an introvert) has a leg up on others seeking to get into the data analytics game.

Analytics results will only confirm what you already know

The old saying “you don’t know what you don’t know” rings true in the world of data analysis. In many cases, business leaders feel they are the ones that intimately know their business and their market. Therefore, there’s very little that can be learned from big data analytics exercises. This is more of a problem with breaking the mold of conventional wisdom in business than a true myth regarding big data. Ultimately, it’s been proven many times over that data-driven organizations gain key business insight that has the potential to provide true competitive advantages.

All analytics platforms are the same

Analytics platforms are all the same. The difference is in what you’re looking for. These statements couldn’t be further from the truth and show a lack of research in the field of big data and analytics. The best way to prove this is to pilot several platforms using the same data and looking for the same answers. You’ll likely be surprised how different the results will be. Use this information to find the right fit for your data and business vertical.

Data lake is just a fancy name for data warehouse

Enterprise data warehouses have been around for a long time. So, when the term data lake began cropping up, many assumed that this was simply a buzzword used to spice up the concept of EDW’s. While data lake does sound like a term concocted in the conference room of a marketing firm, there are distinct differences between EDW and data lakes — largely revolving around structured vs. unstructured data storage methods.

It’s important to be able to separate fact from fiction in this fast-paced world of enterprise IT. Believing in myths can paralyze an IT department into holding off too long on technologies that truly can propel a business forward. That’s why it’s so critical that as a community, we identify and knock down myths in all technologies, including big data.