GreatVines’ New Data Infrastructure Technology

The speed of technology advancement in the data world has been staggering in the last several years. Just as your smartphone can become outdated in a handful of years, the cutting-edge in data processing can quickly become last year’s flip phone. Recognizing this, we have begun a project to update our data infrastructure to leverage several new technologies that have arisen in the last several years.

The current GreatVines data architecture was developed 8 years ago, when cloud computing was still in its infancy. It leveraged several technologies that were very new at the time, and provided a way to easily scale our processing as the company grew without requiring a large on-premise physical presence. However, it still required a lot of maintenance and administration, and the development of data “pathways” was still very involved and complex.

In the last several years, new technologies have come to market directly targeting these areas, such as cloud-based “Data Lake” architectures and Integration Platform-as-a-Service or “iPaas”. Late last year we decided that the technology was mature enough to embark on a project to update our own data architecture and integration systems.

The new data infrastructure will allow us to decrease overnight processing time, reduce maintenance, improve stability, increase the speed we can provide new functionality, and allow us to be more flexible with client customizations.

Over the next several months, we’ll begin rolling out the first of these improvements which is focused on our internal GreatVines to GoodData processes. Throughout next year, we’ll continue to release new functionality focused first on our 3rd party data integrations, followed by upgrades to our integration with client ERP systems.

It’s an exciting time to be working on the GreatVines data team, and I look forward to providing more information on our progress and the improvements you’ll be able to see in the coming months.