The challenge of integrating new data and collection tech with historic data

To further strengthen our commitment to providing industry-leading data technology coverage, VentureBeat is pleased to welcome Andrew Brust and Tony Baer as regular contributors. Keep an eye out for his articles on Data Pipeline.

Whenever new technologies or systems become too valuable or ubiquitous not to be integrated into industry businesses, there are reluctance to cling to old ways or prioritize the familiar over the innovative. Those organizations tend not to last long.

But even among the adopters, there are companies that try to merge the old with the new and fail, while others make it happen. We are seeing this on full display in the areas of esports, where organizations are challenged to integrate legacy data with new collection technologies and data sets. What distinguishes success stories?

When faced with waves of new data due to advances in automation and data collection methods, a sports organization must first acknowledge that it is a good problem. With technology like lidar, for example (a laser-based motion tracking system), which focuses on improving accuracy, depth of information, and fluidity of data collection, performance evaluators now have access to a huge trove of untapped data that can be used to better inform your decisions. The question then is: how does a club manage this influx of new data?

First, preach patience. Consider that organizations and their data teams have been using the same methods and approaches, making the same assumptions and associations, for years. Old habits die hard. And because advanced analytics can be applied to everything from game strategy to the optimal types of refreshments served at stadium concession stands, an organization adopting these technologies for the first time will need widespread acceptance. That takes time.

However, the biggest challenge is integrating an organization’s historical data with modern information. Harvesting technologies and methods are not all that has changed in this area. today’s data appearance very different from the past and, in some cases, the types of measurements do not align with previous data sets. How do an organization’s data teams solve this problem? Start here:

  • Run translation exercises. Reserve a transition period during which a detailed analysis of all data and methods, both modern and historical, is carried out.
  • Accumulate a statistically significant amount of data. Avoid any statistical noise or false positives that might be generated by too small a sample size. You want to get it right the first time.
  • Be aware of biases. Certain biases could occur in system calibration. It is important to identify and correct them to avoid biasing your baselines and future calculations.
  • Please note the differences in data collection methods. Different sports facilities use a variety of tracking technology, some of which have inherent limitations that influence the data collected.
  • Be aware that some translations may be probabilistic in nature. Measure to a constant: In other words, player X runs at a speed of Y, so the new measurement output should be equal to Y.
  • Integrating old and new data can be laborious. Ensuring that old data sets are not lost while the insights that unlock new data are adopted can be costly and time consuming. But it is important to remember after the exercise that an organization will be better positioned to make personnel decisions.

The key for sports organizations to integrate old and new technologies, methodologies, and information is to dig deep into the data. Raw historical data doesn’t help most clubs. New user profiles must easily understand data to be actionable, which takes valuable time and can lose all usefulness in the process.

A schism may exist between datasets that track similar or identical movements using different technologies or approaches. When measuring the force of a kick on the field, for example, data collected from wearable devices attached to a player’s boot may not easily integrate with data collected that measured that same kick using laser-based lidar.

And because wearable technologies are limiting where and how often those measurements can be tracked, there may be gaps in technology feedback due to missing data points. Data smoothing cannot unite this information.

Upgrading to new technology is of course often worth it. For example, lidar, which is more accurate while being more portable and discreet from a player’s point of view than previous technology. The data integration challenge is the only notable downside to adopting LIDAR for a club’s player evaluation department. And with the right plan, even that challenge can be solved.

Raf Keustermans is the CEO of Sportlight Technology.


Welcome to the VentureBeat community!

DataDecisionMakers is where experts, including data techies, can share data-related ideas and innovations.

If you want to read about cutting-edge ideas and up-to-date information, best practices, and the future of data and data technology, join us at DataDecisionMakers.

You might even consider contributing an article of your own!

Read more about DataDecisionMakers

Leave a Comment